[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:22:12.539637 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:36683
I20260709 12:22:12.541460 24862 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:22:12.542651 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:12.562908 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:12.563040 24869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:12.564723 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:13.799683 24870 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 12:22:13.799801 24862 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:22:13.804280 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:13.804550 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:13.804718 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599733804699 us; error 0 us; skew 500 ppm
I20260709 12:22:13.805567 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:13.812879 24862 webserver.cc:533] Webserver started at http://127.24.71.190:37149/ using document root <none> and password file <none>
I20260709 12:22:13.814215 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:13.814548 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:13.815042 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:13.821233 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "206f892206f242f388d6074d5412884d"
format_stamp: "Formatted at 2026-07-09 12:22:13 on dist-test-slave-3zgk"
I20260709 12:22:13.829358 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:22:13.835248 24877 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:13.836583 24862 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:22:13.836961 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "206f892206f242f388d6074d5412884d"
format_stamp: "Formatted at 2026-07-09 12:22:13 on dist-test-slave-3zgk"
I20260709 12:22:13.837316 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:13.901135 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:13.902719 24862 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:22:13.903201 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:13.979002 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:36683
I20260709 12:22:13.979128 24928 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:36683 every 8 connection(s)
I20260709 12:22:13.987828 24929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.011050 24929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d: Bootstrap starting.
I20260709 12:22:14.017201 24929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.019173 24929 log.cc:826] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d: Log is configured to *not* fsync() on all Append() calls
I20260709 12:22:14.024114 24929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d: No bootstrap required, opened a new log
I20260709 12:22:14.047907 24929 raft_consensus.cc:359] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206f892206f242f388d6074d5412884d" member_type: VOTER }
I20260709 12:22:14.048573 24929 raft_consensus.cc:385] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.048785 24929 raft_consensus.cc:740] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 206f892206f242f388d6074d5412884d, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.049643 24929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206f892206f242f388d6074d5412884d" member_type: VOTER }
I20260709 12:22:14.050137 24929 raft_consensus.cc:399] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:14.050382 24929 raft_consensus.cc:493] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:14.050679 24929 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:14.054906 24929 raft_consensus.cc:515] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206f892206f242f388d6074d5412884d" member_type: VOTER }
I20260709 12:22:14.055613 24929 leader_election.cc:304] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 206f892206f242f388d6074d5412884d; no voters: 
I20260709 12:22:14.057474 24929 leader_election.cc:290] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:14.057966 24932 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:14.060518 24932 raft_consensus.cc:697] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 1 LEADER]: Becoming Leader. State: Replica: 206f892206f242f388d6074d5412884d, State: Running, Role: LEADER
I20260709 12:22:14.061355 24932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206f892206f242f388d6074d5412884d" member_type: VOTER }
I20260709 12:22:14.062002 24929 sys_catalog.cc:565] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:14.074353 24933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "206f892206f242f388d6074d5412884d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206f892206f242f388d6074d5412884d" member_type: VOTER } }
I20260709 12:22:14.075212 24933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:14.077379 24934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 206f892206f242f388d6074d5412884d. Latest consensus state: current_term: 1 leader_uuid: "206f892206f242f388d6074d5412884d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206f892206f242f388d6074d5412884d" member_type: VOTER } }
I20260709 12:22:14.078207 24941 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:14.078244 24934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:14.090395 24941 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:14.094558 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:14.110167 24941 catalog_manager.cc:1383] Generated new cluster ID: fd6d4b905f764fe98fcad48324fd30de
I20260709 12:22:14.110519 24941 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:14.129850 24941 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:14.132337 24941 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:14.150759 24941 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d: Generated new TSK 0
I20260709 12:22:14.151896 24941 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:14.162147 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.172492 24951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.174957 24952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.178253 24954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.178985 24862 server_base.cc:1061] running on GCE node
I20260709 12:22:14.180044 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:14.180320 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:14.180553 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599734180534 us; error 0 us; skew 500 ppm
I20260709 12:22:14.181193 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:14.186087 24862 webserver.cc:533] Webserver started at http://127.24.71.129:35639/ using document root <none> and password file <none>
I20260709 12:22:14.186731 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:14.186978 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:14.187319 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:14.188683 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "d8361c2ca34342dd8440184d607f222f"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-3zgk"
I20260709 12:22:14.195238 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:14.200526 24959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.201524 24862 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:22:14.201861 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "d8361c2ca34342dd8440184d607f222f"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-3zgk"
I20260709 12:22:14.202191 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:14.226073 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:14.227706 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:14.230885 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:14.236111 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:14.236434 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:22:14.236757 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:14.236967 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.359805 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:34381
I20260709 12:22:14.359896 25021 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:34381 every 8 connection(s)
I20260709 12:22:14.368541 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.380002 25026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.383363 25027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.391959 25029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.394099 24862 server_base.cc:1061] running on GCE node
I20260709 12:22:14.395150 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:14.395450 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:14.395694 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599734395673 us; error 0 us; skew 500 ppm
I20260709 12:22:14.396605 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:14.396991 25022 heartbeater.cc:344] Connected to a master server at 127.24.71.190:36683
I20260709 12:22:14.397455 25022 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:14.398872 25022 heartbeater.cc:507] Master 127.24.71.190:36683 requested a full tablet report, sending...
I20260709 12:22:14.400094 24862 webserver.cc:533] Webserver started at http://127.24.71.130:41435/ using document root <none> and password file <none>
I20260709 12:22:14.400833 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:14.401145 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:14.401532 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:14.403287 24894 ts_manager.cc:194] Registered new tserver with Master: d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381)
I20260709 12:22:14.403326 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "6c7ea858326b44d384343e348af19c8a"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-3zgk"
I20260709 12:22:14.407053 24894 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55766
I20260709 12:22:14.411911 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:22:14.417933 25034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.418946 24862 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:14.419329 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "6c7ea858326b44d384343e348af19c8a"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-3zgk"
I20260709 12:22:14.419691 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:14.442757 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:14.444331 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:14.447281 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:14.451797 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:14.452059 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.452346 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:14.452544 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.564432 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:46763
I20260709 12:22:14.564793 25096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:46763 every 8 connection(s)
I20260709 12:22:14.574249 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.581197 25100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.589239 24862 server_base.cc:1061] running on GCE node
I20260709 12:22:14.589653 25097 heartbeater.cc:344] Connected to a master server at 127.24.71.190:36683
I20260709 12:22:14.590106 25097 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:14.591158 25097 heartbeater.cc:507] Master 127.24.71.190:36683 requested a full tablet report, sending...
W20260709 12:22:14.591297 25103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.591708 25101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.593152 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:14.593446 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:14.593474 24894 ts_manager.cc:194] Registered new tserver with Master: 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:14.593750 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599734593731 us; error 0 us; skew 500 ppm
I20260709 12:22:14.594648 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:14.595171 24894 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55778
I20260709 12:22:14.597900 24862 webserver.cc:533] Webserver started at http://127.24.71.131:36183/ using document root <none> and password file <none>
I20260709 12:22:14.598610 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:14.598872 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:14.599225 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:14.600805 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "e008c2a93c144707a9b0a021fa99d1f8"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-3zgk"
I20260709 12:22:14.607271 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:14.611771 25108 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.612788 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:14.613173 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "e008c2a93c144707a9b0a021fa99d1f8"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-3zgk"
I20260709 12:22:14.613595 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:14.632877 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:14.634882 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:14.637427 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:14.641492 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:14.641798 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.642093 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:14.642289 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.733847 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:43883
I20260709 12:22:14.733939 25170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:43883 every 8 connection(s)
I20260709 12:22:14.752612 25171 heartbeater.cc:344] Connected to a master server at 127.24.71.190:36683
I20260709 12:22:14.753053 25171 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:14.753988 25171 heartbeater.cc:507] Master 127.24.71.190:36683 requested a full tablet report, sending...
I20260709 12:22:14.756489 24894 ts_manager.cc:194] Registered new tserver with Master: e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:14.757184 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016126791s
I20260709 12:22:14.758224 24894 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55788
I20260709 12:22:14.821409 24894 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55802:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:14.826630 24894 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:14.946368 25062 tablet_service.cc:1511] Processing CreateTablet for tablet 981240fbed2c4a16a6e4c90e85a90277 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:14.947142 25136 tablet_service.cc:1511] Processing CreateTablet for tablet 981240fbed2c4a16a6e4c90e85a90277 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:14.948216 25062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981240fbed2c4a16a6e4c90e85a90277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.948714 25136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981240fbed2c4a16a6e4c90e85a90277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.949314 25058 tablet_service.cc:1511] Processing CreateTablet for tablet 4e119a49418946f79833fea8659bab66 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:14.950688 25058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e119a49418946f79833fea8659bab66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.953720 25133 tablet_service.cc:1511] Processing CreateTablet for tablet e8cbec0e858a4b82b2420d2a6c4c866d (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:14.955309 25133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8cbec0e858a4b82b2420d2a6c4c866d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.955231 25059 tablet_service.cc:1511] Processing CreateTablet for tablet e8cbec0e858a4b82b2420d2a6c4c866d (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:14.956800 25059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8cbec0e858a4b82b2420d2a6c4c866d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.947129 25060 tablet_service.cc:1511] Processing CreateTablet for tablet 99129cacd1a34d588a2218095f047a78 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:14.958508 25060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99129cacd1a34d588a2218095f047a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.951997 25134 tablet_service.cc:1511] Processing CreateTablet for tablet 99129cacd1a34d588a2218095f047a78 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:14.961658 25134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99129cacd1a34d588a2218095f047a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.961730 25135 tablet_service.cc:1511] Processing CreateTablet for tablet 31511785bceb4dae889ed5a85e3cc9fd (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:14.953713 25061 tablet_service.cc:1511] Processing CreateTablet for tablet 31511785bceb4dae889ed5a85e3cc9fd (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:14.965902 25061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31511785bceb4dae889ed5a85e3cc9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.966670 25135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31511785bceb4dae889ed5a85e3cc9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.982372 24984 tablet_service.cc:1511] Processing CreateTablet for tablet e8cbec0e858a4b82b2420d2a6c4c866d (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:14.974661 24986 tablet_service.cc:1511] Processing CreateTablet for tablet 31511785bceb4dae889ed5a85e3cc9fd (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:14.986786 24984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8cbec0e858a4b82b2420d2a6c4c866d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.976698 24987 tablet_service.cc:1511] Processing CreateTablet for tablet 981240fbed2c4a16a6e4c90e85a90277 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:14.988112 24983 tablet_service.cc:1511] Processing CreateTablet for tablet 4e119a49418946f79833fea8659bab66 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:14.991904 24985 tablet_service.cc:1511] Processing CreateTablet for tablet 99129cacd1a34d588a2218095f047a78 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:14.997676 24986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31511785bceb4dae889ed5a85e3cc9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.998323 25132 tablet_service.cc:1511] Processing CreateTablet for tablet 4e119a49418946f79833fea8659bab66 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:15.000965 24985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99129cacd1a34d588a2218095f047a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.004659 25132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e119a49418946f79833fea8659bab66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.006227 24983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e119a49418946f79833fea8659bab66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.007690 24987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981240fbed2c4a16a6e4c90e85a90277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.006273 24982 tablet_service.cc:1511] Processing CreateTablet for tablet f51cac45d9664bfa834f293c39e73ebf (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:15.010574 24982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51cac45d9664bfa834f293c39e73ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.015189 25057 tablet_service.cc:1511] Processing CreateTablet for tablet f51cac45d9664bfa834f293c39e73ebf (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:15.024931 25131 tablet_service.cc:1511] Processing CreateTablet for tablet f51cac45d9664bfa834f293c39e73ebf (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:15.028172 25131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51cac45d9664bfa834f293c39e73ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.031874 25057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51cac45d9664bfa834f293c39e73ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.054999 24981 tablet_service.cc:1511] Processing CreateTablet for tablet 642b30d2b44b4bb098a6a9a5a2bdeab2 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:15.060465 24981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642b30d2b44b4bb098a6a9a5a2bdeab2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:15.092564 24942 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:15.092458 25056 tablet_service.cc:1511] Processing CreateTablet for tablet 642b30d2b44b4bb098a6a9a5a2bdeab2 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:15.097698 25056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642b30d2b44b4bb098a6a9a5a2bdeab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.116173 25130 tablet_service.cc:1511] Processing CreateTablet for tablet 642b30d2b44b4bb098a6a9a5a2bdeab2 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:15.126040 25130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642b30d2b44b4bb098a6a9a5a2bdeab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.161670 25190 tablet_bootstrap.cc:492] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.211480 25061 tablet_service.cc:1511] Processing CreateTablet for tablet 3a5519e8cae74e6691ceb6183d919e1d (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:15.215092 25191 tablet_bootstrap.cc:492] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.214221 25131 tablet_service.cc:1511] Processing CreateTablet for tablet 3a5519e8cae74e6691ceb6183d919e1d (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:15.216370 24984 tablet_service.cc:1511] Processing CreateTablet for tablet 3a5519e8cae74e6691ceb6183d919e1d (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:15.215651 25061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a5519e8cae74e6691ceb6183d919e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.217778 24984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a5519e8cae74e6691ceb6183d919e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.218044 25189 tablet_bootstrap.cc:492] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.216861 25131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a5519e8cae74e6691ceb6183d919e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.227967 25190 tablet_bootstrap.cc:654] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.228458 24981 tablet_service.cc:1511] Processing CreateTablet for tablet 6ba09945d8a941978c714a7f4cc9b7a8 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:15.236337 25057 tablet_service.cc:1511] Processing CreateTablet for tablet 6ba09945d8a941978c714a7f4cc9b7a8 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:15.237628 25057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ba09945d8a941978c714a7f4cc9b7a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.237943 25130 tablet_service.cc:1511] Processing CreateTablet for tablet 6ba09945d8a941978c714a7f4cc9b7a8 (DEFAULT_TABLE table=test-workload [id=0a4e3cace97b477fa5d3074083ba2eff]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:15.239178 25130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ba09945d8a941978c714a7f4cc9b7a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.244469 24981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ba09945d8a941978c714a7f4cc9b7a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.245882 25189 tablet_bootstrap.cc:654] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.262290 25189 tablet_bootstrap.cc:492] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.272410 25189 ts_tablet_manager.cc:1403] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.005s
I20260709 12:22:15.274993 25191 tablet_bootstrap.cc:654] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.274941 25190 tablet_bootstrap.cc:492] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.275646 25190 ts_tablet_manager.cc:1403] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.011s
I20260709 12:22:15.275803 25189 raft_consensus.cc:359] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.276727 25189 raft_consensus.cc:385] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.277050 25189 raft_consensus.cc:740] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.277819 25189 consensus_queue.cc:260] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.279032 25190 raft_consensus.cc:359] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.280411 25190 raft_consensus.cc:385] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.280862 25190 raft_consensus.cc:740] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.282080 25190 consensus_queue.cc:260] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.284348 25189 ts_tablet_manager.cc:1434] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 12:22:15.285566 25189 tablet_bootstrap.cc:492] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.285912 25022 heartbeater.cc:499] Master 127.24.71.190:36683 was elected leader, sending a full tablet report...
I20260709 12:22:15.293928 25191 tablet_bootstrap.cc:492] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.294560 25191 ts_tablet_manager.cc:1403] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.008s
I20260709 12:22:15.298506 25191 raft_consensus.cc:359] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.299474 25191 raft_consensus.cc:385] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.299860 25191 raft_consensus.cc:740] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.301023 25191 consensus_queue.cc:260] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.309557 25189 tablet_bootstrap.cc:654] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.310504 25191 ts_tablet_manager.cc:1434] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.013s
I20260709 12:22:15.310776 25097 heartbeater.cc:499] Master 127.24.71.190:36683 was elected leader, sending a full tablet report...
I20260709 12:22:15.311439 25191 tablet_bootstrap.cc:492] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.319180 25191 tablet_bootstrap.cc:654] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:15.323463 25098 tablet.cc:2406] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:15.335896 25191 tablet_bootstrap.cc:492] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.336735 25191 ts_tablet_manager.cc:1403] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 12:22:15.337423 25171 heartbeater.cc:499] Master 127.24.71.190:36683 was elected leader, sending a full tablet report...
I20260709 12:22:15.338023 25190 ts_tablet_manager.cc:1434] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.062s	user 0.011s	sys 0.039s
I20260709 12:22:15.339005 25190 tablet_bootstrap.cc:492] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.339989 25191 raft_consensus.cc:359] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.340847 25191 raft_consensus.cc:385] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.341212 25191 raft_consensus.cc:740] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.342307 25191 consensus_queue.cc:260] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.344971 25191 ts_tablet_manager.cc:1434] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:22:15.345944 25190 tablet_bootstrap.cc:654] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.346135 25191 tablet_bootstrap.cc:492] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.350802 25189 tablet_bootstrap.cc:492] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.355337 25189 ts_tablet_manager.cc:1403] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.004s
I20260709 12:22:15.356540 25191 tablet_bootstrap.cc:654] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.358829 25189 raft_consensus.cc:359] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.359705 25189 raft_consensus.cc:385] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.360028 25189 raft_consensus.cc:740] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.362422 25189 consensus_queue.cc:260] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.369020 25189 ts_tablet_manager.cc:1434] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:22:15.370148 25189 tablet_bootstrap.cc:492] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.371716 25190 tablet_bootstrap.cc:492] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.372814 25190 ts_tablet_manager.cc:1403] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.012s
I20260709 12:22:15.376271 25190 raft_consensus.cc:359] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.377265 25190 raft_consensus.cc:385] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.377643 25190 raft_consensus.cc:740] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.378698 25190 consensus_queue.cc:260] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.381456 25191 tablet_bootstrap.cc:492] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.381480 25190 ts_tablet_manager.cc:1434] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:15.382246 25191 ts_tablet_manager.cc:1403] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.005s
I20260709 12:22:15.382817 25190 tablet_bootstrap.cc:492] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.386966 25191 raft_consensus.cc:359] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.388022 25191 raft_consensus.cc:385] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.388438 25191 raft_consensus.cc:740] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.388871 25189 tablet_bootstrap.cc:654] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.389448 25191 consensus_queue.cc:260] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.391968 25191 ts_tablet_manager.cc:1434] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:22:15.392987 25191 tablet_bootstrap.cc:492] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.395076 25190 tablet_bootstrap.cc:654] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.399330 25197 raft_consensus.cc:493] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.399863 25197 raft_consensus.cc:515] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.405282 25197 leader_election.cc:290] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.412014 25189 tablet_bootstrap.cc:492] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.412521 25189 ts_tablet_manager.cc:1403] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.011s
I20260709 12:22:15.413002 25191 tablet_bootstrap.cc:654] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.417608 25189 raft_consensus.cc:359] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.418421 25189 raft_consensus.cc:385] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.418741 25189 raft_consensus.cc:740] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.419526 25189 consensus_queue.cc:260] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.431902 25189 ts_tablet_manager.cc:1434] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:22:15.431680 24997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.432652 24997 raft_consensus.cc:2468] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 0.
I20260709 12:22:15.433008 25189 tablet_bootstrap.cc:492] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.434510 25037 leader_election.cc:304] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.435449 25197 raft_consensus.cc:2804] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.435812 25197 raft_consensus.cc:493] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.436182 25197 raft_consensus.cc:3060] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.440207 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:15.441195 25189 tablet_bootstrap.cc:654] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:15.442458 25035 leader_election.cc:343] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.447119 25191 tablet_bootstrap.cc:492] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.447644 25191 ts_tablet_manager.cc:1403] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.011s
I20260709 12:22:15.447526 25197 raft_consensus.cc:515] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.448330 25189 tablet_bootstrap.cc:492] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.448787 25189 ts_tablet_manager.cc:1403] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 12:22:15.450639 25191 raft_consensus.cc:359] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.451376 25191 raft_consensus.cc:385] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.451790 25191 raft_consensus.cc:740] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.451598 25189 raft_consensus.cc:359] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.452419 25189 raft_consensus.cc:385] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.451824 24997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:15.452742 25189 raft_consensus.cc:740] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.452692 25191 consensus_queue.cc:260] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.453399 24997 raft_consensus.cc:3060] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.453722 25189 consensus_queue.cc:260] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.456310 25189 ts_tablet_manager.cc:1434] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:15.457278 25189 tablet_bootstrap.cc:492] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.464207 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
W20260709 12:22:15.465975 25035 leader_election.cc:343] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.466434 25190 tablet_bootstrap.cc:492] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.466729 25197 leader_election.cc:290] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.466902 25190 ts_tablet_manager.cc:1403] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.084s	user 0.008s	sys 0.020s
I20260709 12:22:15.468511 24997 raft_consensus.cc:2468] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 1.
I20260709 12:22:15.470067 25190 raft_consensus.cc:359] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.470211 25191 ts_tablet_manager.cc:1434] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.009s
I20260709 12:22:15.470820 25190 raft_consensus.cc:385] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.471143 25190 raft_consensus.cc:740] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.471763 25191 tablet_bootstrap.cc:492] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.471837 25190 consensus_queue.cc:260] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.464491 25189 tablet_bootstrap.cc:654] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.476823 25195 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.477351 25195 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.477854 25037 leader_election.cc:304] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:15.480041 25195 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.482080 25190 ts_tablet_manager.cc:1434] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:22:15.483566 25190 tablet_bootstrap.cc:492] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.487712 25191 tablet_bootstrap.cc:654] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.498198 25190 tablet_bootstrap.cc:654] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.501583 25208 raft_consensus.cc:2804] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.502571 25208 raft_consensus.cc:697] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:15.503701 25208 consensus_queue.cc:237] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.520226 25190 tablet_bootstrap.cc:492] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.524606 25190 ts_tablet_manager.cc:1403] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.000s
I20260709 12:22:15.533831 25189 tablet_bootstrap.cc:492] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.534063 25190 raft_consensus.cc:359] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.534560 25189 ts_tablet_manager.cc:1403] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.077s	user 0.007s	sys 0.028s
I20260709 12:22:15.534965 25190 raft_consensus.cc:385] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.535379 25190 raft_consensus.cc:740] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.536456 25191 tablet_bootstrap.cc:492] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.538787 25191 ts_tablet_manager.cc:1403] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.009s
I20260709 12:22:15.554527 25196 raft_consensus.cc:493] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.555526 25196 raft_consensus.cc:515] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.568430 25208 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.569164 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:15.566887 25196 leader_election.cc:290] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:15.570428 25208 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.567466 25190 consensus_queue.cc:260] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.573359 25072 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:15.584991 25189 raft_consensus.cc:359] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.586403 25189 raft_consensus.cc:385] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.586743 25189 raft_consensus.cc:740] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.587386 25208 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.588651 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:15.590607 25190 ts_tablet_manager.cc:1434] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 12:22:15.591012 25189 consensus_queue.cc:260] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.591729 25190 tablet_bootstrap.cc:492] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
W20260709 12:22:15.591957 25035 leader_election.cc:343] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.590323 24997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.595022 24997 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 0.
I20260709 12:22:15.596508 25196 raft_consensus.cc:493] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.597179 25196 raft_consensus.cc:515] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.607893 24961 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.596742 25037 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:15.611543 25195 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.611756 25208 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.611974 25195 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.612172 25208 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.612509 25195 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.594575 25189 ts_tablet_manager.cc:1434] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.005s
I20260709 12:22:15.612773 25208 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.597378 25191 raft_consensus.cc:359] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.597254 24893 catalog_manager.cc:5697] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 1 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.619038 25189 tablet_bootstrap.cc:492] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.615528 25191 raft_consensus.cc:385] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.623742 25191 raft_consensus.cc:740] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.624919 25191 consensus_queue.cc:260] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.629557 25196 leader_election.cc:290] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:15.630184 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
W20260709 12:22:15.632213 24960 leader_election.cc:343] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.636705 25191 ts_tablet_manager.cc:1434] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.097s	user 0.010s	sys 0.014s
I20260709 12:22:15.636284 25196 raft_consensus.cc:493] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.637854 25191 tablet_bootstrap.cc:492] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.637485 25196 raft_consensus.cc:515] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.636096 25195 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.640154 25195 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.633303 25208 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.643636 25189 tablet_bootstrap.cc:654] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.644930 25195 raft_consensus.cc:493] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.646476 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:15.651494 24997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:15.650900 25196 leader_election.cc:290] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
W20260709 12:22:15.655655 24960 leader_election.cc:343] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.658185 25190 tablet_bootstrap.cc:654] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.656337 24997 raft_consensus.cc:2393] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7ea858326b44d384343e348af19c8a in current term 1: Already voted for candidate d8361c2ca34342dd8440184d607f222f in this term.
I20260709 12:22:15.661198 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:15.662147 25072 raft_consensus.cc:2393] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8361c2ca34342dd8440184d607f222f in current term 1: Already voted for candidate 6c7ea858326b44d384343e348af19c8a in this term.
I20260709 12:22:15.647161 25195 raft_consensus.cc:515] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.667099 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:15.667994 24961 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8361c2ca34342dd8440184d607f222f; no voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:15.665144 25191 tablet_bootstrap.cc:654] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:15.668601 25035 leader_election.cc:343] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.664834 25208 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.671852 25035 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a; no voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:15.674142 25219 raft_consensus.cc:2749] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.675343 25208 raft_consensus.cc:2749] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.676971 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:15.678313 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:15.679596 25071 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 0.
I20260709 12:22:15.681705 25195 leader_election.cc:290] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.681602 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:15.683514 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:15.685595 24997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.684926 25072 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:15.685492 25146 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:15.688717 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.689487 24997 raft_consensus.cc:2468] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 0.
I20260709 12:22:15.690490 24995 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 0.
I20260709 12:22:15.687876 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.689965 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
W20260709 12:22:15.691258 25110 leader_election.cc:343] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.692800 24961 leader_election.cc:304] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.691193 24996 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 0.
I20260709 12:22:15.693394 25111 leader_election.cc:304] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:15.694783 25110 leader_election.cc:304] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:15.694825 25111 leader_election.cc:304] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:15.695713 25196 raft_consensus.cc:2804] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.696461 25196 raft_consensus.cc:493] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.698902 25196 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.696830 25195 raft_consensus.cc:2804] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.703426 25195 raft_consensus.cc:493] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.709033 25195 raft_consensus.cc:3060] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.716544 25222 raft_consensus.cc:2804] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.717015 25222 raft_consensus.cc:493] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.717617 25222 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.722093 25189 tablet_bootstrap.cc:492] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.722882 25189 ts_tablet_manager.cc:1403] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.108s	user 0.006s	sys 0.013s
I20260709 12:22:15.725766 25189 raft_consensus.cc:359] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.726505 25189 raft_consensus.cc:385] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.726789 25189 raft_consensus.cc:740] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.728196 25189 consensus_queue.cc:260] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
W20260709 12:22:15.731061 25110 leader_election.cc:343] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.734256 25189 ts_tablet_manager.cc:1434] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 12:22:15.734513 25190 tablet_bootstrap.cc:492] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.735005 25190 ts_tablet_manager.cc:1403] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.143s	user 0.006s	sys 0.014s
I20260709 12:22:15.737509 25196 raft_consensus.cc:515] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.740092 25190 raft_consensus.cc:359] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.740935 25190 raft_consensus.cc:385] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.735677 25189 tablet_bootstrap.cc:492] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.742967 25196 leader_election.cc:290] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:15.744495 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:15.741901 25190 raft_consensus.cc:740] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.745601 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:15.740092 25195 raft_consensus.cc:515] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
W20260709 12:22:15.746502 25110 leader_election.cc:343] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.747290 25190 consensus_queue.cc:260] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.746569 24996 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.740127 25223 raft_consensus.cc:2804] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.749719 25223 raft_consensus.cc:493] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.750175 25223 raft_consensus.cc:3060] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.753077 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:15.753082 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:15.749045 25195 leader_election.cc:290] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.756721 25071 raft_consensus.cc:3060] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.767583 24996 raft_consensus.cc:2468] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 1.
I20260709 12:22:15.771166 25190 ts_tablet_manager.cc:1434] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:22:15.772541 25190 tablet_bootstrap.cc:492] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.780192 25189 tablet_bootstrap.cc:654] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.784546 25071 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 1.
I20260709 12:22:15.786046 24961 leader_election.cc:304] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.782724 25222 raft_consensus.cc:515] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.786090 25111 leader_election.cc:304] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:15.788118 25196 raft_consensus.cc:2804] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.789492 25196 raft_consensus.cc:697] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:15.786942 25195 raft_consensus.cc:2804] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.790431 25196 consensus_queue.cc:237] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.790464 25195 raft_consensus.cc:697] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:15.793900 25195 consensus_queue.cc:237] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.784701 25223 raft_consensus.cc:515] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.801512 25146 raft_consensus.cc:2393] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8361c2ca34342dd8440184d607f222f in current term 1: Already voted for candidate e008c2a93c144707a9b0a021fa99d1f8 in this term.
I20260709 12:22:15.803473 25189 tablet_bootstrap.cc:492] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.803674 25191 tablet_bootstrap.cc:492] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.803954 25189 ts_tablet_manager.cc:1403] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.069s	user 0.006s	sys 0.011s
I20260709 12:22:15.804210 25191 ts_tablet_manager.cc:1403] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.167s	user 0.022s	sys 0.025s
I20260709 12:22:15.810011 25190 tablet_bootstrap.cc:654] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.809881 24893 catalog_manager.cc:5697] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.811935 25222 leader_election.cc:290] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:15.813616 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:15.813938 25191 raft_consensus.cc:359] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.814667 25191 raft_consensus.cc:385] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.814949 25191 raft_consensus.cc:740] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.816475 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:15.817211 24996 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.820286 25189 raft_consensus.cc:359] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.821018 25189 raft_consensus.cc:385] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.821589 25189 raft_consensus.cc:740] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.816293 25191 consensus_queue.cc:260] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.822314 25189 consensus_queue.cc:260] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.828009 25071 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.828374 24892 catalog_manager.cc:5697] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 1 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.827781 25189 ts_tablet_manager.cc:1434] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:22:15.831753 25189 tablet_bootstrap.cc:492] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.832427 24996 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 1.
I20260709 12:22:15.838145 25111 leader_election.cc:304] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:15.839207 25222 raft_consensus.cc:2804] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.839663 25222 raft_consensus.cc:697] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:15.846156 25191 ts_tablet_manager.cc:1434] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.002s
I20260709 12:22:15.845927 25222 consensus_queue.cc:237] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:15.847441 25191 tablet_bootstrap.cc:492] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.847729 25196 raft_consensus.cc:493] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.848346 25196 raft_consensus.cc:515] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.854104 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:15.854650 25071 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 1.
I20260709 12:22:15.856047 25208 raft_consensus.cc:493] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.856143 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:15.857053 25072 raft_consensus.cc:2393] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e008c2a93c144707a9b0a021fa99d1f8 in current term 1: Already voted for candidate d8361c2ca34342dd8440184d607f222f in this term.
I20260709 12:22:15.856710 25208 raft_consensus.cc:515] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.858306 25110 leader_election.cc:304] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e008c2a93c144707a9b0a021fa99d1f8; no voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f
I20260709 12:22:15.858870 25208 leader_election.cc:290] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.855448 25223 leader_election.cc:290] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:15.863421 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
W20260709 12:22:15.864944 25035 leader_election.cc:343] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.866182 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.861438 25227 raft_consensus.cc:2749] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.866971 24995 raft_consensus.cc:2393] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e008c2a93c144707a9b0a021fa99d1f8 in current term 1: Already voted for candidate 6c7ea858326b44d384343e348af19c8a in this term.
I20260709 12:22:15.868062 25196 leader_election.cc:290] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:15.868770 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.869392 24995 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 0.
I20260709 12:22:15.871007 25037 leader_election.cc:304] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:15.872118 25208 raft_consensus.cc:2804] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.872525 25208 raft_consensus.cc:493] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.872860 25208 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.874972 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:15.879168 25110 leader_election.cc:304] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e008c2a93c144707a9b0a021fa99d1f8; no voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f
I20260709 12:22:15.880964 25196 raft_consensus.cc:3060] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.883021 25190 tablet_bootstrap.cc:492] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.887287 25190 ts_tablet_manager.cc:1403] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.115s	user 0.012s	sys 0.006s
I20260709 12:22:15.891124 25208 raft_consensus.cc:515] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.892095 25191 tablet_bootstrap.cc:654] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.893082 25189 tablet_bootstrap.cc:654] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.898908 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:15.900851 24995 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.901592 25196 raft_consensus.cc:2749] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.899495 25208 leader_election.cc:290] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.902473 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:15.908681 25195 raft_consensus.cc:493] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.909209 25195 raft_consensus.cc:515] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.910010 25189 tablet_bootstrap.cc:492] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.910460 25189 ts_tablet_manager.cc:1403] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.002s
I20260709 12:22:15.911540 25195 leader_election.cc:290] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.903319 25190 raft_consensus.cc:359] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.912073 25190 raft_consensus.cc:385] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.912428 25190 raft_consensus.cc:740] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.913159 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:15.913304 25145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:15.914041 25071 raft_consensus.cc:2468] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:15.914345 24995 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 1.
I20260709 12:22:15.915632 24961 leader_election.cc:304] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.916553 25195 raft_consensus.cc:2804] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.915632 25037 leader_election.cc:304] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.916987 25195 raft_consensus.cc:493] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.917398 25195 raft_consensus.cc:3060] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.917104 25190 consensus_queue.cc:260] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.918088 25146 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:15.918597 24960 leader_election.cc:343] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.916695 25189 raft_consensus.cc:359] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.920499 25189 raft_consensus.cc:385] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.919500 25190 ts_tablet_manager.cc:1434] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.027s	user 0.000s	sys 0.007s
I20260709 12:22:15.920823 25189 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.922976 25189 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.920620 25208 raft_consensus.cc:2804] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.924432 25208 raft_consensus.cc:697] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:15.924705 25190 tablet_bootstrap.cc:492] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.927135 25191 tablet_bootstrap.cc:492] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.927542 25191 ts_tablet_manager.cc:1403] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.010s
I20260709 12:22:15.925385 25208 consensus_queue.cc:237] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.929030 25189 ts_tablet_manager.cc:1434] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:22:15.931093 25189 tablet_bootstrap.cc:492] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: Bootstrap starting.
I20260709 12:22:15.936789 25146 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 1.
I20260709 12:22:15.938910 25191 raft_consensus.cc:359] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.941080 25191 raft_consensus.cc:385] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.941437 25191 raft_consensus.cc:740] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.943420 25189 tablet_bootstrap.cc:654] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.942314 25191 consensus_queue.cc:260] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.945628 25190 tablet_bootstrap.cc:654] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.946136 25195 raft_consensus.cc:515] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.950855 25195 leader_election.cc:290] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.951581 25191 ts_tablet_manager.cc:1434] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.002s
I20260709 12:22:15.952535 25191 tablet_bootstrap.cc:492] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:15.952668 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:15.953238 25071 raft_consensus.cc:3060] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.960587 25191 tablet_bootstrap.cc:654] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.962417 25071 raft_consensus.cc:2468] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 1.
W20260709 12:22:15.962651 24960 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:43883, user_credentials={real_user=slave}} blocked reactor thread for 46768.3us
I20260709 12:22:15.963613 24961 leader_election.cc:304] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:15.964437 25145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:15.965818 25195 raft_consensus.cc:2804] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.966481 25195 raft_consensus.cc:697] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:15.967444 25195 consensus_queue.cc:237] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
W20260709 12:22:15.970147 24960 leader_election.cc:343] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.971645 25189 tablet_bootstrap.cc:492] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: No bootstrap required, opened a new log
I20260709 12:22:15.972150 25189 ts_tablet_manager.cc:1403] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.000s
I20260709 12:22:15.972033 24892 catalog_manager.cc:5697] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.975409 25189 raft_consensus.cc:359] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.976452 25189 raft_consensus.cc:385] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.976987 25189 raft_consensus.cc:740] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.978050 25189 consensus_queue.cc:260] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.980422 25189 ts_tablet_manager.cc:1434] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:22:15.981796 24893 catalog_manager.cc:5697] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 1 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.983747 25190 tablet_bootstrap.cc:492] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:15.984357 25190 ts_tablet_manager.cc:1403] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.060s	user 0.008s	sys 0.012s
I20260709 12:22:15.987092 25190 raft_consensus.cc:359] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.987653 25190 raft_consensus.cc:385] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.987903 25190 raft_consensus.cc:740] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.988706 25190 consensus_queue.cc:260] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.988394 24890 catalog_manager.cc:5697] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 1 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.990159 25191 tablet_bootstrap.cc:492] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:15.990659 25191 ts_tablet_manager.cc:1403] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.009s
I20260709 12:22:15.991710 25208 raft_consensus.cc:493] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.992182 25208 raft_consensus.cc:515] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.994328 25208 leader_election.cc:290] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:15.995040 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:15.995352 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:15.995857 24995 raft_consensus.cc:2393] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7ea858326b44d384343e348af19c8a in current term 1: Already voted for candidate e008c2a93c144707a9b0a021fa99d1f8 in this term.
I20260709 12:22:15.996284 25190 ts_tablet_manager.cc:1434] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 12:22:15.996990 25037 leader_election.cc:304] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a; no voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:15.997257 25190 tablet_bootstrap.cc:492] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:15.998027 25208 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.998435 25191 raft_consensus.cc:359] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:15.999091 25191 raft_consensus.cc:385] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.999336 25191 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.000216 25191 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.003005 25191 ts_tablet_manager.cc:1434] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:22:16.003825 25191 tablet_bootstrap.cc:492] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a: Bootstrap starting.
I20260709 12:22:16.004951 25208 raft_consensus.cc:2749] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:16.011596 25191 tablet_bootstrap.cc:654] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.016597 25190 tablet_bootstrap.cc:654] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.017525 25191 tablet_bootstrap.cc:492] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a: No bootstrap required, opened a new log
I20260709 12:22:16.017972 25191 ts_tablet_manager.cc:1403] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:22:16.022097 25191 raft_consensus.cc:359] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.022559 25196 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.022819 25191 raft_consensus.cc:385] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.023154 25191 raft_consensus.cc:740] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.023218 25196 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.023905 25191 consensus_queue.cc:260] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.025755 25196 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:16.026041 25191 ts_tablet_manager.cc:1434] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:16.027308 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:16.027863 24995 raft_consensus.cc:2393] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e008c2a93c144707a9b0a021fa99d1f8 in current term 1: Already voted for candidate d8361c2ca34342dd8440184d607f222f in this term.
I20260709 12:22:16.030959 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:16.031827 25071 raft_consensus.cc:2393] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e008c2a93c144707a9b0a021fa99d1f8 in current term 1: Already voted for candidate 6c7ea858326b44d384343e348af19c8a in this term.
I20260709 12:22:16.032893 25110 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e008c2a93c144707a9b0a021fa99d1f8; no voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f
I20260709 12:22:16.034057 25196 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.034693 25190 tablet_bootstrap.cc:492] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:16.035146 25190 ts_tablet_manager.cc:1403] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.006s
I20260709 12:22:16.037871 25190 raft_consensus.cc:359] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.038564 25190 raft_consensus.cc:385] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.038851 25190 raft_consensus.cc:740] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.039186 25196 raft_consensus.cc:2749] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:16.039882 25195 raft_consensus.cc:493] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.039531 25190 consensus_queue.cc:260] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.040431 25195 raft_consensus.cc:515] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.041790 25190 ts_tablet_manager.cc:1434] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:16.042285 25195 leader_election.cc:290] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:16.042621 25190 tablet_bootstrap.cc:492] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: Bootstrap starting.
I20260709 12:22:16.044622 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:16.045157 25071 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:16.046289 24961 leader_election.cc:304] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:16.047501 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:16.048625 25195 raft_consensus.cc:2804] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:16.048898 25190 tablet_bootstrap.cc:654] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.048998 25195 raft_consensus.cc:493] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.049419 25195 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:16.050483 24960 leader_election.cc:343] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:16.054282 25190 tablet_bootstrap.cc:492] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: No bootstrap required, opened a new log
I20260709 12:22:16.054754 25190 ts_tablet_manager.cc:1403] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:22:16.055353 25195 raft_consensus.cc:515] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.057129 25195 leader_election.cc:290] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:16.057251 25190 raft_consensus.cc:359] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.057946 25190 raft_consensus.cc:385] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.058221 25190 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.058218 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:16.058691 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:16.059140 25071 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.059085 25190 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.060015 25146 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.061404 25190 ts_tablet_manager.cc:1434] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:22:16.066139 25071 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 1.
I20260709 12:22:16.066430 25146 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 1.
I20260709 12:22:16.067517 24961 leader_election.cc:304] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:16.068501 25195 raft_consensus.cc:2804] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:16.069371 25195 raft_consensus.cc:697] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:16.070246 25195 consensus_queue.cc:237] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.079149 24893 catalog_manager.cc:5697] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 1 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:16.097023 24942 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:16.101433 25208 consensus_queue.cc:1048] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.124155 25208 consensus_queue.cc:1048] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:16.133703 25196 raft_consensus.cc:493] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.134611 25196 raft_consensus.cc:515] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.137840 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:16.138181 25196 leader_election.cc:290] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:16.139189 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:16.140033 25071 raft_consensus.cc:2393] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e008c2a93c144707a9b0a021fa99d1f8 in current term 1: Already voted for candidate d8361c2ca34342dd8440184d607f222f in this term.
I20260709 12:22:16.141307 25110 leader_election.cc:304] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e008c2a93c144707a9b0a021fa99d1f8; no voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f
I20260709 12:22:16.143389 25196 raft_consensus.cc:3060] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.146412 25195 raft_consensus.cc:493] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.146909 25195 raft_consensus.cc:515] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.149101 25195 leader_election.cc:290] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:16.150743 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a" is_pre_election: true
I20260709 12:22:16.151317 25196 raft_consensus.cc:2749] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:16.151443 25071 raft_consensus.cc:2468] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:16.152805 24961 leader_election.cc:304] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:16.153391 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:16.153626 25195 raft_consensus.cc:2804] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:16.154043 25195 raft_consensus.cc:493] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.154068 25146 raft_consensus.cc:2468] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 0.
I20260709 12:22:16.154577 25195 raft_consensus.cc:3060] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.160481 25195 raft_consensus.cc:515] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.162303 25195 leader_election.cc:290] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:16.163733 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:16.163538 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:16.164644 25071 raft_consensus.cc:3060] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.164712 25146 raft_consensus.cc:3060] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:16.171415 25071 raft_consensus.cc:2468] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 1.
I20260709 12:22:16.171844 25146 raft_consensus.cc:2468] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 1.
I20260709 12:22:16.173117 24960 leader_election.cc:304] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:16.173969 25195 raft_consensus.cc:2804] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:16.174499 25195 raft_consensus.cc:697] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:16.175310 25195 consensus_queue.cc:237] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:16.184624 24893 catalog_manager.cc:5697] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 1 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:16.213039 25196 consensus_queue.cc:1048] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.227300 25223 consensus_queue.cc:1048] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.257279 25223 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.269866 25222 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.290063 25195 consensus_queue.cc:1048] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.302706 25195 consensus_queue.cc:1048] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.352937 25197 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.367108 25197 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.385696 25195 consensus_queue.cc:1048] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.398223 25256 consensus_queue.cc:1048] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:16.527781 25256 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.541254 25219 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.660140 25219 consensus_queue.cc:1048] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:16.672124 25219 consensus_queue.cc:1048] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:22:17.100554 24942 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:17.623862 25259 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.624574 25259 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.627295 25259 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:17.627934 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8" is_pre_election: true
I20260709 12:22:17.628078 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f" is_pre_election: true
I20260709 12:22:17.628615 25146 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 1.
I20260709 12:22:17.628679 24995 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 1.
I20260709 12:22:17.629747 25035 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:17.630409 25259 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.630699 25259 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.630985 25259 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.635855 25259 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.638146 25259 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:17.638943 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:17.639375 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:17.639710 24995 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.640064 25146 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.645592 24995 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:17.648013 25146 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:17.648010 25037 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:17.648967 25259 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.650252 25259 raft_consensus.cc:697] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:17.651114 25259 consensus_queue.cc:237] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.658625 24890 catalog_manager.cc:5697] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 0 to 2, leader changed from <none> to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 2 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.698133 25146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
mode: GRACEFUL
new_leader_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:22:17.698700 25146 raft_consensus.cc:606] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Received request to transfer leadership to TS d8361c2ca34342dd8440184d607f222f
I20260709 12:22:17.716609 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99129cacd1a34d588a2218095f047a78"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:44154
I20260709 12:22:17.717156 25072 raft_consensus.cc:606] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Received request to transfer leadership to TS d8361c2ca34342dd8440184d607f222f
I20260709 12:22:17.722507 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:44154
I20260709 12:22:17.722972 25072 raft_consensus.cc:606] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Received request to transfer leadership to TS d8361c2ca34342dd8440184d607f222f
I20260709 12:22:17.731298 25146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:22:17.731878 25146 raft_consensus.cc:606] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:17.746464 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:37938
I20260709 12:22:17.747033 24995 raft_consensus.cc:606] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:17.752859 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:37938
I20260709 12:22:17.753530 24995 raft_consensus.cc:606] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:17.754163 25223 raft_consensus.cc:993] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:17.754652 25278 raft_consensus.cc:1081] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:17.756732 25072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44152
I20260709 12:22:17.757321 25072 raft_consensus.cc:493] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.757712 25072 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.759596 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:37938
I20260709 12:22:17.760102 24995 raft_consensus.cc:606] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Received request to transfer leadership to TS e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:17.763265 25072 raft_consensus.cc:515] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.765041 25072 leader_election.cc:290] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:17.766065 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:17.766784 25146 raft_consensus.cc:3055] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.765976 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:17.767120 25146 raft_consensus.cc:740] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:17.767354 24995 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.768013 25146 consensus_queue.cc:260] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.769492 25146 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.774801 24995 raft_consensus.cc:2468] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:17.776003 25037 leader_election.cc:304] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:17.776876 25268 raft_consensus.cc:2804] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.776958 25146 raft_consensus.cc:2468] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:17.777539 25268 raft_consensus.cc:697] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:17.778543 25268 consensus_queue.cc:237] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.783438 25256 raft_consensus.cc:993] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: : Instructing follower e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:17.783922 25256 raft_consensus.cc:1081] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Signalling peer e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:17.785542 25146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:55918
I20260709 12:22:17.786123 25146 raft_consensus.cc:493] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.786500 25146 raft_consensus.cc:3060] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.787369 24890 catalog_manager.cc:5697] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 1 to 2, leader changed from e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 2 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.788591 25223 raft_consensus.cc:993] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: : Instructing follower d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:17.789063 25278 raft_consensus.cc:1081] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Signalling peer d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:17.790536 24995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:37924
I20260709 12:22:17.791109 24995 raft_consensus.cc:493] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.791486 24995 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.792593 25146 raft_consensus.cc:515] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.795077 25146 leader_election.cc:290] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 2 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:17.795745 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:17.796388 25072 raft_consensus.cc:3060] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.798079 24995 raft_consensus.cc:515] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:17.800352 24995 leader_election.cc:290] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 2 election: Requested vote from peers e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:17.801285 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:17.801549 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:17.801896 25146 raft_consensus.cc:3055] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.802117 25071 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.802707 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba09945d8a941978c714a7f4cc9b7a8" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:17.802296 25146 raft_consensus.cc:740] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:17.803195 24996 raft_consensus.cc:3055] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.803123 25072 raft_consensus.cc:2468] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 2.
I20260709 12:22:17.803709 24996 raft_consensus.cc:740] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:17.804563 24996 consensus_queue.cc:260] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.805914 24996 raft_consensus.cc:3060] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.806850 25110 leader_election.cc:304] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:17.807781 25278 raft_consensus.cc:2804] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.808307 25278 raft_consensus.cc:697] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:17.808090 25071 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 2.
I20260709 12:22:17.809175 25278 consensus_queue.cc:237] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:17.810369 24961 leader_election.cc:304] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:17.804641 25146 consensus_queue.cc:260] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:17.811255 25256 raft_consensus.cc:2804] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.811582 25146 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.812840 24996 raft_consensus.cc:2468] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 2.
I20260709 12:22:17.817854 25146 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 2.
I20260709 12:22:17.821046 25256 raft_consensus.cc:697] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:17.822005 25256 consensus_queue.cc:237] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:17.823589 24890 catalog_manager.cc:5697] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 1 to 2, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.831147 24893 catalog_manager.cc:5697] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 1 to 2, leader changed from e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131) to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 2 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.009305 25259 raft_consensus.cc:993] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: : Instructing follower d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:18.009760 25259 raft_consensus.cc:1081] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Signalling peer d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:18.011407 24996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99129cacd1a34d588a2218095f047a78"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:37918
I20260709 12:22:18.011868 24996 raft_consensus.cc:493] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.012117 24996 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.016793 24996 raft_consensus.cc:515] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.018324 24996 leader_election.cc:290] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 2 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:18.019336 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:18.019443 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:18.019949 25146 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.020087 25071 raft_consensus.cc:3055] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.020468 25071 raft_consensus.cc:740] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:18.021263 25071 consensus_queue.cc:260] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.022509 25071 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.025357 25146 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 2.
I20260709 12:22:18.026410 24960 leader_election.cc:304] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:18.027323 25256 raft_consensus.cc:2804] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.027846 25219 raft_consensus.cc:993] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:18.028340 25280 raft_consensus.cc:1081] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:18.028860 25071 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 2.
I20260709 12:22:18.029145 25256 raft_consensus.cc:697] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:18.029924 25071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44142
I20260709 12:22:18.030046 25256 consensus_queue.cc:237] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.030699 25071 raft_consensus.cc:493] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.031236 25071 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.037295 25071 raft_consensus.cc:515] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.039316 25071 leader_election.cc:290] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:18.039933 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:18.040378 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:18.040489 24996 raft_consensus.cc:3055] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.039026 24893 catalog_manager.cc:5697] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 1 to 2, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 2 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.040974 25146 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.040935 24996 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:18.041940 24996 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.042986 24996 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.046586 25146 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:18.047745 25035 leader_election.cc:304] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:18.048619 25259 raft_consensus.cc:2804] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.049077 25259 raft_consensus.cc:697] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:18.048929 24996 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:18.049943 25259 consensus_queue.cc:237] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.056497 24893 catalog_manager.cc:5697] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 1 to 2, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 2 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.096355 25146 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:22:18.097740 25259 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:18.111012 24996 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:22:18.113837 25268 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:18.133086 25289 raft_consensus.cc:993] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: : Instructing follower d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:18.133582 25268 raft_consensus.cc:1081] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Signalling peer d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:18.135118 24996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:37918
I20260709 12:22:18.135684 24996 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.136111 24996 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.140995 24996 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.143115 24996 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 3 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:18.143550 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:18.143810 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:18.144170 25071 raft_consensus.cc:3055] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.144479 25146 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.144563 25071 raft_consensus.cc:740] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:18.145437 25071 consensus_queue.cc:260] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.146657 25071 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.152120 25146 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 3.
I20260709 12:22:18.153319 25071 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 3.
I20260709 12:22:18.153601 24960 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:18.154398 25280 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.154800 25280 raft_consensus.cc:697] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:18.155454 25280 consensus_queue.cc:237] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.162184 24893 catalog_manager.cc:5697] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 2 to 3, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 3 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.227828 25256 raft_consensus.cc:993] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:18.228297 25256 raft_consensus.cc:1081] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:18.229898 25071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44142
I20260709 12:22:18.230348 25071 raft_consensus.cc:493] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.230610 25071 raft_consensus.cc:3060] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.235237 25071 raft_consensus.cc:515] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.236929 25071 leader_election.cc:290] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:18.238034 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:18.238081 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642b30d2b44b4bb098a6a9a5a2bdeab2" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:18.238773 25146 raft_consensus.cc:3060] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.238790 24996 raft_consensus.cc:3055] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.239224 24996 raft_consensus.cc:740] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:18.239962 24996 consensus_queue.cc:260] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.240993 24996 raft_consensus.cc:3060] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.243245 25146 raft_consensus.cc:2468] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:18.244323 25035 leader_election.cc:304] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:18.245040 25268 raft_consensus.cc:2804] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.245543 25268 raft_consensus.cc:697] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:18.246857 24996 raft_consensus.cc:2468] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:18.246455 25268 consensus_queue.cc:237] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:18.254333 24893 catalog_manager.cc:5697] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 1 to 2, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 2 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.279570 25071 raft_consensus.cc:1275] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.281036 25256 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.290252 25146 raft_consensus.cc:1275] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.292330 25219 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.364696 25146 raft_consensus.cc:1275] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.365988 25268 consensus_queue.cc:1048] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.374883 24996 raft_consensus.cc:1275] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.376609 25296 consensus_queue.cc:1048] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:18.377774 25071 raft_consensus.cc:1275] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.378737 24996 raft_consensus.cc:1275] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.379199 25278 consensus_queue.cc:1048] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.381330 25278 consensus_queue.cc:1048] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.444990 24995 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.446563 25296 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.455632 25146 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.457151 25259 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.526326 25071 raft_consensus.cc:1275] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.527570 25219 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.537853 25146 raft_consensus.cc:1275] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.539153 25219 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.657284 24995 raft_consensus.cc:1275] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.657321 25071 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:18.663054 25289 consensus_queue.cc:1048] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.663327 25219 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.677940 25146 raft_consensus.cc:1275] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.679900 25146 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:18.680470 25289 consensus_queue.cc:1048] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.684850 25219 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:20.794234 24862 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:22:20.795073 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:20.820147 25071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:44158
I20260709 12:22:20.820863 25071 raft_consensus.cc:606] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Received request to transfer leadership to TS e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:20.828430 25314 raft_consensus.cc:993] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: : Instructing follower e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:20.828910 25309 raft_consensus.cc:1081] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Signalling peer e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:20.830526 25145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:55910
I20260709 12:22:20.831594 25145 raft_consensus.cc:493] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:20.831980 25145 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.834640 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:37948
I20260709 12:22:20.835243 24995 raft_consensus.cc:606] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 LEADER]: Received request to transfer leadership to TS e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:20.836822 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:20.837455 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:20.837468 25145 raft_consensus.cc:515] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:20.840399 25145 leader_election.cc:290] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 3 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:20.840696 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:20.841377 24995 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.842274 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:20.842792 25071 raft_consensus.cc:3055] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:20.843061 25071 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:20.843791 25071 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:20.844897 25071 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.848637 24995 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 3.
I20260709 12:22:20.850106 25111 leader_election.cc:304] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:20.851049 25301 raft_consensus.cc:2804] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.851640 25071 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 3.
I20260709 12:22:20.852487 25301 raft_consensus.cc:697] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:20.853396 25301 consensus_queue.cc:237] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:20.859994 24894 catalog_manager.cc:5697] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 2 to 3, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:20.872869 25310 raft_consensus.cc:993] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: : Instructing follower e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:20.873373 25219 raft_consensus.cc:1081] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 LEADER]: Signalling peer e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:20.875057 25145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:55918
I20260709 12:22:20.875576 25145 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:20.875890 25145 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.880715 25145 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:20.882418 25145 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 4 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:20.883253 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:20.883390 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:20.883845 24995 raft_consensus.cc:3055] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:20.883949 25071 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.884158 24995 raft_consensus.cc:740] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:20.884980 24995 consensus_queue.cc:260] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:20.885876 24995 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.888300 25071 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 4.
I20260709 12:22:20.889341 25110 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:20.890304 25301 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:20.890718 24995 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 4.
I20260709 12:22:20.890718 25301 raft_consensus.cc:697] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:20.891721 25301 consensus_queue.cc:237] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:20.898587 24894 catalog_manager.cc:5697] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 3 to 4, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 4 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:21.385726 25071 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.386937 25301 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.399688 24995 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.401568 25325 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.426535 24995 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:21.428021 25325 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.437584 25071 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:21.441151 25301 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.838586 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.845163 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:21.845649 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:22.847389 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:22.853678 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:22.854120 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:23.855295 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.861310 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:23.861725 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:24.862725 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.868849 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:24.869285 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.870358 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.877545 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:25.878127 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:26.879119 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:26.885004 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:26.885361 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:27.886399 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:27.891324 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:27.891778 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:28.892980 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.898090 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:28.898548 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:29.899567 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:29.905506 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:29.905871 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:30.939353 24862 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:22:30.944182 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:37938
I20260709 12:22:30.944867 24995 raft_consensus.cc:606] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:30.950419 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:37938
I20260709 12:22:30.950949 24995 raft_consensus.cc:606] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:30.956126 24995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99129cacd1a34d588a2218095f047a78"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:37938
I20260709 12:22:30.956693 24995 raft_consensus.cc:606] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:30.961884 25146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:22:30.962491 25146 raft_consensus.cc:606] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:30.976729 25146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
mode: GRACEFUL
new_leader_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:55930
I20260709 12:22:30.977423 25146 raft_consensus.cc:606] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Received request to transfer leadership to TS 6c7ea858326b44d384343e348af19c8a
I20260709 12:22:30.981648 25341 raft_consensus.cc:993] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:30.982266 25337 raft_consensus.cc:1081] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:30.984151 25071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44142
I20260709 12:22:30.984696 25071 raft_consensus.cc:493] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:30.985038 25071 raft_consensus.cc:3060] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:30.990387 25071 raft_consensus.cc:515] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:30.992627 25071 leader_election.cc:290] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:30.993687 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:30.993672 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31511785bceb4dae889ed5a85e3cc9fd" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:30.994467 24995 raft_consensus.cc:3055] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:30.994477 25146 raft_consensus.cc:3060] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:30.994837 24995 raft_consensus.cc:740] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:30.995632 24995 consensus_queue.cc:260] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:30.996701 24995 raft_consensus.cc:3060] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:30.999212 25146 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:31.000602 25035 leader_election.cc:304] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:31.001536 25336 raft_consensus.cc:2804] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.001833 24995 raft_consensus.cc:2468] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 2.
I20260709 12:22:31.002719 25336 raft_consensus.cc:697] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:31.003850 25336 consensus_queue.cc:237] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.011059 24894 catalog_manager.cc:5697] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 1 to 2, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 2 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.045751 25341 raft_consensus.cc:993] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.046228 25341 raft_consensus.cc:1081] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.047775 25071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99129cacd1a34d588a2218095f047a78"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44142
I20260709 12:22:31.048321 25071 raft_consensus.cc:493] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.048673 25071 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.053023 25071 raft_consensus.cc:515] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.054904 25071 leader_election.cc:290] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 3 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:31.055751 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:31.055962 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:31.056363 24995 raft_consensus.cc:3055] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:31.056512 25146 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.056684 24995 raft_consensus.cc:740] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:31.057402 24995 consensus_queue.cc:260] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.058284 24995 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.060858 25146 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 3.
I20260709 12:22:31.061789 25035 leader_election.cc:304] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:31.062502 25336 raft_consensus.cc:2804] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.062783 24995 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 3.
I20260709 12:22:31.063019 25336 raft_consensus.cc:697] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:31.063869 25336 consensus_queue.cc:237] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.070561 24894 catalog_manager.cc:5697] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 2 to 3, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 3 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.249567 25338 raft_consensus.cc:993] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.250113 25349 raft_consensus.cc:1081] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.251498 25071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44152
I20260709 12:22:31.251968 25071 raft_consensus.cc:493] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.252322 25071 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.256376 25071 raft_consensus.cc:515] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.258288 25071 leader_election.cc:290] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 5 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:31.258577 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:31.258821 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cbec0e858a4b82b2420d2a6c4c866d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:31.259207 24995 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.259353 25146 raft_consensus.cc:3055] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:31.259750 25146 raft_consensus.cc:740] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:31.260386 25146 consensus_queue.cc:260] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.262660 25146 raft_consensus.cc:3060] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.264621 24995 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 5.
I20260709 12:22:31.265734 25037 leader_election.cc:304] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:31.266474 25336 raft_consensus.cc:2804] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:31.266906 25336 raft_consensus.cc:697] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 5 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:31.267134 25146 raft_consensus.cc:2468] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 5.
I20260709 12:22:31.267935 25336 consensus_queue.cc:237] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.277308 25338 raft_consensus.cc:993] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.277985 25349 raft_consensus.cc:1081] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.277308 24894 catalog_manager.cc:5697] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 4 to 5, leader changed from e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 5 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.279696 25337 raft_consensus.cc:993] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: : Instructing follower 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.280264 25341 raft_consensus.cc:1081] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Signalling peer 6c7ea858326b44d384343e348af19c8a to start an election
I20260709 12:22:31.280280 25071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44152
I20260709 12:22:31.281032 25071 raft_consensus.cc:493] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.281684 25071 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.282032 25072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
 from {username='slave'} at 127.0.0.1:44142
I20260709 12:22:31.282809 25072 raft_consensus.cc:493] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.283215 25072 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.287829 25071 raft_consensus.cc:515] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.290081 25071 leader_election.cc:290] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 4 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:31.289844 25072 raft_consensus.cc:515] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:31.290915 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:31.291177 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:31.291494 24995 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.291775 25146 raft_consensus.cc:3055] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:31.292164 25146 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:31.292208 25072 leader_election.cc:290] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 3 election: Requested vote from peers e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883), d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381)
I20260709 12:22:31.293107 25145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:31.293107 25146 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.293808 25145 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.294337 25146 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.297312 24995 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 4.
I20260709 12:22:31.298305 25037 leader_election.cc:304] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, d8361c2ca34342dd8440184d607f222f; no voters: 
I20260709 12:22:31.299018 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "6c7ea858326b44d384343e348af19c8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:31.299194 25336 raft_consensus.cc:2804] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:31.299675 24996 raft_consensus.cc:3055] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:31.300122 24996 raft_consensus.cc:740] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:31.299712 25336 raft_consensus.cc:697] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:31.300705 25146 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 4.
I20260709 12:22:31.300956 24996 consensus_queue.cc:260] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:31.302291 24996 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.302170 25336 consensus_queue.cc:237] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:31.305377 25145 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 3.
I20260709 12:22:31.306886 25035 leader_election.cc:304] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:31.307936 25336 raft_consensus.cc:2804] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.308581 25336 raft_consensus.cc:697] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Becoming Leader. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:31.309320 25336 consensus_queue.cc:237] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:31.309667 24996 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7ea858326b44d384343e348af19c8a in term 3.
I20260709 12:22:31.310817 24894 catalog_manager.cc:5697] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 3 to 4, leader changed from e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 4 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.322568 24894 catalog_manager.cc:5697] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a reported cstate change: term changed from 2 to 3, leader changed from d8361c2ca34342dd8440184d607f222f (127.24.71.129) to 6c7ea858326b44d384343e348af19c8a (127.24.71.130). New cstate: current_term: 3 leader_uuid: "6c7ea858326b44d384343e348af19c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:31.474983 25145 raft_consensus.cc:1275] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.476531 25336 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.481356 24996 raft_consensus.cc:1275] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.482913 25336 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.579448 24996 raft_consensus.cc:1275] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.580911 25336 consensus_queue.cc:1048] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:31.589644 25145 raft_consensus.cc:1275] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.591214 25354 consensus_queue.cc:1048] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:31.690323 25145 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:31.691725 25354 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:31.701581 24996 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:31.703130 25354 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:31.792373 24996 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:31.793785 25354 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:31.803290 25145 raft_consensus.cc:1275] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:31.804782 25342 consensus_queue.cc:1048] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:31.815717 25145 raft_consensus.cc:1275] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.817054 25342 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.826730 24996 raft_consensus.cc:1275] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Refusing update from remote peer 6c7ea858326b44d384343e348af19c8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.830701 25354 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.015383 24862 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:22:34.016489 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:34.024061 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:44158
I20260709 12:22:34.024874 25072 raft_consensus.cc:606] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 LEADER]: Received request to transfer leadership to TS d8361c2ca34342dd8440184d607f222f
I20260709 12:22:34.028441 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e119a49418946f79833fea8659bab66"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:44158
I20260709 12:22:34.028981 25072 raft_consensus.cc:606] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Received request to transfer leadership to TS d8361c2ca34342dd8440184d607f222f
I20260709 12:22:34.031877 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:44158
I20260709 12:22:34.032480 25072 raft_consensus.cc:606] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Received request to transfer leadership to TS d8361c2ca34342dd8440184d607f222f
I20260709 12:22:34.035578 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99129cacd1a34d588a2218095f047a78"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:44158
I20260709 12:22:34.036016 25072 raft_consensus.cc:606] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Received request to transfer leadership to TS e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:34.038206 25072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf"
dest_uuid: "6c7ea858326b44d384343e348af19c8a"
mode: GRACEFUL
new_leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:44158
I20260709 12:22:34.038694 25072 raft_consensus.cc:606] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Received request to transfer leadership to TS e008c2a93c144707a9b0a021fa99d1f8
I20260709 12:22:34.039789 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:34.040388 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:34.098691 25368 raft_consensus.cc:993] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: : Instructing follower d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:34.099206 25336 raft_consensus.cc:1081] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Signalling peer d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:34.100780 24996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e119a49418946f79833fea8659bab66"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:37918
I20260709 12:22:34.101239 24996 raft_consensus.cc:493] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.101513 24996 raft_consensus.cc:3060] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.106419 24996 raft_consensus.cc:515] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.108963 24996 leader_election.cc:290] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 2 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:34.110134 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:34.109958 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e119a49418946f79833fea8659bab66" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:34.110739 25071 raft_consensus.cc:3055] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.110694 25146 raft_consensus.cc:3060] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.111132 25071 raft_consensus.cc:740] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:34.111980 25071 consensus_queue.cc:260] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.113231 25071 raft_consensus.cc:3060] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.117545 25146 raft_consensus.cc:2468] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 2.
I20260709 12:22:34.118836 24960 leader_election.cc:304] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:34.119299 25071 raft_consensus.cc:2468] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 2.
I20260709 12:22:34.120643 25369 raft_consensus.cc:2804] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.121527 25369 raft_consensus.cc:697] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:34.122404 25369 consensus_queue.cc:237] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.129204 24894 catalog_manager.cc:5697] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 1 to 2, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 2 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.202414 25368 raft_consensus.cc:993] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: : Instructing follower d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:34.203158 25336 raft_consensus.cc:1081] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 LEADER]: Signalling peer d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:34.204934 24996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:37918
I20260709 12:22:34.205431 24996 raft_consensus.cc:493] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.205775 24996 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.209975 24996 raft_consensus.cc:515] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.211427 24996 leader_election.cc:290] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 5 election: Requested vote from peers 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763), e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883)
I20260709 12:22:34.212345 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:34.212584 25146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a5519e8cae74e6691ceb6183d919e1d" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:34.212944 25071 raft_consensus.cc:3055] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:34.213011 25146 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.213301 25071 raft_consensus.cc:740] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:34.214092 25071 consensus_queue.cc:260] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.215142 25071 raft_consensus.cc:3060] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.219274 25146 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 5.
I20260709 12:22:34.220539 24960 leader_election.cc:304] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:34.221228 25071 raft_consensus.cc:2468] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 5.
I20260709 12:22:34.221312 25369 raft_consensus.cc:2804] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:34.221853 25369 raft_consensus.cc:697] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 5 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:34.222652 25369 consensus_queue.cc:237] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.230520 24894 catalog_manager.cc:5697] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 4 to 5, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 5 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.284044 25368 raft_consensus.cc:993] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: : Instructing follower e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:34.284582 25368 raft_consensus.cc:1081] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Signalling peer e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:34.285756 25368 raft_consensus.cc:993] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: : Instructing follower e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:34.286212 25368 raft_consensus.cc:1081] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Signalling peer e008c2a93c144707a9b0a021fa99d1f8 to start an election
I20260709 12:22:34.286615 25146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:55910
I20260709 12:22:34.287122 25146 raft_consensus.cc:493] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.287469 25146 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.288054 25145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99129cacd1a34d588a2218095f047a78"
dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
 from {username='slave'} at 127.0.0.1:55910
I20260709 12:22:34.288652 25145 raft_consensus.cc:493] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.289016 25145 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.292728 25146 raft_consensus.cc:515] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.295782 25146 leader_election.cc:290] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 3 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:34.296088 24996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:34.296263 25071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51cac45d9664bfa834f293c39e73ebf" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:34.297166 24996 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.297487 25071 raft_consensus.cc:3055] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.297956 25071 raft_consensus.cc:740] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:34.298552 25145 raft_consensus.cc:515] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.298950 25071 consensus_queue.cc:260] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.300225 25071 raft_consensus.cc:3060] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.301682 24995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d8361c2ca34342dd8440184d607f222f"
I20260709 12:22:34.301822 25145 leader_election.cc:290] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 4 election: Requested vote from peers d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:34.302256 24995 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.303908 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99129cacd1a34d588a2218095f047a78" candidate_uuid: "e008c2a93c144707a9b0a021fa99d1f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:34.304570 25072 raft_consensus.cc:3055] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:34.304924 25072 raft_consensus.cc:740] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:34.305701 25072 consensus_queue.cc:260] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.306950 25072 raft_consensus.cc:3060] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.307637 24996 raft_consensus.cc:2468] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 3.
I20260709 12:22:34.307654 25071 raft_consensus.cc:2468] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 3.
I20260709 12:22:34.309150 25110 leader_election.cc:304] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7ea858326b44d384343e348af19c8a, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:34.309306 24995 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 4.
I20260709 12:22:34.310086 25349 raft_consensus.cc:2804] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:34.310752 25111 leader_election.cc:304] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:34.315339 25072 raft_consensus.cc:2468] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e008c2a93c144707a9b0a021fa99d1f8 in term 4.
I20260709 12:22:34.316830 25349 raft_consensus.cc:697] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:34.318310 25377 raft_consensus.cc:2804] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:34.317840 25349 consensus_queue.cc:237] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.318841 25377 raft_consensus.cc:697] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Becoming Leader. State: Replica: e008c2a93c144707a9b0a021fa99d1f8, State: Running, Role: LEADER
I20260709 12:22:34.319586 25377 consensus_queue.cc:237] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } }
I20260709 12:22:34.327162 24894 catalog_manager.cc:5697] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 2 to 3, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:34.339102 24894 catalog_manager.cc:5697] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 reported cstate change: term changed from 3 to 4, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131). New cstate: current_term: 4 leader_uuid: "e008c2a93c144707a9b0a021fa99d1f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:34.505781 25368 raft_consensus.cc:993] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: : Instructing follower d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:34.506253 25336 raft_consensus.cc:1081] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Signalling peer d8361c2ca34342dd8440184d607f222f to start an election
I20260709 12:22:34.507783 24995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277"
dest_uuid: "d8361c2ca34342dd8440184d607f222f"
 from {username='slave'} at 127.0.0.1:37918
I20260709 12:22:34.508397 24995 raft_consensus.cc:493] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.508671 24995 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.512638 24995 raft_consensus.cc:515] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:34.514129 24995 leader_election.cc:290] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 4 election: Requested vote from peers e008c2a93c144707a9b0a021fa99d1f8 (127.24.71.131:43883), 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763)
I20260709 12:22:34.514968 25145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e008c2a93c144707a9b0a021fa99d1f8"
I20260709 12:22:34.515055 25072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981240fbed2c4a16a6e4c90e85a90277" candidate_uuid: "d8361c2ca34342dd8440184d607f222f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6c7ea858326b44d384343e348af19c8a"
I20260709 12:22:34.515615 25145 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.515844 25072 raft_consensus.cc:3055] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:34.516242 25072 raft_consensus.cc:740] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7ea858326b44d384343e348af19c8a, State: Running, Role: LEADER
I20260709 12:22:34.516919 25072 consensus_queue.cc:260] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:34.517819 25072 raft_consensus.cc:3060] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.520486 25145 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 4.
I20260709 12:22:34.521456 24960 leader_election.cc:304] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8361c2ca34342dd8440184d607f222f, e008c2a93c144707a9b0a021fa99d1f8; no voters: 
I20260709 12:22:34.522272 25369 raft_consensus.cc:2804] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:34.522702 25369 raft_consensus.cc:697] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 4 LEADER]: Becoming Leader. State: Replica: d8361c2ca34342dd8440184d607f222f, State: Running, Role: LEADER
I20260709 12:22:34.522814 25072 raft_consensus.cc:2468] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8361c2ca34342dd8440184d607f222f in term 4.
I20260709 12:22:34.523690 25369 consensus_queue.cc:237] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } }
I20260709 12:22:34.531618 24894 catalog_manager.cc:5697] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f reported cstate change: term changed from 3 to 4, leader changed from 6c7ea858326b44d384343e348af19c8a (127.24.71.130) to d8361c2ca34342dd8440184d607f222f (127.24.71.129). New cstate: current_term: 4 leader_uuid: "d8361c2ca34342dd8440184d607f222f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.676476 25072 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:34.677819 25369 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:34.678704 25145 raft_consensus.cc:1275] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:34.679920 25369 consensus_queue.cc:1048] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:34.701989 25145 raft_consensus.cc:1275] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.703480 25369 consensus_queue.cc:1048] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.714746 25071 raft_consensus.cc:1275] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.717742 25369 consensus_queue.cc:1048] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.769918 24995 raft_consensus.cc:1275] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:34.771502 25377 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:34.782800 25071 raft_consensus.cc:1275] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:34.784265 25392 consensus_queue.cc:1048] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:34.817818 24995 raft_consensus.cc:1275] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.819236 25392 consensus_queue.cc:1048] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8361c2ca34342dd8440184d607f222f" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.829495 25071 raft_consensus.cc:1275] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Refusing update from remote peer e008c2a93c144707a9b0a021fa99d1f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.831117 25378 consensus_queue.cc:1048] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:34.965210 25145 raft_consensus.cc:1275] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:34.966696 25387 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e008c2a93c144707a9b0a021fa99d1f8" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:34.976197 25071 raft_consensus.cc:1275] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Refusing update from remote peer d8361c2ca34342dd8440184d607f222f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:34.978801 25389 consensus_queue.cc:1048] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7ea858326b44d384343e348af19c8a" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 46763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:35.041623 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:35.046618 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:35.047068 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:36.048429 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:36.054651 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:36.055367 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:37.056581 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:37.061565 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:37.062006 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:38.063205 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:38.068786 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:38.069222 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:39.070425 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:39.075438 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:39.075888 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:40.076960 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:40.082202 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:40.082664 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:41.083909 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:41.089694 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:41.090191 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:42.091390 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:42.099376 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:42.099920 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:43.101543 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:43.107563 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:43.108029 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:44.141460 24862 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:22:44.142108 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
W20260709 12:22:44.152325 25037 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:22:44.158317 25037 consensus_peers.cc:597] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a -> Peer d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381): Couldn't send request to peer d8361c2ca34342dd8440184d607f222f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:44.171553 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:44.172508 24862 tablet_replica.cc:333] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.173659 24862 raft_consensus.cc:2243] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.174414 24862 raft_consensus.cc:2272] T 6ba09945d8a941978c714a7f4cc9b7a8 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.177124 24862 tablet_replica.cc:333] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.177744 24862 raft_consensus.cc:2243] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.178555 24862 raft_consensus.cc:2272] T 3a5519e8cae74e6691ceb6183d919e1d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.180404 24862 tablet_replica.cc:333] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.180960 24862 raft_consensus.cc:2243] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.181402 24862 raft_consensus.cc:2272] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.183126 24862 tablet_replica.cc:333] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.183641 24862 raft_consensus.cc:2243] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.184098 24862 raft_consensus.cc:2272] T 99129cacd1a34d588a2218095f047a78 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.185891 24862 tablet_replica.cc:333] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.186401 24862 raft_consensus.cc:2243] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.186839 24862 raft_consensus.cc:2272] T 31511785bceb4dae889ed5a85e3cc9fd P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.188629 24862 tablet_replica.cc:333] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.189167 24862 raft_consensus.cc:2243] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.189632 24862 raft_consensus.cc:2272] T f51cac45d9664bfa834f293c39e73ebf P d8361c2ca34342dd8440184d607f222f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.191344 24862 tablet_replica.cc:333] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.191836 24862 raft_consensus.cc:2243] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.192684 24862 raft_consensus.cc:2272] T 4e119a49418946f79833fea8659bab66 P d8361c2ca34342dd8440184d607f222f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.194363 24862 tablet_replica.cc:333] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.194831 24862 raft_consensus.cc:2243] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.195603 24862 raft_consensus.cc:2272] T 981240fbed2c4a16a6e4c90e85a90277 P d8361c2ca34342dd8440184d607f222f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.197332 24862 tablet_replica.cc:333] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f: stopping tablet replica
I20260709 12:22:44.197786 24862 raft_consensus.cc:2243] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.198231 24862 raft_consensus.cc:2272] T e8cbec0e858a4b82b2420d2a6c4c866d P d8361c2ca34342dd8440184d607f222f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.294870 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
W20260709 12:22:44.295557 25111 consensus_peers.cc:597] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 -> Peer d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381): Couldn't send request to peer d8361c2ca34342dd8440184d607f222f. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:34381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:44.295987 25111 consensus_peers.cc:597] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 -> Peer d8361c2ca34342dd8440184d607f222f (127.24.71.129:34381): Couldn't send request to peer d8361c2ca34342dd8440184d607f222f. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:34381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:44.313930 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
W20260709 12:22:44.334900 25110 consensus_peers.cc:597] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 -> Peer 6c7ea858326b44d384343e348af19c8a (127.24.71.130:46763): Couldn't send request to peer 6c7ea858326b44d384343e348af19c8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:44.346539 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:44.347290 24862 tablet_replica.cc:333] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.348194 24862 raft_consensus.cc:2243] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.348850 24862 raft_consensus.cc:2272] T 6ba09945d8a941978c714a7f4cc9b7a8 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.350755 24862 tablet_replica.cc:333] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.351332 24862 raft_consensus.cc:2243] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.351804 24862 raft_consensus.cc:2272] T 3a5519e8cae74e6691ceb6183d919e1d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.353668 24862 tablet_replica.cc:333] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.354203 24862 raft_consensus.cc:2243] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.355039 24862 raft_consensus.cc:2272] T 31511785bceb4dae889ed5a85e3cc9fd P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.357131 24862 tablet_replica.cc:333] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.357676 24862 raft_consensus.cc:2243] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.358505 24862 raft_consensus.cc:2272] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.360200 24862 tablet_replica.cc:333] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.360982 24862 raft_consensus.cc:2243] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.361428 24862 raft_consensus.cc:2272] T 99129cacd1a34d588a2218095f047a78 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.363297 24862 tablet_replica.cc:333] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.363822 24862 raft_consensus.cc:2243] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.364761 24862 raft_consensus.cc:2272] T e8cbec0e858a4b82b2420d2a6c4c866d P 6c7ea858326b44d384343e348af19c8a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.366621 24862 tablet_replica.cc:333] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.367193 24862 raft_consensus.cc:2243] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.367673 24862 raft_consensus.cc:2272] T f51cac45d9664bfa834f293c39e73ebf P 6c7ea858326b44d384343e348af19c8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.369493 24862 tablet_replica.cc:333] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.370021 24862 raft_consensus.cc:2243] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.370479 24862 raft_consensus.cc:2272] T 981240fbed2c4a16a6e4c90e85a90277 P 6c7ea858326b44d384343e348af19c8a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.372220 24862 tablet_replica.cc:333] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a: stopping tablet replica
I20260709 12:22:44.372723 24862 raft_consensus.cc:2243] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.373186 24862 raft_consensus.cc:2272] T 4e119a49418946f79833fea8659bab66 P 6c7ea858326b44d384343e348af19c8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.404371 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
I20260709 12:22:44.423997 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
I20260709 12:22:44.453585 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:44.454581 24862 tablet_replica.cc:333] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.455221 24862 raft_consensus.cc:2243] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.455749 24862 raft_consensus.cc:2272] T 3a5519e8cae74e6691ceb6183d919e1d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.457849 24862 tablet_replica.cc:333] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.458408 24862 raft_consensus.cc:2243] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.459213 24862 raft_consensus.cc:2272] T 6ba09945d8a941978c714a7f4cc9b7a8 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.461179 24862 tablet_replica.cc:333] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.461762 24862 raft_consensus.cc:2243] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.462211 24862 raft_consensus.cc:2272] T 642b30d2b44b4bb098a6a9a5a2bdeab2 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.463968 24862 tablet_replica.cc:333] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.464579 24862 raft_consensus.cc:2243] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.465359 24862 raft_consensus.cc:2272] T 99129cacd1a34d588a2218095f047a78 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.467087 24862 tablet_replica.cc:333] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.467594 24862 raft_consensus.cc:2243] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.468075 24862 raft_consensus.cc:2272] T 981240fbed2c4a16a6e4c90e85a90277 P e008c2a93c144707a9b0a021fa99d1f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.469813 24862 tablet_replica.cc:333] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.470278 24862 raft_consensus.cc:2243] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.470685 24862 raft_consensus.cc:2272] T 4e119a49418946f79833fea8659bab66 P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.472225 24862 tablet_replica.cc:333] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.472733 24862 raft_consensus.cc:2243] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.473459 24862 raft_consensus.cc:2272] T f51cac45d9664bfa834f293c39e73ebf P e008c2a93c144707a9b0a021fa99d1f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.475066 24862 tablet_replica.cc:333] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.475540 24862 raft_consensus.cc:2243] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.476032 24862 raft_consensus.cc:2272] T 31511785bceb4dae889ed5a85e3cc9fd P e008c2a93c144707a9b0a021fa99d1f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.477726 24862 tablet_replica.cc:333] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8: stopping tablet replica
I20260709 12:22:44.478163 24862 raft_consensus.cc:2243] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.478564 24862 raft_consensus.cc:2272] T e8cbec0e858a4b82b2420d2a6c4c866d P e008c2a93c144707a9b0a021fa99d1f8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.507488 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:22:44.527582 24862 master.cc:562] Master@127.24.71.190:36683 shutting down...
I20260709 12:22:44.755856 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.756533 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.756850 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P 206f892206f242f388d6074d5412884d: stopping tablet replica
I20260709 12:22:44.776458 24862 master.cc:584] Master@127.24.71.190:36683 shutdown complete.
6c7ea858326b44d384343e348af19c8a  4
d8361c2ca34342dd8440184d607f222f  4
e008c2a93c144707a9b0a021fa99d1f8  1
6c7ea858326b44d384343e348af19c8a  3
d8361c2ca34342dd8440184d607f222f  3
e008c2a93c144707a9b0a021fa99d1f8  3
6c7ea858326b44d384343e348af19c8a  8
d8361c2ca34342dd8440184d607f222f  0
e008c2a93c144707a9b0a021fa99d1f8  1
6c7ea858326b44d384343e348af19c8a  3
d8361c2ca34342dd8440184d607f222f  3
e008c2a93c144707a9b0a021fa99d1f8  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (32283 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:22:44.818881 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:42771
I20260709 12:22:44.820137 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:44.826934 25412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.827051 25411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.827129 25414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:44.828502 24862 server_base.cc:1061] running on GCE node
I20260709 12:22:44.829772 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:44.830018 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:44.830200 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599764830184 us; error 0 us; skew 500 ppm
I20260709 12:22:44.830713 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:44.833681 24862 webserver.cc:533] Webserver started at http://127.24.71.190:42203/ using document root <none> and password file <none>
I20260709 12:22:44.834273 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:44.834486 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:44.834764 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:44.835956 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "fb82d23fc0be4c0ca0398b9d574df107"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-3zgk"
I20260709 12:22:44.840857 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:22:44.844949 25419 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.845911 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:22:44.846509 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "fb82d23fc0be4c0ca0398b9d574df107"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-3zgk"
I20260709 12:22:44.847081 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:44.874826 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:44.876165 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:44.917491 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:42771
I20260709 12:22:44.917584 25470 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:42771 every 8 connection(s)
I20260709 12:22:44.921926 25471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:44.931684 25471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107: Bootstrap starting.
I20260709 12:22:44.937762 25471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.942245 25471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107: No bootstrap required, opened a new log
I20260709 12:22:44.944542 25471 raft_consensus.cc:359] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb82d23fc0be4c0ca0398b9d574df107" member_type: VOTER }
I20260709 12:22:44.945009 25471 raft_consensus.cc:385] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.945235 25471 raft_consensus.cc:740] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb82d23fc0be4c0ca0398b9d574df107, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.945880 25471 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb82d23fc0be4c0ca0398b9d574df107" member_type: VOTER }
I20260709 12:22:44.946528 25471 raft_consensus.cc:399] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:44.946784 25471 raft_consensus.cc:493] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:44.947048 25471 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.951588 25471 raft_consensus.cc:515] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb82d23fc0be4c0ca0398b9d574df107" member_type: VOTER }
I20260709 12:22:44.952162 25471 leader_election.cc:304] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb82d23fc0be4c0ca0398b9d574df107; no voters: 
I20260709 12:22:44.953445 25471 leader_election.cc:290] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:44.953830 25474 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.955158 25474 raft_consensus.cc:697] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 1 LEADER]: Becoming Leader. State: Replica: fb82d23fc0be4c0ca0398b9d574df107, State: Running, Role: LEADER
I20260709 12:22:44.955801 25474 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb82d23fc0be4c0ca0398b9d574df107" member_type: VOTER }
I20260709 12:22:44.956598 25471 sys_catalog.cc:565] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:44.960603 25475 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb82d23fc0be4c0ca0398b9d574df107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb82d23fc0be4c0ca0398b9d574df107" member_type: VOTER } }
I20260709 12:22:44.961453 25475 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:44.961163 25476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb82d23fc0be4c0ca0398b9d574df107. Latest consensus state: current_term: 1 leader_uuid: "fb82d23fc0be4c0ca0398b9d574df107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb82d23fc0be4c0ca0398b9d574df107" member_type: VOTER } }
I20260709 12:22:44.962744 25476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:44.968333 25482 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:44.977597 25482 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:44.978652 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:44.986701 25482 catalog_manager.cc:1383] Generated new cluster ID: 78a9a13401c840018426fdd847ac72be
I20260709 12:22:44.987017 25482 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:45.013365 25482 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:45.014676 25482 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:45.027127 25482 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107: Generated new TSK 0
I20260709 12:22:45.027781 25482 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:45.046106 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:45.052598 25492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:45.053902 25493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:45.056335 24862 server_base.cc:1061] running on GCE node
W20260709 12:22:45.056605 25495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:45.057756 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:45.057961 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:45.058192 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599765058173 us; error 0 us; skew 500 ppm
I20260709 12:22:45.058743 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:45.061578 24862 webserver.cc:533] Webserver started at http://127.24.71.129:39369/ using document root <none> and password file <none>
I20260709 12:22:45.062073 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:45.062247 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:45.062491 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:45.063705 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "a2324ad57b524759aa58ebd154b785ae"
format_stamp: "Formatted at 2026-07-09 12:22:45 on dist-test-slave-3zgk"
I20260709 12:22:45.068938 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:45.072677 25500 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.073645 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:22:45.073902 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "a2324ad57b524759aa58ebd154b785ae"
format_stamp: "Formatted at 2026-07-09 12:22:45 on dist-test-slave-3zgk"
I20260709 12:22:45.074314 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:45.085151 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:45.086485 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:45.087934 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:45.090494 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:45.090740 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.091027 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:45.091241 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.136101 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:39055
I20260709 12:22:45.136169 25562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:39055 every 8 connection(s)
I20260709 12:22:45.141239 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:45.148207 25567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:45.149415 25568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:45.153496 25571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:45.153751 24862 server_base.cc:1061] running on GCE node
I20260709 12:22:45.155395 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:45.155670 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:45.155826 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599765155813 us; error 0 us; skew 500 ppm
I20260709 12:22:45.156424 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:45.159710 24862 webserver.cc:533] Webserver started at http://127.24.71.130:39815/ using document root <none> and password file <none>
I20260709 12:22:45.160320 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:45.160501 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:45.160835 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:45.161348 25563 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42771
I20260709 12:22:45.161808 25563 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:45.162212 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "017067fdf9694126b910ddb698f09512"
format_stamp: "Formatted at 2026-07-09 12:22:45 on dist-test-slave-3zgk"
I20260709 12:22:45.162752 25563 heartbeater.cc:507] Master 127.24.71.190:42771 requested a full tablet report, sending...
I20260709 12:22:45.165285 25436 ts_manager.cc:194] Registered new tserver with Master: a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055)
I20260709 12:22:45.167436 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:22:45.167529 25436 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45576
I20260709 12:22:45.171661 25575 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.172643 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:45.172982 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "017067fdf9694126b910ddb698f09512"
format_stamp: "Formatted at 2026-07-09 12:22:45 on dist-test-slave-3zgk"
I20260709 12:22:45.173444 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:45.185487 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:45.186858 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:45.188942 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:45.191442 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:45.191720 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.192005 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:45.192216 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.237169 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:35637
I20260709 12:22:45.237323 25637 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:35637 every 8 connection(s)
I20260709 12:22:45.242533 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:45.252818 25641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:45.255450 25642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:45.256094 24862 server_base.cc:1061] running on GCE node
W20260709 12:22:45.256392 25645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:45.256646 25638 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42771
I20260709 12:22:45.257097 25638 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:45.257730 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:45.257916 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:45.257913 25638 heartbeater.cc:507] Master 127.24.71.190:42771 requested a full tablet report, sending...
I20260709 12:22:45.258304 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599765258263 us; error 0 us; skew 500 ppm
I20260709 12:22:45.259006 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:45.260687 25436 ts_manager.cc:194] Registered new tserver with Master: 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:45.261902 24862 webserver.cc:533] Webserver started at http://127.24.71.131:33199/ using document root <none> and password file <none>
I20260709 12:22:45.262425 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:45.262624 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:45.262591 25436 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45584
I20260709 12:22:45.263116 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:45.264540 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "5e4193cbdd464c2f96e2a7012d691e60"
format_stamp: "Formatted at 2026-07-09 12:22:45 on dist-test-slave-3zgk"
I20260709 12:22:45.269435 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:22:45.273178 25649 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.274171 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:45.274513 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "5e4193cbdd464c2f96e2a7012d691e60"
format_stamp: "Formatted at 2026-07-09 12:22:45 on dist-test-slave-3zgk"
I20260709 12:22:45.274852 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:45.318190 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:45.319640 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:45.321460 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:45.323916 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:45.324218 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.324527 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:45.324728 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:45.371445 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:41445
I20260709 12:22:45.371575 25711 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:41445 every 8 connection(s)
I20260709 12:22:45.389056 25712 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42771
I20260709 12:22:45.389532 25712 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:45.390501 25712 heartbeater.cc:507] Master 127.24.71.190:42771 requested a full tablet report, sending...
I20260709 12:22:45.393105 25436 ts_manager.cc:194] Registered new tserver with Master: 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:45.394024 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018899618s
I20260709 12:22:45.394810 25436 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45600
I20260709 12:22:45.423883 25436 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45612:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:45.427178 25436 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:45.502415 25603 tablet_service.cc:1511] Processing CreateTablet for tablet 06b5b4b43edf4449bc4c3fd0c478ebc4 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:45.503988 25603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b5b4b43edf4449bc4c3fd0c478ebc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.505527 25676 tablet_service.cc:1511] Processing CreateTablet for tablet 482670924d0f4c5cb9626cc251a59e0a (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:45.507030 25676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482670924d0f4c5cb9626cc251a59e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.508689 25674 tablet_service.cc:1511] Processing CreateTablet for tablet 3726e81c7a4948d3a48d9c51df49e932 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:45.509500 25677 tablet_service.cc:1511] Processing CreateTablet for tablet 06b5b4b43edf4449bc4c3fd0c478ebc4 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:45.510093 25674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3726e81c7a4948d3a48d9c51df49e932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.510540 25528 tablet_service.cc:1511] Processing CreateTablet for tablet 06b5b4b43edf4449bc4c3fd0c478ebc4 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:45.510808 25677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b5b4b43edf4449bc4c3fd0c478ebc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.511686 25528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b5b4b43edf4449bc4c3fd0c478ebc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.513454 25675 tablet_service.cc:1511] Processing CreateTablet for tablet eb48973785a94c09a20462dc03b8c5fa (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:45.514863 25675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb48973785a94c09a20462dc03b8c5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.514734 25526 tablet_service.cc:1511] Processing CreateTablet for tablet eb48973785a94c09a20462dc03b8c5fa (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:45.517599 25525 tablet_service.cc:1511] Processing CreateTablet for tablet 3726e81c7a4948d3a48d9c51df49e932 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:45.517771 25673 tablet_service.cc:1511] Processing CreateTablet for tablet c0fc2a6dbdd64698a222003a32680557 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:45.519132 25673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fc2a6dbdd64698a222003a32680557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.513465 25527 tablet_service.cc:1511] Processing CreateTablet for tablet 482670924d0f4c5cb9626cc251a59e0a (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:45.520115 25524 tablet_service.cc:1511] Processing CreateTablet for tablet c0fc2a6dbdd64698a222003a32680557 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:45.520576 25526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb48973785a94c09a20462dc03b8c5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.521884 25524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fc2a6dbdd64698a222003a32680557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.522590 25527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482670924d0f4c5cb9626cc251a59e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.522861 25672 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec865ad451c482d80468abf199358be (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:45.524525 25672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec865ad451c482d80468abf199358be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.528769 25525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3726e81c7a4948d3a48d9c51df49e932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.533756 25523 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec865ad451c482d80468abf199358be (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:45.535290 25523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec865ad451c482d80468abf199358be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.534883 25601 tablet_service.cc:1511] Processing CreateTablet for tablet eb48973785a94c09a20462dc03b8c5fa (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:45.537555 25602 tablet_service.cc:1511] Processing CreateTablet for tablet 482670924d0f4c5cb9626cc251a59e0a (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:45.538874 25600 tablet_service.cc:1511] Processing CreateTablet for tablet 3726e81c7a4948d3a48d9c51df49e932 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:45.537555 25522 tablet_service.cc:1511] Processing CreateTablet for tablet af334351da5349c1870b007d2ab06bab (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:45.540426 25522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af334351da5349c1870b007d2ab06bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.541711 25597 tablet_service.cc:1511] Processing CreateTablet for tablet af334351da5349c1870b007d2ab06bab (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:45.541980 25598 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec865ad451c482d80468abf199358be (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:45.544528 25599 tablet_service.cc:1511] Processing CreateTablet for tablet c0fc2a6dbdd64698a222003a32680557 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:45.546105 25601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb48973785a94c09a20462dc03b8c5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.546777 25597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af334351da5349c1870b007d2ab06bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.547485 25602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482670924d0f4c5cb9626cc251a59e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.548564 25600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3726e81c7a4948d3a48d9c51df49e932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.553941 25671 tablet_service.cc:1511] Processing CreateTablet for tablet af334351da5349c1870b007d2ab06bab (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:45.555387 25671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af334351da5349c1870b007d2ab06bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.556505 25521 tablet_service.cc:1511] Processing CreateTablet for tablet 254b81c1822048b1a238ec3ce9e88aeb (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:45.557991 25521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 254b81c1822048b1a238ec3ce9e88aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.559278 25596 tablet_service.cc:1511] Processing CreateTablet for tablet 254b81c1822048b1a238ec3ce9e88aeb (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:45.561698 25670 tablet_service.cc:1511] Processing CreateTablet for tablet 254b81c1822048b1a238ec3ce9e88aeb (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:45.563694 25520 tablet_service.cc:1511] Processing CreateTablet for tablet 5cc9a86bc0cd4342ab1c311897d8c350 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:45.565095 25520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cc9a86bc0cd4342ab1c311897d8c350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.568609 25670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 254b81c1822048b1a238ec3ce9e88aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.570312 25669 tablet_service.cc:1511] Processing CreateTablet for tablet 5cc9a86bc0cd4342ab1c311897d8c350 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:45.584924 25669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cc9a86bc0cd4342ab1c311897d8c350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.592588 25599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fc2a6dbdd64698a222003a32680557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.596633 25598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec865ad451c482d80468abf199358be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.599718 25595 tablet_service.cc:1511] Processing CreateTablet for tablet 5cc9a86bc0cd4342ab1c311897d8c350 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:45.601392 25595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cc9a86bc0cd4342ab1c311897d8c350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.604274 25596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 254b81c1822048b1a238ec3ce9e88aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.672802 25727 tablet_bootstrap.cc:492] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:45.660183 25524 tablet_service.cc:1511] Processing CreateTablet for tablet 32bd0706d39e4b8bbfbde2fe3664e0a6 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:45.701280 25524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32bd0706d39e4b8bbfbde2fe3664e0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.704548 25728 tablet_bootstrap.cc:492] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:45.708201 25727 tablet_bootstrap.cc:654] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.714985 25601 tablet_service.cc:1511] Processing CreateTablet for tablet 32bd0706d39e4b8bbfbde2fe3664e0a6 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:45.703502 25729 tablet_bootstrap.cc:492] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:45.716382 25601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32bd0706d39e4b8bbfbde2fe3664e0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.716518 25670 tablet_service.cc:1511] Processing CreateTablet for tablet 32bd0706d39e4b8bbfbde2fe3664e0a6 (DEFAULT_TABLE table=test-workload [id=906aa527e4e547a887fae105747ca94d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:45.718875 25670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32bd0706d39e4b8bbfbde2fe3664e0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.729296 25729 tablet_bootstrap.cc:654] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.734992 25728 tablet_bootstrap.cc:654] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.738775 25727 tablet_bootstrap.cc:492] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:45.739508 25727 ts_tablet_manager.cc:1403] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.000s
I20260709 12:22:45.742353 25727 raft_consensus.cc:359] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.743063 25727 raft_consensus.cc:385] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.743361 25727 raft_consensus.cc:740] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.743991 25729 tablet_bootstrap.cc:492] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:45.744763 25729 ts_tablet_manager.cc:1403] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.014s
I20260709 12:22:45.745332 25727 consensus_queue.cc:260] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.749344 25729 raft_consensus.cc:359] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.750608 25729 raft_consensus.cc:385] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.750815 25712 heartbeater.cc:499] Master 127.24.71.190:42771 was elected leader, sending a full tablet report...
I20260709 12:22:45.751374 25727 ts_tablet_manager.cc:1434] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 12:22:45.751695 25729 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.752816 25727 tablet_bootstrap.cc:492] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:45.752660 25729 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.754501 25728 tablet_bootstrap.cc:492] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:45.755074 25728 ts_tablet_manager.cc:1403] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.008s
I20260709 12:22:45.761157 25727 tablet_bootstrap.cc:654] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.758457 25728 raft_consensus.cc:359] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.763039 25728 raft_consensus.cc:385] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.763496 25728 raft_consensus.cc:740] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.764778 25728 consensus_queue.cc:260] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.768885 25638 heartbeater.cc:499] Master 127.24.71.190:42771 was elected leader, sending a full tablet report...
I20260709 12:22:45.772693 25728 ts_tablet_manager.cc:1434] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 12:22:45.773474 25563 heartbeater.cc:499] Master 127.24.71.190:42771 was elected leader, sending a full tablet report...
I20260709 12:22:45.768944 25729 ts_tablet_manager.cc:1434] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.024s	user 0.019s	sys 0.003s
I20260709 12:22:45.776983 25729 tablet_bootstrap.cc:492] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:45.782407 25728 tablet_bootstrap.cc:492] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:45.787949 25729 tablet_bootstrap.cc:654] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.791128 25727 tablet_bootstrap.cc:492] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:45.791663 25727 ts_tablet_manager.cc:1403] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.022s
I20260709 12:22:45.794891 25729 tablet_bootstrap.cc:492] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:45.794762 25727 raft_consensus.cc:359] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.794998 25728 tablet_bootstrap.cc:654] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.795615 25727 raft_consensus.cc:385] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.795697 25729 ts_tablet_manager.cc:1403] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.003s
I20260709 12:22:45.796047 25727 raft_consensus.cc:740] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.796994 25727 consensus_queue.cc:260] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.799180 25727 ts_tablet_manager.cc:1434] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:22:45.800182 25727 tablet_bootstrap.cc:492] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:45.799655 25729 raft_consensus.cc:359] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.800840 25729 raft_consensus.cc:385] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.801348 25729 raft_consensus.cc:740] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.802379 25729 consensus_queue.cc:260] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.805567 25729 ts_tablet_manager.cc:1434] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:45.806900 25729 tablet_bootstrap.cc:492] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:45.807098 25727 tablet_bootstrap.cc:654] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.816514 25729 tablet_bootstrap.cc:654] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.833729 25729 tablet_bootstrap.cc:492] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:45.834520 25729 ts_tablet_manager.cc:1403] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.011s
I20260709 12:22:45.838447 25729 raft_consensus.cc:359] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.839581 25729 raft_consensus.cc:385] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.840076 25729 raft_consensus.cc:740] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.841334 25729 consensus_queue.cc:260] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.843544 25728 tablet_bootstrap.cc:492] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:45.844123 25728 ts_tablet_manager.cc:1403] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.046s
I20260709 12:22:45.844179 25729 ts_tablet_manager.cc:1434] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:45.845337 25729 tablet_bootstrap.cc:492] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:45.846736 25727 tablet_bootstrap.cc:492] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:45.847232 25727 ts_tablet_manager.cc:1403] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.032s
I20260709 12:22:45.847493 25728 raft_consensus.cc:359] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.848321 25728 raft_consensus.cc:385] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.848691 25728 raft_consensus.cc:740] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.849635 25728 consensus_queue.cc:260] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.850502 25727 raft_consensus.cc:359] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:45.851217 25727 raft_consensus.cc:385] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.851493 25727 raft_consensus.cc:740] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.851778 25729 tablet_bootstrap.cc:654] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.852470 25727 consensus_queue.cc:260] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:45.853722 25728 ts_tablet_manager.cc:1434] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:22:45.854771 25728 tablet_bootstrap.cc:492] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:45.862323 25728 tablet_bootstrap.cc:654] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.862496 25727 ts_tablet_manager.cc:1434] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:22:45.864010 25727 tablet_bootstrap.cc:492] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:45.867830 25736 raft_consensus.cc:493] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.868367 25736 raft_consensus.cc:515] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.873718 25727 tablet_bootstrap.cc:654] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:45.876544 25713 tablet.cc:2406] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:45.878379 25736 leader_election.cc:290] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:45.886318 25729 tablet_bootstrap.cc:492] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:45.886873 25729 ts_tablet_manager.cc:1403] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.020s
I20260709 12:22:45.893141 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:45.894393 25729 raft_consensus.cc:359] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
W20260709 12:22:45.894899 25503 leader_election.cc:343] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.895426 25729 raft_consensus.cc:385] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.895864 25729 raft_consensus.cc:740] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.897923 25729 consensus_queue.cc:260] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.900161 25729 ts_tablet_manager.cc:1434] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:22:45.901144 25729 tablet_bootstrap.cc:492] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:45.907651 25729 tablet_bootstrap.cc:654] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.907338 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
W20260709 12:22:45.909305 25502 leader_election.cc:343] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.909819 25502 leader_election.cc:304] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2324ad57b524759aa58ebd154b785ae; no voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:45.911041 25736 raft_consensus.cc:2749] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.911224 25727 tablet_bootstrap.cc:492] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:45.900648 25728 tablet_bootstrap.cc:492] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:45.911804 25727 ts_tablet_manager.cc:1403] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.048s	user 0.021s	sys 0.021s
I20260709 12:22:45.912367 25728 ts_tablet_manager.cc:1403] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.004s
I20260709 12:22:45.915176 25727 raft_consensus.cc:359] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.916015 25727 raft_consensus.cc:385] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.916365 25727 raft_consensus.cc:740] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.916365 25728 raft_consensus.cc:359] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:45.917385 25728 raft_consensus.cc:385] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.917738 25728 raft_consensus.cc:740] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.917572 25727 consensus_queue.cc:260] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.918656 25728 consensus_queue.cc:260] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:45.921240 25728 ts_tablet_manager.cc:1434] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:45.922309 25728 tablet_bootstrap.cc:492] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:45.926504 25727 ts_tablet_manager.cc:1434] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:22:45.930241 25729 tablet_bootstrap.cc:492] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:45.930814 25729 ts_tablet_manager.cc:1403] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.013s
I20260709 12:22:45.932610 25727 tablet_bootstrap.cc:492] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:45.933350 25728 tablet_bootstrap.cc:654] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.934090 25729 raft_consensus.cc:359] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.935001 25729 raft_consensus.cc:385] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.935360 25729 raft_consensus.cc:740] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.936388 25729 consensus_queue.cc:260] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.941985 25729 ts_tablet_manager.cc:1434] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 12:22:45.942073 25727 tablet_bootstrap.cc:654] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.943240 25729 tablet_bootstrap.cc:492] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:45.952157 25734 raft_consensus.cc:493] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.952975 25734 raft_consensus.cc:515] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.952502 25729 tablet_bootstrap.cc:654] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.955580 25727 tablet_bootstrap.cc:492] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:45.955878 25734 leader_election.cc:290] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:45.956357 25727 ts_tablet_manager.cc:1403] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.003s
I20260709 12:22:45.952111 25728 tablet_bootstrap.cc:492] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:45.962970 25727 raft_consensus.cc:359] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.963888 25728 ts_tablet_manager.cc:1403] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.010s
I20260709 12:22:45.965179 25727 raft_consensus.cc:385] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.965577 25727 raft_consensus.cc:740] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.966437 25727 consensus_queue.cc:260] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
W20260709 12:22:45.969023 25484 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:45.972043 25727 ts_tablet_manager.cc:1434] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 12:22:45.970585 25728 raft_consensus.cc:359] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.973268 25727 tablet_bootstrap.cc:492] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:45.986140 25728 raft_consensus.cc:385] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.986527 25728 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.986413 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:45.987288 25687 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 0.
I20260709 12:22:45.988823 25734 raft_consensus.cc:493] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.988616 25728 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.989305 25734 raft_consensus.cc:515] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.990126 25729 tablet_bootstrap.cc:492] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:45.990592 25729 ts_tablet_manager.cc:1403] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.018s
I20260709 12:22:45.991477 25734 leader_election.cc:290] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:45.992367 25728 ts_tablet_manager.cc:1434] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.002s
I20260709 12:22:45.993242 25727 tablet_bootstrap.cc:654] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.993494 25728 tablet_bootstrap.cc:492] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:45.993539 25729 raft_consensus.cc:359] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.994408 25729 raft_consensus.cc:385] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.994709 25729 raft_consensus.cc:740] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.995477 25729 consensus_queue.cc:260] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.997499 25736 raft_consensus.cc:493] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.997807 25729 ts_tablet_manager.cc:1434] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:22:45.997996 25736 raft_consensus.cc:515] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:45.999212 25729 tablet_bootstrap.cc:492] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:46.001263 25728 tablet_bootstrap.cc:654] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.005447 25727 tablet_bootstrap.cc:492] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:46.005570 25733 raft_consensus.cc:493] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.006166 25733 raft_consensus.cc:515] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:46.006693 25727 ts_tablet_manager.cc:1403] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.002s
I20260709 12:22:46.007124 25729 tablet_bootstrap.cc:654] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.009121 25736 leader_election.cc:290] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.017371 25686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.010600 25733 leader_election.cc:290] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055)
I20260709 12:22:46.018054 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.011622 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.013440 25728 tablet_bootstrap.cc:492] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:46.030932 25728 ts_tablet_manager.cc:1403] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.038s	user 0.004s	sys 0.014s
I20260709 12:22:46.038242 25686 raft_consensus.cc:2468] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 0.
I20260709 12:22:46.038404 25727 raft_consensus.cc:359] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.039438 25727 raft_consensus.cc:385] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.040045 25727 raft_consensus.cc:740] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.041129 25727 consensus_queue.cc:260] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.041802 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.046020 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.048454 25613 raft_consensus.cc:2468] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 0.
I20260709 12:22:46.047518 25728 raft_consensus.cc:359] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.051240 25728 raft_consensus.cc:385] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:22:46.051539 25502 leader_election.cc:343] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.053224 25729 tablet_bootstrap.cc:492] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:46.044080 25727 ts_tablet_manager.cc:1434] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.003s
I20260709 12:22:46.038285 25577 leader_election.cc:304] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:46.053740 25729 ts_tablet_manager.cc:1403] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.026s
I20260709 12:22:46.054433 25727 tablet_bootstrap.cc:492] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
W20260709 12:22:46.056954 25576 leader_election.cc:343] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.058362 25734 raft_consensus.cc:2804] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.051573 25728 raft_consensus.cc:740] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.060424 25734 raft_consensus.cc:493] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.058502 25729 raft_consensus.cc:359] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:46.060906 25734 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.059682 25503 leader_election.cc:304] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.060995 25728 consensus_queue.cc:260] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.062142 25736 raft_consensus.cc:2804] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.062525 25736 raft_consensus.cc:493] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.063177 25736 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.063723 25727 tablet_bootstrap.cc:654] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.067764 25733 raft_consensus.cc:493] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.068305 25733 raft_consensus.cc:515] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.070429 25733 leader_election.cc:290] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.071755 25736 raft_consensus.cc:515] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.073040 25734 raft_consensus.cc:515] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.074957 25729 raft_consensus.cc:385] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.075302 25729 raft_consensus.cc:740] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.075415 25736 leader_election.cc:290] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.075683 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.076108 25729 consensus_queue.cc:260] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:46.077997 25734 leader_election.cc:290] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
W20260709 12:22:46.079115 25502 leader_election.cc:343] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.081120 25729 ts_tablet_manager.cc:1434] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.004s
I20260709 12:22:46.082176 25729 tablet_bootstrap.cc:492] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:46.083135 25577 leader_election.cc:304] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:46.091168 25729 tablet_bootstrap.cc:654] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.097576 25759 raft_consensus.cc:2804] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.098209 25759 raft_consensus.cc:493] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.095259 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.098886 25538 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 0.
I20260709 12:22:46.095764 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
W20260709 12:22:46.099309 25576 leader_election.cc:343] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.101259 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.102052 25687 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.102241 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.103113 25537 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.105247 25650 leader_election.cc:304] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
W20260709 12:22:46.106487 25650 leader_election.cc:343] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.107272 25733 raft_consensus.cc:2804] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.107806 25733 raft_consensus.cc:493] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.108327 25733 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.108775 25729 tablet_bootstrap.cc:492] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:46.109328 25729 ts_tablet_manager.cc:1403] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.007s
I20260709 12:22:46.111369 25687 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
I20260709 12:22:46.112526 25759 raft_consensus.cc:3060] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.112195 25729 raft_consensus.cc:359] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.113037 25729 raft_consensus.cc:385] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.113389 25729 raft_consensus.cc:740] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.114173 25729 consensus_queue.cc:260] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.116638 25729 ts_tablet_manager.cc:1434] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 12:22:46.094548 25727 tablet_bootstrap.cc:492] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:46.117731 25729 tablet_bootstrap.cc:492] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:46.120354 25759 raft_consensus.cc:515] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.117764 25727 ts_tablet_manager.cc:1403] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.005s
I20260709 12:22:46.122632 25759 leader_election.cc:290] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.124027 25727 raft_consensus.cc:359] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.124910 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.125553 25613 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.125820 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.126459 25687 raft_consensus.cc:3060] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.127489 25537 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
I20260709 12:22:46.124118 25733 raft_consensus.cc:515] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:46.128579 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.129248 25612 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 0.
I20260709 12:22:46.131387 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.132177 25612 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 0.
I20260709 12:22:46.133083 25729 tablet_bootstrap.cc:654] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.133908 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.131359 25577 leader_election.cc:304] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:46.134294 25613 raft_consensus.cc:2468] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 1.
I20260709 12:22:46.135818 25503 leader_election.cc:304] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.136320 25652 leader_election.cc:304] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.137216 25736 raft_consensus.cc:2804] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.137801 25736 raft_consensus.cc:697] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:46.138777 25736 consensus_queue.cc:237] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.139360 25687 raft_consensus.cc:2468] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
I20260709 12:22:46.125025 25727 raft_consensus.cc:385] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.140434 25727 raft_consensus.cc:740] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.140976 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.142426 25733 leader_election.cc:290] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055)
I20260709 12:22:46.142616 25728 ts_tablet_manager.cc:1434] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.111s	user 0.021s	sys 0.044s
I20260709 12:22:46.143078 25733 raft_consensus.cc:2804] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.143477 25733 raft_consensus.cc:493] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.143788 25728 tablet_bootstrap.cc:492] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:46.143864 25733 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.147303 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.148068 25613 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.141625 25537 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.149986 25727 consensus_queue.cc:260] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
W20260709 12:22:46.145450 25576 leader_election.cc:343] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.152490 25728 tablet_bootstrap.cc:654] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.152856 25759 raft_consensus.cc:2804] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.158598 25759 raft_consensus.cc:697] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.159698 25759 consensus_queue.cc:237] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.160537 25734 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.161190 25734 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.163290 25577 leader_election.cc:304] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.163582 25734 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.162911 25432 catalog_manager.cc:5697] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 0 to 1, leader changed from <none> to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 1 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.164288 25734 raft_consensus.cc:2804] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.164822 25734 raft_consensus.cc:697] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.165930 25734 consensus_queue.cc:237] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.167250 25729 tablet_bootstrap.cc:492] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:46.167963 25729 ts_tablet_manager.cc:1403] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.003s
I20260709 12:22:46.161660 25727 ts_tablet_manager.cc:1434] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.005s
I20260709 12:22:46.171631 25613 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 1.
I20260709 12:22:46.174651 25729 raft_consensus.cc:359] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.176138 25729 raft_consensus.cc:385] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.184691 25729 raft_consensus.cc:740] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.176865 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.185839 25538 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 0.
I20260709 12:22:46.186172 25729 consensus_queue.cc:260] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.174696 25727 tablet_bootstrap.cc:492] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
W20260709 12:22:46.180171 25652 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:35637, user_credentials={real_user=slave}} blocked reactor thread for 44080.4us
I20260709 12:22:46.195648 25652 leader_election.cc:304] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:46.188094 25576 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:46.194409 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.198855 25761 raft_consensus.cc:2804] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.199853 25759 raft_consensus.cc:2804] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.200372 25759 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.200897 25759 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.201239 25734 raft_consensus.cc:493] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.200654 25435 catalog_manager.cc:5697] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.202049 25734 raft_consensus.cc:515] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.203069 25435 catalog_manager.cc:5697] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.205659 25734 leader_election.cc:290] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.207619 25729 ts_tablet_manager.cc:1434] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:22:46.208017 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.212337 25538 raft_consensus.cc:2468] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 0.
I20260709 12:22:46.208644 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
W20260709 12:22:46.209450 25577 leader_election.cc:343] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.210316 25761 raft_consensus.cc:697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:46.212745 25729 tablet_bootstrap.cc:492] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512: Bootstrap starting.
I20260709 12:22:46.214778 25761 consensus_queue.cc:237] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
W20260709 12:22:46.215355 25577 leader_election.cc:343] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.216053 25727 tablet_bootstrap.cc:654] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.216615 25576 leader_election.cc:304] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.211143 25759 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.218466 25734 raft_consensus.cc:2804] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.219033 25734 raft_consensus.cc:493] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.219494 25734 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.220851 25759 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.221177 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.224298 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.224927 25729 tablet_bootstrap.cc:654] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.224849 25733 raft_consensus.cc:515] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
W20260709 12:22:46.226109 25577 leader_election.cc:343] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.225266 25538 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.229311 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.229562 25537 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 1.
I20260709 12:22:46.233384 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.245796 25770 raft_consensus.cc:493] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.246450 25770 raft_consensus.cc:515] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.248101 25728 tablet_bootstrap.cc:492] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:46.232424 25733 leader_election.cc:290] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.251063 25759 raft_consensus.cc:493] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.256713 25759 raft_consensus.cc:515] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.257473 25770 leader_election.cc:290] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.260769 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.261765 25612 raft_consensus.cc:2468] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 0.
I20260709 12:22:46.264290 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cc9a86bc0cd4342ab1c311897d8c350" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.266024 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cc9a86bc0cd4342ab1c311897d8c350" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.265910 25652 leader_election.cc:304] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:46.266841 25687 raft_consensus.cc:2468] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 0.
I20260709 12:22:46.267540 25770 raft_consensus.cc:2804] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.267789 25538 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
I20260709 12:22:46.264348 25759 leader_election.cc:290] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
W20260709 12:22:46.233880 25650 leader_election.cc:343] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.269057 25728 ts_tablet_manager.cc:1403] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.125s	user 0.033s	sys 0.034s
I20260709 12:22:46.268160 25770 raft_consensus.cc:493] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:46.269901 25576 leader_election.cc:343] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.275254 25576 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.275604 25577 leader_election.cc:304] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.276736 25765 raft_consensus.cc:2804] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.273685 25770 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.277887 25759 raft_consensus.cc:2804] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.278661 25759 raft_consensus.cc:493] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.279867 25759 raft_consensus.cc:3060] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.287501 25727 tablet_bootstrap.cc:492] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:46.288175 25727 ts_tablet_manager.cc:1403] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.114s	user 0.018s	sys 0.009s
I20260709 12:22:46.291388 25765 raft_consensus.cc:697] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.292132 25728 raft_consensus.cc:359] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.292557 25765 consensus_queue.cc:237] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.293277 25728 raft_consensus.cc:385] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.293807 25728 raft_consensus.cc:740] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.294893 25728 consensus_queue.cc:260] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.296473 25734 raft_consensus.cc:515] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.303057 25759 raft_consensus.cc:515] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.303545 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.303057 25770 raft_consensus.cc:515] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.305387 25727 raft_consensus.cc:359] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.305644 25734 leader_election.cc:290] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.306032 25727 raft_consensus.cc:385] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.306366 25727 raft_consensus.cc:740] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.307186 25727 consensus_queue.cc:260] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
W20260709 12:22:46.307780 25577 leader_election.cc:343] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.308590 25770 leader_election.cc:290] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.308984 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.309024 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.309638 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.309787 25538 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.311630 25759 leader_election.cc:290] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.312151 25728 ts_tablet_manager.cc:1434] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 12:22:46.314215 25728 tablet_bootstrap.cc:492] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:46.314982 25727 ts_tablet_manager.cc:1434] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:22:46.316213 25727 tablet_bootstrap.cc:492] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:46.318503 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.309769 25612 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.322947 25727 tablet_bootstrap.cc:654] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.323367 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cc9a86bc0cd4342ab1c311897d8c350" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.324013 25537 raft_consensus.cc:3060] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.312168 25613 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:46.325634 25650 leader_election.cc:343] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.322649 25538 raft_consensus.cc:2468] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
W20260709 12:22:46.327497 25650 leader_election.cc:343] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:46.327330 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cc9a86bc0cd4342ab1c311897d8c350" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.328177 25687 raft_consensus.cc:3060] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.328341 25576 leader_election.cc:304] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.329480 25729 tablet_bootstrap.cc:492] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512: No bootstrap required, opened a new log
I20260709 12:22:46.329627 25759 raft_consensus.cc:2804] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.330101 25759 raft_consensus.cc:697] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.329934 25729 ts_tablet_manager.cc:1403] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512: Time spent bootstrapping tablet: real 0.117s	user 0.031s	sys 0.010s
I20260709 12:22:46.330981 25759 consensus_queue.cc:237] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.338758 25613 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 1.
I20260709 12:22:46.341269 25652 leader_election.cc:304] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.342473 25770 raft_consensus.cc:2804] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.343052 25770 raft_consensus.cc:697] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:46.343876 25770 consensus_queue.cc:237] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.339190 25612 raft_consensus.cc:2468] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 1.
I20260709 12:22:46.351135 25652 leader_election.cc:304] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.356112 25770 raft_consensus.cc:2804] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.356781 25770 raft_consensus.cc:697] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:46.358103 25729 raft_consensus.cc:359] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.358858 25729 raft_consensus.cc:385] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.358408 25770 consensus_queue.cc:237] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.359320 25729 raft_consensus.cc:740] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.359987 25729 consensus_queue.cc:260] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.360124 25537 raft_consensus.cc:2468] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
I20260709 12:22:46.365307 25729 ts_tablet_manager.cc:1434] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 12:22:46.366447 25576 leader_election.cc:304] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:46.367161 25759 raft_consensus.cc:2804] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.367748 25728 tablet_bootstrap.cc:654] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.367882 25759 raft_consensus.cc:697] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.368806 25759 consensus_queue.cc:237] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.372468 25727 tablet_bootstrap.cc:492] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:46.372954 25727 ts_tablet_manager.cc:1403] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.003s
I20260709 12:22:46.378520 25727 raft_consensus.cc:359] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.379415 25727 raft_consensus.cc:385] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.379830 25727 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.376916 25687 raft_consensus.cc:2468] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 1.
I20260709 12:22:46.380730 25727 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.393258 25727 ts_tablet_manager.cc:1434] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.005s
I20260709 12:22:46.394762 25727 tablet_bootstrap.cc:492] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60: Bootstrap starting.
I20260709 12:22:46.390775 25434 catalog_manager.cc:5697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.401484 25434 catalog_manager.cc:5697] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:46.405189 25727 tablet_bootstrap.cc:654] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.404333 25434 catalog_manager.cc:5697] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:46.410953 25728 tablet_bootstrap.cc:492] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:46.411391 25727 tablet_bootstrap.cc:492] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60: No bootstrap required, opened a new log
I20260709 12:22:46.411507 25728 ts_tablet_manager.cc:1403] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.098s	user 0.013s	sys 0.015s
I20260709 12:22:46.411913 25727 ts_tablet_manager.cc:1403] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.003s
I20260709 12:22:46.414921 25727 raft_consensus.cc:359] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.415674 25727 raft_consensus.cc:385] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.416010 25727 raft_consensus.cc:740] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.416818 25727 consensus_queue.cc:260] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.417129 25728 raft_consensus.cc:359] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.417873 25728 raft_consensus.cc:385] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.418215 25728 raft_consensus.cc:740] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.419096 25727 ts_tablet_manager.cc:1434] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:22:46.418751 25435 catalog_manager.cc:5697] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.419677 25728 consensus_queue.cc:260] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.420708 25435 catalog_manager.cc:5697] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.422441 25435 catalog_manager.cc:5697] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.423923 25728 ts_tablet_manager.cc:1434] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.007s
I20260709 12:22:46.425084 25728 tablet_bootstrap.cc:492] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:46.432379 25728 tablet_bootstrap.cc:654] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.443058 25728 tablet_bootstrap.cc:492] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:46.443503 25728 ts_tablet_manager.cc:1403] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:22:46.445477 25728 raft_consensus.cc:359] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.445983 25728 raft_consensus.cc:385] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.446205 25728 raft_consensus.cc:740] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.446781 25728 consensus_queue.cc:260] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.448464 25728 ts_tablet_manager.cc:1434] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:22:46.449569 25728 tablet_bootstrap.cc:492] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:46.455257 25728 tablet_bootstrap.cc:654] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.459383 25728 tablet_bootstrap.cc:492] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:46.459790 25728 ts_tablet_manager.cc:1403] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.010s	user 0.001s	sys 0.008s
I20260709 12:22:46.461663 25728 raft_consensus.cc:359] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.462152 25728 raft_consensus.cc:385] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.462416 25728 raft_consensus.cc:740] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.462991 25728 consensus_queue.cc:260] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.464473 25728 ts_tablet_manager.cc:1434] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.003s
I20260709 12:22:46.465554 25728 tablet_bootstrap.cc:492] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: Bootstrap starting.
I20260709 12:22:46.465582 25770 raft_consensus.cc:493] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.466122 25770 raft_consensus.cc:515] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.468326 25770 leader_election.cc:290] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.469014 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.469913 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.470716 25612 raft_consensus.cc:2393] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e4193cbdd464c2f96e2a7012d691e60 in current term 1: Already voted for candidate a2324ad57b524759aa58ebd154b785ae in this term.
I20260709 12:22:46.471663 25652 leader_election.cc:304] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60; no voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.472045 25728 tablet_bootstrap.cc:654] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:46.472515 25770 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.476588 25728 tablet_bootstrap.cc:492] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: No bootstrap required, opened a new log
I20260709 12:22:46.477048 25728 ts_tablet_manager.cc:1403] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 12:22:46.479147 25728 raft_consensus.cc:359] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.479420 25770 raft_consensus.cc:2749] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:46.479650 25728 raft_consensus.cc:385] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:46.479977 25728 raft_consensus.cc:740] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Initialized, Role: FOLLOWER
I20260709 12:22:46.480520 25728 consensus_queue.cc:260] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.482270 25728 ts_tablet_manager.cc:1434] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:46.482880 25736 raft_consensus.cc:493] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.483337 25736 raft_consensus.cc:515] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.485814 25736 leader_election.cc:290] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.486757 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.487354 25612 raft_consensus.cc:2468] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 0.
I20260709 12:22:46.487208 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.487951 25687 raft_consensus.cc:2468] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 0.
I20260709 12:22:46.488411 25503 leader_election.cc:304] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:46.489171 25736 raft_consensus.cc:2804] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:46.489734 25736 raft_consensus.cc:493] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.490059 25736 raft_consensus.cc:3060] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.494901 25736 raft_consensus.cc:515] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.496814 25736 leader_election.cc:290] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.497836 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.497999 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.498512 25612 raft_consensus.cc:3060] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.498590 25687 raft_consensus.cc:3060] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.504804 25687 raft_consensus.cc:2468] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 1.
I20260709 12:22:46.504952 25612 raft_consensus.cc:2468] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 1.
I20260709 12:22:46.505854 25502 leader_election.cc:304] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:46.506654 25736 raft_consensus.cc:2804] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:46.507161 25736 raft_consensus.cc:697] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:46.507876 25736 consensus_queue.cc:237] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.515542 25432 catalog_manager.cc:5697] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 0 to 1, leader changed from <none> to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 1 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.532703 25736 consensus_queue.cc:1048] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.532814 25770 raft_consensus.cc:493] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.533638 25770 raft_consensus.cc:515] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.536609 25770 leader_election.cc:290] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.536831 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.537199 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.537878 25537 raft_consensus.cc:2393] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e4193cbdd464c2f96e2a7012d691e60 in current term 1: Already voted for candidate 017067fdf9694126b910ddb698f09512 in this term.
I20260709 12:22:46.538268 25736 raft_consensus.cc:493] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.539057 25736 raft_consensus.cc:515] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.539276 25650 leader_election.cc:304] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60; no voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.541115 25770 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.542382 25736 leader_election.cc:290] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.548441 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.549312 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.552196 25612 raft_consensus.cc:2393] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2324ad57b524759aa58ebd154b785ae in current term 1: Already voted for candidate 5e4193cbdd464c2f96e2a7012d691e60 in this term.
I20260709 12:22:46.552067 25759 consensus_queue.cc:1048] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.553709 25503 leader_election.cc:304] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2324ad57b524759aa58ebd154b785ae; no voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.553864 25770 raft_consensus.cc:2749] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:46.555994 25736 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.559077 25537 raft_consensus.cc:3060] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.565568 25736 raft_consensus.cc:2749] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:46.577816 25792 consensus_queue.cc:1048] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:46.580086 25765 consensus_queue.cc:1048] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.623085 25765 consensus_queue.cc:1048] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.634163 25774 consensus_queue.cc:1048] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:46.638896 25770 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.666790 25770 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.691504 25686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:56230
I20260709 12:22:46.692196 25686 raft_consensus.cc:606] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.708001 25612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:22:46.708578 25612 raft_consensus.cc:606] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.713666 25802 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.714185 25802 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.714612 25612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:22:46.715234 25612 raft_consensus.cc:606] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.716703 25802 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.716995 25537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2324ad57b524759aa58ebd154b785ae" is_pre_election: true
I20260709 12:22:46.717403 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.717861 25537 raft_consensus.cc:2393] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e4193cbdd464c2f96e2a7012d691e60 in current term 1: Already voted for candidate 017067fdf9694126b910ddb698f09512 in this term.
I20260709 12:22:46.719046 25792 raft_consensus.cc:493] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.719082 25650 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60; no voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.720012 25802 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.719664 25792 raft_consensus.cc:515] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.721952 25792 leader_election.cc:290] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.722589 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "017067fdf9694126b910ddb698f09512" is_pre_election: true
I20260709 12:22:46.723170 25612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:22:46.723537 25613 raft_consensus.cc:2393] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2324ad57b524759aa58ebd154b785ae in current term 1: Already voted for candidate 5e4193cbdd464c2f96e2a7012d691e60 in this term.
I20260709 12:22:46.723399 25686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60" is_pre_election: true
I20260709 12:22:46.723780 25612 raft_consensus.cc:606] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.725476 25502 leader_election.cc:304] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2324ad57b524759aa58ebd154b785ae; no voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.726161 25792 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:46.727130 25802 raft_consensus.cc:2749] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:46.731351 25612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:22:46.731624 25774 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.731989 25612 raft_consensus.cc:606] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:22:46.733672 25792 raft_consensus.cc:2749] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:46.735651 25802 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.740000 25765 raft_consensus.cc:993] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:46.740499 25765 raft_consensus.cc:1081] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:46.744503 25687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ec865ad451c482d80468abf199358be"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
mode: GRACEFUL
new_leader_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:56230
I20260709 12:22:46.745126 25687 raft_consensus.cc:606] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Received request to transfer leadership to TS 017067fdf9694126b910ddb698f09512
I20260709 12:22:46.751462 25734 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.755633 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:22:46.756384 25536 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.756793 25536 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.764204 25536 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.766645 25536 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.768728 25687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.770319 25774 consensus_queue.cc:1048] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:46.770929 25761 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.792013 25774 consensus_queue.cc:1048] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:22:46.801721 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.804109 25613 raft_consensus.cc:3055] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.804556 25613 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.805707 25613 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.807264 25613 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.821867 25613 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:46.825040 25503 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.826598 25792 raft_consensus.cc:2804] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.827086 25792 raft_consensus.cc:697] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:46.827891 25792 consensus_queue.cc:237] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.833866 25759 consensus_queue.cc:1048] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.839063 25432 catalog_manager.cc:5697] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 1 to 2, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.844614 25774 consensus_queue.cc:1048] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.848047 25759 raft_consensus.cc:993] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:46.848630 25759 raft_consensus.cc:1081] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:46.850387 25538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:22:46.850967 25538 raft_consensus.cc:493] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.851346 25538 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.854260 25537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af334351da5349c1870b007d2ab06bab"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:48964
I20260709 12:22:46.854897 25537 raft_consensus.cc:606] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Received request to transfer leadership to TS 017067fdf9694126b910ddb698f09512
I20260709 12:22:46.859776 25538 raft_consensus.cc:515] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.861380 25685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
mode: GRACEFUL
new_leader_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:56230
I20260709 12:22:46.861994 25685 raft_consensus.cc:606] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Received request to transfer leadership to TS 017067fdf9694126b910ddb698f09512
I20260709 12:22:46.862560 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.863296 25612 raft_consensus.cc:3055] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.863313 25538 leader_election.cc:290] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.863765 25612 raft_consensus.cc:740] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.864621 25612 consensus_queue.cc:260] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.865773 25612 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.865700 25686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.866411 25686 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.872403 25538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:48964
I20260709 12:22:46.873154 25538 raft_consensus.cc:606] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Received request to transfer leadership to TS 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:46.873314 25612 raft_consensus.cc:2468] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:46.874665 25503 leader_election.cc:304] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:46.875775 25792 raft_consensus.cc:2804] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.876545 25792 raft_consensus.cc:697] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:46.876941 25686 raft_consensus.cc:2468] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:46.877478 25792 consensus_queue.cc:237] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.885141 25434 catalog_manager.cc:5697] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 1 to 2, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.907150 25761 consensus_queue.cc:1048] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.914069 25733 consensus_queue.cc:1048] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.923780 25770 raft_consensus.cc:993] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: : Instructing follower 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:22:46.924289 25761 raft_consensus.cc:1081] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Signalling peer 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:22:46.926308 25612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ec865ad451c482d80468abf199358be"
dest_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:52858
I20260709 12:22:46.926882 25612 raft_consensus.cc:493] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.927215 25612 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.932171 25612 raft_consensus.cc:515] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.934626 25612 leader_election.cc:290] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 2 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:46.935210 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.935603 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:46.935837 25536 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.936193 25685 raft_consensus.cc:3055] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.936555 25685 raft_consensus.cc:740] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:46.937368 25685 consensus_queue.cc:260] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.938284 25792 consensus_queue.cc:1048] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.938568 25685 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.943277 25536 raft_consensus.cc:2468] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 2.
I20260709 12:22:46.944577 25576 leader_election.cc:304] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:46.945474 25759 raft_consensus.cc:2804] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.945957 25759 raft_consensus.cc:697] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:46.946851 25759 consensus_queue.cc:237] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.946487 25685 raft_consensus.cc:2468] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 2.
I20260709 12:22:46.950985 25736 raft_consensus.cc:993] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:22:46.951437 25736 raft_consensus.cc:1081] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Signalling peer 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:22:46.955282 25685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:56216
I20260709 12:22:46.956684 25685 raft_consensus.cc:493] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.957119 25685 raft_consensus.cc:3060] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.956475 25434 catalog_manager.cc:5697] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 1 to 2, leader changed from 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131) to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.960031 25792 consensus_queue.cc:1048] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.965206 25685 raft_consensus.cc:515] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.970320 25685 leader_election.cc:290] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 2 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:46.971511 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:46.972141 25612 raft_consensus.cc:3060] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.973389 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bd0706d39e4b8bbfbde2fe3664e0a6" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:46.973980 25536 raft_consensus.cc:3055] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.974345 25536 raft_consensus.cc:740] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:46.975152 25536 consensus_queue.cc:260] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.976538 25536 raft_consensus.cc:3060] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.979871 25612 raft_consensus.cc:2468] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 2.
I20260709 12:22:46.981082 25652 leader_election.cc:304] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:46.981907 25761 raft_consensus.cc:2804] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.982370 25761 raft_consensus.cc:697] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:46.983217 25761 consensus_queue.cc:237] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:46.984551 25536 raft_consensus.cc:2468] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 2.
I20260709 12:22:46.990091 25432 catalog_manager.cc:5697] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 1 to 2, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.074640 25765 raft_consensus.cc:993] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:47.075093 25759 raft_consensus.cc:1081] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:47.076812 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:22:47.077426 25536 raft_consensus.cc:493] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.077804 25536 raft_consensus.cc:3060] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.082727 25536 raft_consensus.cc:515] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.084517 25536 leader_election.cc:290] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:47.085566 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:47.085723 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3726e81c7a4948d3a48d9c51df49e932" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:47.086093 25612 raft_consensus.cc:3055] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.086251 25685 raft_consensus.cc:3060] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.086431 25612 raft_consensus.cc:740] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:47.087177 25612 consensus_queue.cc:260] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.088094 25612 raft_consensus.cc:3060] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.092445 25685 raft_consensus.cc:2468] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:47.093873 25759 raft_consensus.cc:993] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:47.093752 25502 leader_election.cc:304] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:47.094331 25765 raft_consensus.cc:1081] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:47.094789 25757 raft_consensus.cc:2804] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.095299 25757 raft_consensus.cc:697] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:47.095392 25612 raft_consensus.cc:2468] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:47.096490 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:22:47.096177 25757 consensus_queue.cc:237] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.097056 25536 raft_consensus.cc:493] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.097460 25536 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.103855 25536 raft_consensus.cc:515] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.103758 25432 catalog_manager.cc:5697] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 1 to 2, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.106083 25536 leader_election.cc:290] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:47.106799 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:47.107146 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:47.107431 25612 raft_consensus.cc:3055] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.107911 25685 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.107908 25612 raft_consensus.cc:740] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:47.109020 25612 consensus_queue.cc:260] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.110278 25612 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.114943 25685 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:47.116080 25502 leader_election.cc:304] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:47.116104 25612 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:47.117033 25757 raft_consensus.cc:2804] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.117497 25757 raft_consensus.cc:697] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:47.118273 25757 consensus_queue.cc:237] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.124719 25432 catalog_manager.cc:5697] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 1 to 2, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.134943 25792 raft_consensus.cc:993] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:22:47.135394 25792 raft_consensus.cc:1081] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Signalling peer 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:22:47.136981 25612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af334351da5349c1870b007d2ab06bab"
dest_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:52850
I20260709 12:22:47.137530 25612 raft_consensus.cc:493] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.137782 25612 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.142627 25612 raft_consensus.cc:515] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.144224 25612 leader_election.cc:290] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 2 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:47.145368 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:47.145454 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:47.146039 25536 raft_consensus.cc:3055] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.146106 25685 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.146365 25536 raft_consensus.cc:740] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:47.147151 25536 consensus_queue.cc:260] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.148085 25536 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.152738 25685 raft_consensus.cc:2468] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 2.
I20260709 12:22:47.153342 25536 raft_consensus.cc:2468] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 2.
I20260709 12:22:47.153945 25577 leader_election.cc:304] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:47.154768 25765 raft_consensus.cc:2804] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.155351 25765 raft_consensus.cc:697] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:47.156414 25765 consensus_queue.cc:237] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.163648 25432 catalog_manager.cc:5697] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 1 to 2, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.195223 25761 raft_consensus.cc:993] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:47.195705 25802 raft_consensus.cc:1081] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:22:47.197397 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48958
I20260709 12:22:47.197981 25536 raft_consensus.cc:493] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.198405 25536 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.203631 25536 raft_consensus.cc:515] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:47.205374 25536 leader_election.cc:290] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:47.206328 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:47.206636 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:47.206975 25612 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.207195 25685 raft_consensus.cc:3055] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.207556 25685 raft_consensus.cc:740] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:47.208477 25685 consensus_queue.cc:260] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:47.209709 25685 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.214128 25612 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:47.215390 25503 leader_election.cc:304] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:47.216307 25792 raft_consensus.cc:2804] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.216734 25792 raft_consensus.cc:697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:47.216804 25685 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 2.
I20260709 12:22:47.217605 25792 consensus_queue.cc:237] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:47.224030 25432 catalog_manager.cc:5697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 1 to 2, leader changed from 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.267409 25612 raft_consensus.cc:1275] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.268834 25792 consensus_queue.cc:1048] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.278666 25685 raft_consensus.cc:1275] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.280071 25736 consensus_queue.cc:1048] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.294763 25685 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.299142 25761 raft_consensus.cc:993] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: : Instructing follower 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:22:47.299669 25802 raft_consensus.cc:1081] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Signalling peer 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:22:47.301347 25612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb"
dest_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:52858
I20260709 12:22:47.301975 25612 raft_consensus.cc:493] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.302172 25685 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.302380 25612 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.303596 25736 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.308416 25612 raft_consensus.cc:515] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.312107 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:47.312705 25536 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.313705 25612 leader_election.cc:290] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 2 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:22:47.316862 25612 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.315946 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:22:47.318220 25757 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.318437 25685 raft_consensus.cc:3055] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.318821 25685 raft_consensus.cc:740] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:47.319737 25685 consensus_queue.cc:260] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.321343 25536 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 2.
I20260709 12:22:47.321007 25685 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.325045 25576 leader_election.cc:304] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:22:47.326103 25765 raft_consensus.cc:2804] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.326581 25765 raft_consensus.cc:697] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:22:47.327389 25765 consensus_queue.cc:237] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:47.331135 25685 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 2.
I20260709 12:22:47.335552 25432 catalog_manager.cc:5697] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 1 to 2, leader changed from 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131) to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.517100 25536 raft_consensus.cc:1275] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.518376 25765 consensus_queue.cc:1048] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.526477 25685 raft_consensus.cc:1275] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.526477 25538 raft_consensus.cc:1275] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.528374 25774 consensus_queue.cc:1048] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.530113 25802 consensus_queue.cc:1048] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.539857 25612 raft_consensus.cc:1275] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.541742 25770 consensus_queue.cc:1048] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.613440 25536 raft_consensus.cc:1275] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.614869 25774 consensus_queue.cc:1048] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.624382 25685 raft_consensus.cc:1275] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.626444 25774 consensus_queue.cc:1048] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.651790 25685 raft_consensus.cc:1275] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.653235 25757 consensus_queue.cc:1048] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.657845 25686 raft_consensus.cc:1275] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.659420 25757 consensus_queue.cc:1048] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.663095 25612 raft_consensus.cc:1275] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.664937 25757 consensus_queue.cc:1048] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.677024 25612 raft_consensus.cc:1275] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.678581 25757 consensus_queue.cc:1048] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.731725 25685 raft_consensus.cc:1275] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.733079 25757 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.742496 25612 raft_consensus.cc:1275] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.744083 25757 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.804884 25536 raft_consensus.cc:1275] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.806420 25774 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.815686 25685 raft_consensus.cc:1275] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:47.817653 25765 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:49.910960 24862 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:22:49.911758 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:49.929522 25536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:48968
I20260709 12:22:49.930184 25536 raft_consensus.cc:606] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Received request to transfer leadership to TS 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:49.933483 25536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:48968
I20260709 12:22:49.934123 25536 raft_consensus.cc:606] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Received request to transfer leadership to TS 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:22:49.935225 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:49.935710 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:50.094022 25736 raft_consensus.cc:993] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:22:50.094555 25832 raft_consensus.cc:1081] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Signalling peer 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:22:50.096122 25686 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:56216
I20260709 12:22:50.096714 25686 raft_consensus.cc:493] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:50.097012 25686 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.101989 25686 raft_consensus.cc:515] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:50.103771 25686 leader_election.cc:290] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 3 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055)
I20260709 12:22:50.104632 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:50.104919 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:50.105340 25612 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.105450 25536 raft_consensus.cc:3055] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:50.105875 25536 raft_consensus.cc:740] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:50.106581 25536 consensus_queue.cc:260] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:50.107501 25536 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.109767 25612 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 3.
I20260709 12:22:50.110880 25652 leader_election.cc:304] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:50.111637 25831 raft_consensus.cc:2804] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:50.113457 25536 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 3.
I20260709 12:22:50.113396 25831 raft_consensus.cc:697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:50.114742 25831 consensus_queue.cc:237] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:22:50.121775 25434 catalog_manager.cc:5697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 2 to 3, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:50.312673 25736 raft_consensus.cc:993] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:22:50.315412 25832 raft_consensus.cc:1081] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Signalling peer 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:22:50.317237 25686 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:56216
I20260709 12:22:50.317891 25686 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:50.318325 25686 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.326288 25686 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:50.328778 25686 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 3 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:22:50.329366 25536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:22:50.329815 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:22:50.329962 25536 raft_consensus.cc:3055] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:50.330363 25612 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.330639 25536 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:22:50.331408 25536 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:50.332413 25536 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.335218 25612 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 3.
I20260709 12:22:50.336347 25652 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:22:50.337077 25831 raft_consensus.cc:2804] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:50.337505 25831 raft_consensus.cc:697] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:22:50.337625 25536 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 3.
I20260709 12:22:50.338279 25831 consensus_queue.cc:237] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:22:50.345268 25434 catalog_manager.cc:5697] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 2 to 3, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:50.513489 25536 raft_consensus.cc:1275] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:50.514879 25831 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:50.531888 25612 raft_consensus.cc:1275] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:50.534816 25845 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:50.757256 25612 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:50.758704 25845 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:50.767136 25536 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:50.768667 25831 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:50.936765 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.942185 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:50.942548 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:51.943461 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:51.949252 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:51.949750 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:52.950949 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.958278 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:52.958750 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:53.959821 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:53.965152 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:53.965632 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:54.966724 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:54.972029 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:54.972491 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:55.973456 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:55.980758 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:55.981323 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:56.982853 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:56.989065 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:56.989421 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:57.990614 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:57.995965 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:57.996426 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:58.997434 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:59.003744 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:59.004308 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:00.039880 24862 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:23:00.044922 25686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:56230
I20260709 12:23:00.045861 25686 raft_consensus.cc:606] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:23:00.055011 25686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:56230
I20260709 12:23:00.055570 25686 raft_consensus.cc:606] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:23:00.067097 25613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ec865ad451c482d80468abf199358be"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:23:00.067747 25613 raft_consensus.cc:606] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:23:00.073221 25613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af334351da5349c1870b007d2ab06bab"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:23:00.073786 25613 raft_consensus.cc:606] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:23:00.079192 25613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb"
dest_uuid: "017067fdf9694126b910ddb698f09512"
mode: GRACEFUL
new_leader_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:52866
I20260709 12:23:00.079805 25613 raft_consensus.cc:606] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Received request to transfer leadership to TS a2324ad57b524759aa58ebd154b785ae
I20260709 12:23:00.234858 25845 raft_consensus.cc:993] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.235311 25845 raft_consensus.cc:1081] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.236977 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48958
I20260709 12:23:00.237428 25536 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:00.237792 25536 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:00.243371 25536 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.245218 25536 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 4 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:00.246470 25686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:00.246410 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:00.247275 25686 raft_consensus.cc:3055] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:00.247437 25613 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:00.247562 25686 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:23:00.248548 25686 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.249567 25686 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:00.253191 25613 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 4.
I20260709 12:23:00.254271 25686 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 4.
I20260709 12:23:00.254520 25503 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:23:00.255559 25736 raft_consensus.cc:2804] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:00.256577 25736 raft_consensus.cc:697] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:00.257337 25736 consensus_queue.cc:237] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.265650 25432 catalog_manager.cc:5697] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 3 to 4, leader changed from 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 4 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.538731 25855 raft_consensus.cc:993] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.539239 25855 raft_consensus.cc:1081] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.540385 25855 raft_consensus.cc:993] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.540899 25869 raft_consensus.cc:1081] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.540962 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af334351da5349c1870b007d2ab06bab"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:23:00.541651 25536 raft_consensus.cc:493] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:00.541941 25536 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.542575 25538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:23:00.543097 25538 raft_consensus.cc:493] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:00.543496 25538 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.549032 25536 raft_consensus.cc:515] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.550576 25538 raft_consensus.cc:515] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.551128 25536 leader_election.cc:290] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 3 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:00.552136 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:00.552477 25686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af334351da5349c1870b007d2ab06bab" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:00.552769 25538 leader_election.cc:290] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 3 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:00.552891 25613 raft_consensus.cc:3055] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:00.553040 25686 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.553467 25613 raft_consensus.cc:740] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:23:00.553788 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:00.554314 25612 raft_consensus.cc:3055] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:00.554617 25612 raft_consensus.cc:740] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:23:00.555320 25612 consensus_queue.cc:260] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.555732 25613 consensus_queue.cc:260] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.556618 25612 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.557304 25613 raft_consensus.cc:3060] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.555330 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:00.559073 25685 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.560083 25686 raft_consensus.cc:2468] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 3.
I20260709 12:23:00.561725 25502 leader_election.cc:304] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:23:00.562592 25736 raft_consensus.cc:2804] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:00.563102 25736 raft_consensus.cc:697] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:00.563992 25736 consensus_queue.cc:237] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.566001 25613 raft_consensus.cc:2468] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 3.
I20260709 12:23:00.566426 25685 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 3.
I20260709 12:23:00.569568 25502 leader_election.cc:304] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:23:00.569836 25612 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 3.
I20260709 12:23:00.570529 25736 raft_consensus.cc:2804] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:00.571036 25736 raft_consensus.cc:697] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:00.571965 25736 consensus_queue.cc:237] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.573184 25869 raft_consensus.cc:993] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.573673 25869 raft_consensus.cc:1081] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.574185 25432 catalog_manager.cc:5697] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 2 to 3, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 3 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.575963 25861 raft_consensus.cc:993] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: : Instructing follower a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.576112 25536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ec865ad451c482d80468abf199358be"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48956
I20260709 12:23:00.576449 25861 raft_consensus.cc:1081] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Signalling peer a2324ad57b524759aa58ebd154b785ae to start an election
I20260709 12:23:00.576620 25536 raft_consensus.cc:493] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:00.576983 25536 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.578171 25538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
 from {username='slave'} at 127.0.0.1:48958
I20260709 12:23:00.578748 25538 raft_consensus.cc:493] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:00.579164 25538 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:00.586091 25536 raft_consensus.cc:515] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.586272 25538 raft_consensus.cc:515] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:23:00.588980 25536 leader_election.cc:290] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 3 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:00.589262 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:00.589673 25613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:00.589815 25612 raft_consensus.cc:3055] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:00.590353 25612 raft_consensus.cc:740] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:23:00.589622 25432 catalog_manager.cc:5697] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 2 to 3, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 3 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.590921 25538 leader_election.cc:290] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 4 election: Requested vote from peers 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:00.590406 25613 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:00.589896 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b5b4b43edf4449bc4c3fd0c478ebc4" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:00.592504 25685 raft_consensus.cc:3055] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:00.592803 25685 raft_consensus.cc:740] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:23:00.592998 25686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec865ad451c482d80468abf199358be" candidate_uuid: "a2324ad57b524759aa58ebd154b785ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:00.593605 25686 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.593559 25685 consensus_queue.cc:260] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:23:00.594907 25685 raft_consensus.cc:3060] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:00.598186 25613 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 4.
I20260709 12:23:00.598870 25612 consensus_queue.cc:260] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.599584 25503 leader_election.cc:304] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:23:00.600029 25686 raft_consensus.cc:2468] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 3.
I20260709 12:23:00.600370 25736 raft_consensus.cc:2804] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:00.600083 25612 raft_consensus.cc:3060] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.600978 25736 raft_consensus.cc:697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:00.602020 25502 leader_election.cc:304] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4193cbdd464c2f96e2a7012d691e60, a2324ad57b524759aa58ebd154b785ae; no voters: 
I20260709 12:23:00.601778 25736 consensus_queue.cc:237] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } }
I20260709 12:23:00.602907 25862 raft_consensus.cc:2804] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:00.603768 25685 raft_consensus.cc:2468] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 4.
I20260709 12:23:00.605374 25862 raft_consensus.cc:697] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Becoming Leader. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:00.606676 25862 consensus_queue.cc:237] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:00.610016 25612 raft_consensus.cc:2468] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2324ad57b524759aa58ebd154b785ae in term 3.
I20260709 12:23:00.617635 25432 catalog_manager.cc:5697] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 3 to 4, leader changed from 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 4 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:00.630056 25432 catalog_manager.cc:5697] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae reported cstate change: term changed from 2 to 3, leader changed from 017067fdf9694126b910ddb698f09512 (127.24.71.130) to a2324ad57b524759aa58ebd154b785ae (127.24.71.129). New cstate: current_term: 3 leader_uuid: "a2324ad57b524759aa58ebd154b785ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.665325 25612 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:00.666559 25862 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:00.680444 25686 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:00.681874 25736 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:00.978130 25612 raft_consensus.cc:1275] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:00.979544 25736 consensus_queue.cc:1048] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:00.989876 25686 raft_consensus.cc:1275] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:00.990598 25612 raft_consensus.cc:1275] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:00.991451 25862 consensus_queue.cc:1048] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:00.991883 25736 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:01.004289 25686 raft_consensus.cc:1275] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:01.006327 25873 consensus_queue.cc:1048] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:01.072284 25686 raft_consensus.cc:1275] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:01.073135 25685 raft_consensus.cc:1275] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:01.073657 25873 consensus_queue.cc:1048] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:01.074496 25736 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:01.087136 25612 raft_consensus.cc:1275] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:01.089488 25612 raft_consensus.cc:1275] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Refusing update from remote peer a2324ad57b524759aa58ebd154b785ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:01.089668 25736 consensus_queue.cc:1048] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:01.090839 25736 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:03.126829 24862 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:23:03.127635 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:03.135454 25538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:48968
I20260709 12:23:03.136274 25538 raft_consensus.cc:606] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Received request to transfer leadership to TS 017067fdf9694126b910ddb698f09512
I20260709 12:23:03.139441 25538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:48968
I20260709 12:23:03.139948 25538 raft_consensus.cc:606] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Received request to transfer leadership to TS 017067fdf9694126b910ddb698f09512
I20260709 12:23:03.143097 25538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:48968
I20260709 12:23:03.143817 25538 raft_consensus.cc:606] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Received request to transfer leadership to TS 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:23:03.146795 25538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
mode: GRACEFUL
new_leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:48968
I20260709 12:23:03.147301 25538 raft_consensus.cc:606] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Received request to transfer leadership to TS 5e4193cbdd464c2f96e2a7012d691e60
I20260709 12:23:03.148599 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:03.149297 25862 raft_consensus.cc:993] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:23:03.149374 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:03.149983 25890 raft_consensus.cc:1081] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Signalling peer 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:23:03.151902 25612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb"
dest_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:52850
I20260709 12:23:03.152366 25612 raft_consensus.cc:493] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.152665 25612 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:03.158170 25612 raft_consensus.cc:515] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.160221 25612 leader_election.cc:290] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 4 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:03.161762 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:03.161720 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254b81c1822048b1a238ec3ce9e88aeb" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:23:03.162680 25685 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:03.162804 25538 raft_consensus.cc:3055] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:03.163182 25538 raft_consensus.cc:740] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:03.164047 25538 consensus_queue.cc:260] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.165525 25538 raft_consensus.cc:3060] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:03.170250 25685 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 4.
I20260709 12:23:03.172051 25577 leader_election.cc:304] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:23:03.173033 25538 raft_consensus.cc:2468] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 4.
I20260709 12:23:03.173188 25855 raft_consensus.cc:2804] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:03.175086 25855 raft_consensus.cc:697] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 4 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:23:03.176297 25855 consensus_queue.cc:237] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.184221 25435 catalog_manager.cc:5697] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 3 to 4, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 4 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.359464 25862 raft_consensus.cc:993] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:23:03.359913 25890 raft_consensus.cc:1081] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Signalling peer 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:23:03.361675 25685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:56216
I20260709 12:23:03.362279 25685 raft_consensus.cc:493] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.362557 25685 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:03.366875 25685 raft_consensus.cc:515] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.369668 25685 leader_election.cc:290] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 5 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:23:03.370721 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:23:03.370851 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb48973785a94c09a20462dc03b8c5fa" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:03.371275 25538 raft_consensus.cc:3055] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:03.371476 25612 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:03.371603 25538 raft_consensus.cc:740] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:03.372766 25538 consensus_queue.cc:260] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.373921 25538 raft_consensus.cc:3060] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:03.377388 25612 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 5.
I20260709 12:23:03.378752 25652 leader_election.cc:304] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:23:03.379791 25845 raft_consensus.cc:2804] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:03.380162 25538 raft_consensus.cc:2468] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 5.
I20260709 12:23:03.381042 25845 raft_consensus.cc:697] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 5 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:23:03.382018 25845 consensus_queue.cc:237] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.390300 25435 catalog_manager.cc:5697] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 4 to 5, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 5 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.433106 25862 raft_consensus.cc:993] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:23:03.433758 25890 raft_consensus.cc:1081] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Signalling peer 5e4193cbdd464c2f96e2a7012d691e60 to start an election
I20260709 12:23:03.435457 25685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557"
dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
 from {username='slave'} at 127.0.0.1:56216
I20260709 12:23:03.436007 25685 raft_consensus.cc:493] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.436507 25685 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:03.441151 25685 raft_consensus.cc:515] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.442991 25685 leader_election.cc:290] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 3 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637)
I20260709 12:23:03.443719 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:23:03.443954 25612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fc2a6dbdd64698a222003a32680557" candidate_uuid: "5e4193cbdd464c2f96e2a7012d691e60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "017067fdf9694126b910ddb698f09512"
I20260709 12:23:03.444357 25538 raft_consensus.cc:3055] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:03.444623 25612 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:03.444721 25538 raft_consensus.cc:740] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:03.445744 25538 consensus_queue.cc:260] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.446679 25538 raft_consensus.cc:3060] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:03.449641 25612 raft_consensus.cc:2468] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 3.
I20260709 12:23:03.450592 25652 leader_election.cc:304] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:23:03.451280 25845 raft_consensus.cc:2804] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:03.451637 25845 raft_consensus.cc:697] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Becoming Leader. State: Replica: 5e4193cbdd464c2f96e2a7012d691e60, State: Running, Role: LEADER
I20260709 12:23:03.451989 25538 raft_consensus.cc:2468] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4193cbdd464c2f96e2a7012d691e60 in term 3.
I20260709 12:23:03.452337 25845 consensus_queue.cc:237] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.459388 25435 catalog_manager.cc:5697] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 reported cstate change: term changed from 2 to 3, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "5e4193cbdd464c2f96e2a7012d691e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.526163 25890 raft_consensus.cc:993] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: : Instructing follower 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:23:03.526644 25862 raft_consensus.cc:1081] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Signalling peer 017067fdf9694126b910ddb698f09512 to start an election
I20260709 12:23:03.528370 25612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a"
dest_uuid: "017067fdf9694126b910ddb698f09512"
 from {username='slave'} at 127.0.0.1:52850
I20260709 12:23:03.528867 25612 raft_consensus.cc:493] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.529104 25612 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:03.533510 25612 raft_consensus.cc:515] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.535159 25612 leader_election.cc:290] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 3 election: Requested vote from peers a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055), 5e4193cbdd464c2f96e2a7012d691e60 (127.24.71.131:41445)
I20260709 12:23:03.536044 25538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2324ad57b524759aa58ebd154b785ae"
I20260709 12:23:03.536204 25685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482670924d0f4c5cb9626cc251a59e0a" candidate_uuid: "017067fdf9694126b910ddb698f09512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e4193cbdd464c2f96e2a7012d691e60"
I20260709 12:23:03.536701 25538 raft_consensus.cc:3055] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:03.536749 25685 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:03.537050 25538 raft_consensus.cc:740] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2324ad57b524759aa58ebd154b785ae, State: Running, Role: LEADER
I20260709 12:23:03.537894 25538 consensus_queue.cc:260] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.538867 25538 raft_consensus.cc:3060] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:03.541464 25685 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 3.
I20260709 12:23:03.542562 25577 leader_election.cc:304] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017067fdf9694126b910ddb698f09512, 5e4193cbdd464c2f96e2a7012d691e60; no voters: 
I20260709 12:23:03.543365 25855 raft_consensus.cc:2804] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:03.543673 25538 raft_consensus.cc:2468] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017067fdf9694126b910ddb698f09512 in term 3.
I20260709 12:23:03.543781 25855 raft_consensus.cc:697] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 3 LEADER]: Becoming Leader. State: Replica: 017067fdf9694126b910ddb698f09512, State: Running, Role: LEADER
I20260709 12:23:03.544902 25855 consensus_queue.cc:237] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } }
I20260709 12:23:03.552011 25435 catalog_manager.cc:5697] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 reported cstate change: term changed from 2 to 3, leader changed from a2324ad57b524759aa58ebd154b785ae (127.24.71.129) to 017067fdf9694126b910ddb698f09512 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "017067fdf9694126b910ddb698f09512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.601806 25538 raft_consensus.cc:1275] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:03.603842 25855 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:03.618966 25685 raft_consensus.cc:1275] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:03.623970 25855 consensus_queue.cc:1048] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:03.800889 25612 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 5 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:03.802241 25845 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:03.812144 25538 raft_consensus.cc:1275] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 5 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:03.813952 25898 consensus_queue.cc:1048] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:03.843214 25612 raft_consensus.cc:1275] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.844710 25898 consensus_queue.cc:1048] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "017067fdf9694126b910ddb698f09512" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:03.855105 25538 raft_consensus.cc:1275] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Refusing update from remote peer 5e4193cbdd464c2f96e2a7012d691e60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.858726 25911 consensus_queue.cc:1048] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:03.980047 25685 raft_consensus.cc:1275] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.981318 25855 consensus_queue.cc:1048] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4193cbdd464c2f96e2a7012d691e60" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:03.990921 25538 raft_consensus.cc:1275] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Refusing update from remote peer 017067fdf9694126b910ddb698f09512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.994572 25891 consensus_queue.cc:1048] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2324ad57b524759aa58ebd154b785ae" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:04.150704 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:04.156880 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:04.157384 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:05.158592 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:05.164178 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:05.164734 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:06.165789 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:06.171500 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:06.171965 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:07.172960 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:07.178886 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:07.179355 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:08.180555 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:08.186832 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:08.187343 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:09.188661 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:09.194013 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:09.194504 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:10.195531 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:10.202440 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:10.202982 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:11.203903 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:11.209415 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:11.209880 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:12.211024 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:12.218762 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:12.219244 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:13.256686 24862 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:23:13.257345 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
W20260709 12:23:13.282027 25650 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 12:23:13.292024 25650 consensus_peers.cc:597] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 -> Peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Couldn't send request to peer a2324ad57b524759aa58ebd154b785ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.302762 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:13.303543 24862 tablet_replica.cc:333] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.304387 24862 raft_consensus.cc:2243] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.305591 24862 raft_consensus.cc:2272] T 3726e81c7a4948d3a48d9c51df49e932 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.308017 24862 tablet_replica.cc:333] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.308687 24862 raft_consensus.cc:2243] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.309334 24862 raft_consensus.cc:2272] T 254b81c1822048b1a238ec3ce9e88aeb P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.311280 24862 tablet_replica.cc:333] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.311813 24862 raft_consensus.cc:2243] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.312340 24862 raft_consensus.cc:2272] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P a2324ad57b524759aa58ebd154b785ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.314172 24862 tablet_replica.cc:333] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.314679 24862 raft_consensus.cc:2243] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.315153 24862 raft_consensus.cc:2272] T 482670924d0f4c5cb9626cc251a59e0a P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.317065 24862 tablet_replica.cc:333] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.317581 24862 raft_consensus.cc:2243] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.318043 24862 raft_consensus.cc:2272] T eb48973785a94c09a20462dc03b8c5fa P a2324ad57b524759aa58ebd154b785ae [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.319813 24862 tablet_replica.cc:333] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.320482 24862 raft_consensus.cc:2243] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.321305 24862 raft_consensus.cc:2272] T 6ec865ad451c482d80468abf199358be P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.323159 24862 tablet_replica.cc:333] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.323658 24862 raft_consensus.cc:2243] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.324110 24862 raft_consensus.cc:2272] T 5cc9a86bc0cd4342ab1c311897d8c350 P a2324ad57b524759aa58ebd154b785ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.325872 24862 tablet_replica.cc:333] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.326452 24862 raft_consensus.cc:2243] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.326973 24862 raft_consensus.cc:2272] T c0fc2a6dbdd64698a222003a32680557 P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.328980 24862 tablet_replica.cc:333] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.329488 24862 raft_consensus.cc:2243] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.330351 24862 raft_consensus.cc:2272] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P a2324ad57b524759aa58ebd154b785ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.333267 24862 tablet_replica.cc:333] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae: stopping tablet replica
I20260709 12:23:13.334342 24862 raft_consensus.cc:2243] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.335950 24862 raft_consensus.cc:2272] T af334351da5349c1870b007d2ab06bab P a2324ad57b524759aa58ebd154b785ae [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:13.337903 25576 consensus_peers.cc:597] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 -> Peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Couldn't send request to peer a2324ad57b524759aa58ebd154b785ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.372715 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
I20260709 12:23:13.394797 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
I20260709 12:23:13.423629 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:13.424463 24862 tablet_replica.cc:333] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.425140 24862 raft_consensus.cc:2243] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.425652 24862 raft_consensus.cc:2272] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.427481 24862 tablet_replica.cc:333] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.428020 24862 raft_consensus.cc:2243] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.428972 24862 raft_consensus.cc:2272] T 254b81c1822048b1a238ec3ce9e88aeb P 017067fdf9694126b910ddb698f09512 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.430738 24862 tablet_replica.cc:333] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.431248 24862 raft_consensus.cc:2243] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.431670 24862 raft_consensus.cc:2272] T 3726e81c7a4948d3a48d9c51df49e932 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.433521 24862 tablet_replica.cc:333] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.434140 24862 raft_consensus.cc:2243] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.434686 24862 raft_consensus.cc:2272] T af334351da5349c1870b007d2ab06bab P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.437026 24862 tablet_replica.cc:333] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.437621 24862 raft_consensus.cc:2243] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.438179 24862 raft_consensus.cc:2272] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 017067fdf9694126b910ddb698f09512 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.440092 24862 tablet_replica.cc:333] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512: stopping tablet replica
W20260709 12:23:13.440409 25650 consensus_peers.cc:597] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 -> Peer a2324ad57b524759aa58ebd154b785ae (127.24.71.129:39055): Couldn't send request to peer a2324ad57b524759aa58ebd154b785ae. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:39055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.440837 24862 raft_consensus.cc:2243] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.441457 24862 raft_consensus.cc:2272] T eb48973785a94c09a20462dc03b8c5fa P 017067fdf9694126b910ddb698f09512 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.443353 24862 tablet_replica.cc:333] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.443866 24862 raft_consensus.cc:2243] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.444733 24862 raft_consensus.cc:2272] T 482670924d0f4c5cb9626cc251a59e0a P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.446506 24862 tablet_replica.cc:333] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.447055 24862 raft_consensus.cc:2243] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.447533 24862 raft_consensus.cc:2272] T 6ec865ad451c482d80468abf199358be P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.449898 24862 tablet_replica.cc:333] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.450501 24862 raft_consensus.cc:2243] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.451699 24862 raft_consensus.cc:2272] T 5cc9a86bc0cd4342ab1c311897d8c350 P 017067fdf9694126b910ddb698f09512 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.453500 24862 tablet_replica.cc:333] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512: stopping tablet replica
I20260709 12:23:13.453966 24862 raft_consensus.cc:2243] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.454469 24862 raft_consensus.cc:2272] T c0fc2a6dbdd64698a222003a32680557 P 017067fdf9694126b910ddb698f09512 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.486258 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
I20260709 12:23:13.507543 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
W20260709 12:23:13.540302 25652 consensus_peers.cc:597] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 -> Peer 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637): Couldn't send request to peer 017067fdf9694126b910ddb698f09512. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:35637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:13.540902 25652 consensus_peers.cc:597] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 -> Peer 017067fdf9694126b910ddb698f09512 (127.24.71.130:35637): Couldn't send request to peer 017067fdf9694126b910ddb698f09512. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:35637: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.541138 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:13.541962 24862 tablet_replica.cc:333] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.542985 24862 raft_consensus.cc:2243] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.543695 24862 raft_consensus.cc:2272] T af334351da5349c1870b007d2ab06bab P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.545747 24862 tablet_replica.cc:333] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.546250 24862 raft_consensus.cc:2243] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.546657 24862 raft_consensus.cc:2272] T 254b81c1822048b1a238ec3ce9e88aeb P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.548586 24862 tablet_replica.cc:333] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.549162 24862 raft_consensus.cc:2243] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.550060 24862 raft_consensus.cc:2272] T c0fc2a6dbdd64698a222003a32680557 P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.551868 24862 tablet_replica.cc:333] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.552464 24862 raft_consensus.cc:2243] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.552942 24862 raft_consensus.cc:2272] T 6ec865ad451c482d80468abf199358be P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.554795 24862 tablet_replica.cc:333] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.555256 24862 raft_consensus.cc:2243] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.555822 24862 raft_consensus.cc:2272] T 5cc9a86bc0cd4342ab1c311897d8c350 P 5e4193cbdd464c2f96e2a7012d691e60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.557538 24862 tablet_replica.cc:333] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.558120 24862 raft_consensus.cc:2243] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.558930 24862 raft_consensus.cc:2272] T 32bd0706d39e4b8bbfbde2fe3664e0a6 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.560882 24862 tablet_replica.cc:333] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.561434 24862 raft_consensus.cc:2243] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.562175 24862 raft_consensus.cc:2272] T eb48973785a94c09a20462dc03b8c5fa P 5e4193cbdd464c2f96e2a7012d691e60 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.564046 24862 tablet_replica.cc:333] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.564608 24862 raft_consensus.cc:2243] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.565094 24862 raft_consensus.cc:2272] T 482670924d0f4c5cb9626cc251a59e0a P 5e4193cbdd464c2f96e2a7012d691e60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.566825 24862 tablet_replica.cc:333] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.567302 24862 raft_consensus.cc:2243] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.567729 24862 raft_consensus.cc:2272] T 06b5b4b43edf4449bc4c3fd0c478ebc4 P 5e4193cbdd464c2f96e2a7012d691e60 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.569689 24862 tablet_replica.cc:333] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60: stopping tablet replica
I20260709 12:23:13.570168 24862 raft_consensus.cc:2243] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.570575 24862 raft_consensus.cc:2272] T 3726e81c7a4948d3a48d9c51df49e932 P 5e4193cbdd464c2f96e2a7012d691e60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.601270 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:23:13.628054 24862 master.cc:562] Master@127.24.71.190:42771 shutting down...
I20260709 12:23:13.655879 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.656569 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.656878 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P fb82d23fc0be4c0ca0398b9d574df107: stopping tablet replica
I20260709 12:23:13.677446 24862 master.cc:584] Master@127.24.71.190:42771 shutdown complete.
017067fdf9694126b910ddb698f09512  4
5e4193cbdd464c2f96e2a7012d691e60  1
a2324ad57b524759aa58ebd154b785ae  5
017067fdf9694126b910ddb698f09512  4
5e4193cbdd464c2f96e2a7012d691e60  3
a2324ad57b524759aa58ebd154b785ae  3
017067fdf9694126b910ddb698f09512  1
5e4193cbdd464c2f96e2a7012d691e60  1
a2324ad57b524759aa58ebd154b785ae  8
017067fdf9694126b910ddb698f09512  3
5e4193cbdd464c2f96e2a7012d691e60  3
a2324ad57b524759aa58ebd154b785ae  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28902 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:23:13.721540 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:42395
I20260709 12:23:13.722627 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:13.728792 25928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.728803 25927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.729823 25930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:13.730145 24862 server_base.cc:1061] running on GCE node
I20260709 12:23:13.731086 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:13.731310 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:13.731539 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599793731439 us; error 0 us; skew 500 ppm
I20260709 12:23:13.732074 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:13.734736 24862 webserver.cc:533] Webserver started at http://127.24.71.190:36945/ using document root <none> and password file <none>
I20260709 12:23:13.735229 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:13.735420 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:13.735702 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:13.736924 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "5eede3841e1242afbeab30840148b558"
format_stamp: "Formatted at 2026-07-09 12:23:13 on dist-test-slave-3zgk"
I20260709 12:23:13.741784 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:13.745443 25935 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:13.746157 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:23:13.746531 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "5eede3841e1242afbeab30840148b558"
format_stamp: "Formatted at 2026-07-09 12:23:13 on dist-test-slave-3zgk"
I20260709 12:23:13.746832 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:13.774920 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:13.776464 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:13.816556 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:42395
I20260709 12:23:13.816645 25986 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:42395 every 8 connection(s)
I20260709 12:23:13.820487 25987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:13.830154 25987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Bootstrap starting.
I20260709 12:23:13.834816 25987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.839481 25987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: No bootstrap required, opened a new log
I20260709 12:23:13.841684 25987 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eede3841e1242afbeab30840148b558" member_type: VOTER }
I20260709 12:23:13.842109 25987 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.842386 25987 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eede3841e1242afbeab30840148b558, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.843065 25987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eede3841e1242afbeab30840148b558" member_type: VOTER }
I20260709 12:23:13.843555 25987 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:13.843782 25987 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:13.844053 25987 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.848352 25987 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eede3841e1242afbeab30840148b558" member_type: VOTER }
I20260709 12:23:13.849064 25987 leader_election.cc:304] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eede3841e1242afbeab30840148b558; no voters: 
I20260709 12:23:13.850343 25987 leader_election.cc:290] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:13.850761 25990 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:13.852082 25990 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 1 LEADER]: Becoming Leader. State: Replica: 5eede3841e1242afbeab30840148b558, State: Running, Role: LEADER
I20260709 12:23:13.852809 25990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eede3841e1242afbeab30840148b558" member_type: VOTER }
I20260709 12:23:13.853371 25987 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:13.857618 25992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eede3841e1242afbeab30840148b558. Latest consensus state: current_term: 1 leader_uuid: "5eede3841e1242afbeab30840148b558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eede3841e1242afbeab30840148b558" member_type: VOTER } }
I20260709 12:23:13.859254 25992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:13.857481 25991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eede3841e1242afbeab30840148b558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eede3841e1242afbeab30840148b558" member_type: VOTER } }
I20260709 12:23:13.860416 25991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:13.861956 25998 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:13.867401 25998 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:13.872843 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:13.877566 25998 catalog_manager.cc:1383] Generated new cluster ID: 4d60869270be445887ca48e8d79d704a
I20260709 12:23:13.877861 25998 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:13.892370 25998 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:13.893676 25998 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:13.910048 25998 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Generated new TSK 0
I20260709 12:23:13.910689 25998 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:13.941375 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:13.947867 26008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.948804 26009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.950838 26011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:13.951300 24862 server_base.cc:1061] running on GCE node
I20260709 12:23:13.952313 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:13.952509 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:13.952685 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599793952649 us; error 0 us; skew 500 ppm
I20260709 12:23:13.953238 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:13.956027 24862 webserver.cc:533] Webserver started at http://127.24.71.129:42679/ using document root <none> and password file <none>
I20260709 12:23:13.956561 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:13.956758 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:13.957048 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:13.958192 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
format_stamp: "Formatted at 2026-07-09 12:23:13 on dist-test-slave-3zgk"
I20260709 12:23:13.963408 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.003s
I20260709 12:23:13.967162 26016 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:13.968042 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:23:13.968406 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
format_stamp: "Formatted at 2026-07-09 12:23:13 on dist-test-slave-3zgk"
I20260709 12:23:13.968735 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:13.987933 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:13.989317 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:13.990837 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:13.993537 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:13.993760 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:13.994016 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:13.994201 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.037418 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:44989
I20260709 12:23:14.037524 26078 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:44989 every 8 connection(s)
I20260709 12:23:14.042455 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:14.051827 26084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.051832 26083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.056340 24862 server_base.cc:1061] running on GCE node
W20260709 12:23:14.056396 26086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.057813 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:14.058053 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:14.058419 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599794058399 us; error 0 us; skew 500 ppm
I20260709 12:23:14.059156 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:14.062076 26079 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42395
I20260709 12:23:14.062136 24862 webserver.cc:533] Webserver started at http://127.24.71.130:37697/ using document root <none> and password file <none>
I20260709 12:23:14.062460 26079 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.062920 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:14.063287 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:14.063396 26079 heartbeater.cc:507] Master 127.24.71.190:42395 requested a full tablet report, sending...
I20260709 12:23:14.063628 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:14.065079 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "9e23389325a6411c9b0634f4f6af54c3"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-3zgk"
I20260709 12:23:14.065896 25952 ts_manager.cc:194] Registered new tserver with Master: 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:14.067683 25952 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52646
I20260709 12:23:14.070392 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:23:14.074214 26091 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.074968 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:23:14.075240 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "9e23389325a6411c9b0634f4f6af54c3"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-3zgk"
I20260709 12:23:14.075562 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.094316 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.095600 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.097183 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:14.099459 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:14.099694 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.099959 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:14.100138 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.142084 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:40611
I20260709 12:23:14.142279 26153 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:40611 every 8 connection(s)
I20260709 12:23:14.146965 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:14.154325 26157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.155206 26158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.159456 26160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.161609 26154 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42395
I20260709 12:23:14.162026 26154 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.162623 24862 server_base.cc:1061] running on GCE node
I20260709 12:23:14.163010 26154 heartbeater.cc:507] Master 127.24.71.190:42395 requested a full tablet report, sending...
I20260709 12:23:14.163775 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:14.164067 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:14.164294 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599794164272 us; error 0 us; skew 500 ppm
I20260709 12:23:14.165050 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:14.165746 25952 ts_manager.cc:194] Registered new tserver with Master: 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:14.167295 25952 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52652
I20260709 12:23:14.168730 24862 webserver.cc:533] Webserver started at http://127.24.71.131:41363/ using document root <none> and password file <none>
I20260709 12:23:14.169318 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:14.169569 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:14.169855 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:14.171070 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "a981c9a7972c4e99a0903924c64fbb5c"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-3zgk"
I20260709 12:23:14.175793 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.005s
I20260709 12:23:14.180212 26165 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.181345 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:23:14.181641 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "a981c9a7972c4e99a0903924c64fbb5c"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-3zgk"
I20260709 12:23:14.181916 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.227609 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.228834 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.230336 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:14.232784 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:14.232970 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.233299 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:14.233453 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.279505 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:38261
I20260709 12:23:14.279639 26227 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:38261 every 8 connection(s)
I20260709 12:23:14.296080 26228 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42395
I20260709 12:23:14.296582 26228 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.297442 26228 heartbeater.cc:507] Master 127.24.71.190:42395 requested a full tablet report, sending...
I20260709 12:23:14.299872 25952 ts_manager.cc:194] Registered new tserver with Master: a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:14.300450 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017377835s
I20260709 12:23:14.301467 25952 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52656
I20260709 12:23:14.329846 25952 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52666:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:14.337772 25952 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:14.542356 26044 tablet_service.cc:1511] Processing CreateTablet for tablet be17917578e1415f98071364583240c1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:14.543957 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17917578e1415f98071364583240c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.543655 26043 tablet_service.cc:1511] Processing CreateTablet for tablet c49cc98db5354047a91f3ec2f90daa28 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:14.545303 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49cc98db5354047a91f3ec2f90daa28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.548810 26042 tablet_service.cc:1511] Processing CreateTablet for tablet d0d1927cc7714e91945918d6aa1216e5 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:14.549710 26118 tablet_service.cc:1511] Processing CreateTablet for tablet c49cc98db5354047a91f3ec2f90daa28 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:14.550523 26042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d1927cc7714e91945918d6aa1216e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.555024 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49cc98db5354047a91f3ec2f90daa28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.557307 26115 tablet_service.cc:1511] Processing CreateTablet for tablet e9b765338c61436481fe416f17637ab2 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:14.559397 26115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b765338c61436481fe416f17637ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.561080 26116 tablet_service.cc:1511] Processing CreateTablet for tablet b4a62528d9404c66b98e87e76ac0f21e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:14.561080 26041 tablet_service.cc:1511] Processing CreateTablet for tablet b4a62528d9404c66b98e87e76ac0f21e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:14.561082 26117 tablet_service.cc:1511] Processing CreateTablet for tablet d0d1927cc7714e91945918d6aa1216e5 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:14.562402 26040 tablet_service.cc:1511] Processing CreateTablet for tablet e9b765338c61436481fe416f17637ab2 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:14.562858 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a62528d9404c66b98e87e76ac0f21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.562858 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a62528d9404c66b98e87e76ac0f21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.563923 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b765338c61436481fe416f17637ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.564343 26117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d1927cc7714e91945918d6aa1216e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.561268 26119 tablet_service.cc:1511] Processing CreateTablet for tablet be17917578e1415f98071364583240c1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:14.569775 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17917578e1415f98071364583240c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.573905 26039 tablet_service.cc:1511] Processing CreateTablet for tablet de16ce1800b64c3c94c918aa6e9cae3d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:14.575959 26039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de16ce1800b64c3c94c918aa6e9cae3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.579622 26190 tablet_service.cc:1511] Processing CreateTablet for tablet b4a62528d9404c66b98e87e76ac0f21e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:14.579945 26192 tablet_service.cc:1511] Processing CreateTablet for tablet c49cc98db5354047a91f3ec2f90daa28 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:14.581100 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a62528d9404c66b98e87e76ac0f21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.582314 26191 tablet_service.cc:1511] Processing CreateTablet for tablet d0d1927cc7714e91945918d6aa1216e5 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:14.583665 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d1927cc7714e91945918d6aa1216e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.585351 26189 tablet_service.cc:1511] Processing CreateTablet for tablet e9b765338c61436481fe416f17637ab2 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:14.586830 26189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b765338c61436481fe416f17637ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.589496 26193 tablet_service.cc:1511] Processing CreateTablet for tablet be17917578e1415f98071364583240c1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:14.590799 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17917578e1415f98071364583240c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.592695 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49cc98db5354047a91f3ec2f90daa28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.621418 26244 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:14.632282 26118 tablet_service.cc:1511] Processing CreateTablet for tablet de16ce1800b64c3c94c918aa6e9cae3d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:14.633028 26188 tablet_service.cc:1511] Processing CreateTablet for tablet de16ce1800b64c3c94c918aa6e9cae3d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:14.633914 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de16ce1800b64c3c94c918aa6e9cae3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.634438 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de16ce1800b64c3c94c918aa6e9cae3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.651190 26244 tablet_bootstrap.cc:654] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.664047 26243 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:14.674916 26243 tablet_bootstrap.cc:654] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.675195 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 7585ccc5a92245cfb1bba6ff29291731 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:14.684453 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 7585ccc5a92245cfb1bba6ff29291731 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:14.686600 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7585ccc5a92245cfb1bba6ff29291731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.685308 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7585ccc5a92245cfb1bba6ff29291731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.693722 26243 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:14.694360 26243 ts_tablet_manager.cc:1403] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.013s
I20260709 12:23:14.697746 26243 raft_consensus.cc:359] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.698721 26243 raft_consensus.cc:385] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.699121 26243 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.700186 26243 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.706642 26190 tablet_service.cc:1511] Processing CreateTablet for tablet 7585ccc5a92245cfb1bba6ff29291731 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:14.708406 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7585ccc5a92245cfb1bba6ff29291731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.711745 26243 ts_tablet_manager.cc:1434] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 12:23:14.713366 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 772287743e884fe8a6dc6c516c3ad901 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:14.714812 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772287743e884fe8a6dc6c516c3ad901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.718935 26244 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:14.719527 26244 ts_tablet_manager.cc:1403] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.099s	user 0.022s	sys 0.017s
I20260709 12:23:14.721349 26245 tablet_bootstrap.cc:492] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:14.724583 26154 heartbeater.cc:499] Master 127.24.71.190:42395 was elected leader, sending a full tablet report...
I20260709 12:23:14.724354 26244 raft_consensus.cc:359] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.725142 26244 raft_consensus.cc:385] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.725497 26244 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.725466 26044 tablet_service.cc:1511] Processing CreateTablet for tablet 772287743e884fe8a6dc6c516c3ad901 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:14.726974 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772287743e884fe8a6dc6c516c3ad901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.726568 26244 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.727669 26243 tablet_bootstrap.cc:492] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:14.734809 26243 tablet_bootstrap.cc:654] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.735242 26079 heartbeater.cc:499] Master 127.24.71.190:42395 was elected leader, sending a full tablet report...
I20260709 12:23:14.739274 26244 ts_tablet_manager.cc:1434] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.007s
I20260709 12:23:14.740481 26244 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:14.746874 26244 tablet_bootstrap.cc:654] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.760061 26245 tablet_bootstrap.cc:654] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.771463 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 772287743e884fe8a6dc6c516c3ad901 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:14.775466 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 0cbe5295e8db495985835a50d39e4438 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:14.775964 26249 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.777236 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772287743e884fe8a6dc6c516c3ad901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.777062 26249 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.777947 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cbe5295e8db495985835a50d39e4438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.782398 26249 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:14.797222 26244 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:14.797820 26244 ts_tablet_manager.cc:1403] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.003s
I20260709 12:23:14.800993 26244 raft_consensus.cc:359] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.801874 26244 raft_consensus.cc:385] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.802304 26244 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.802755 26243 tablet_bootstrap.cc:492] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:14.803334 26243 ts_tablet_manager.cc:1403] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.076s	user 0.025s	sys 0.018s
I20260709 12:23:14.803397 26244 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.804206 26245 tablet_bootstrap.cc:492] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:14.806363 26244 ts_tablet_manager.cc:1434] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:23:14.811645 26245 ts_tablet_manager.cc:1403] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.002s
I20260709 12:23:14.812400 26244 tablet_bootstrap.cc:492] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:14.814092 26193 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3998a28d0e4c4da61419183dc7ffb4 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:14.816350 26116 tablet_service.cc:1511] Processing CreateTablet for tablet 0cbe5295e8db495985835a50d39e4438 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:14.816951 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3998a28d0e4c4da61419183dc7ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.817705 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cbe5295e8db495985835a50d39e4438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.810650 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 0cbe5295e8db495985835a50d39e4438 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:14.820446 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cbe5295e8db495985835a50d39e4438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.821367 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3998a28d0e4c4da61419183dc7ffb4 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:14.822763 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3998a28d0e4c4da61419183dc7ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.826769 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:14.830543 26243 raft_consensus.cc:359] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.831373 26243 raft_consensus.cc:385] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.831723 26243 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.831934 26244 tablet_bootstrap.cc:654] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.835234 26044 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3998a28d0e4c4da61419183dc7ffb4 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:14.835928 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:14.836747 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3998a28d0e4c4da61419183dc7ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:14.837879 26020 leader_election.cc:343] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.839859 26245 raft_consensus.cc:359] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.840700 26245 raft_consensus.cc:385] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.841087 26245 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.841957 26245 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.848932 26117 tablet_service.cc:1511] Processing CreateTablet for tablet 8c8d039bdedc4e1db6d66b58d4b56c4f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:14.849006 26243 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.849548 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 8c8d039bdedc4e1db6d66b58d4b56c4f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:14.850538 26117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8d039bdedc4e1db6d66b58d4b56c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.851444 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8d039bdedc4e1db6d66b58d4b56c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.862445 26243 ts_tablet_manager.cc:1434] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
W20260709 12:23:14.863859 26019 leader_election.cc:343] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.864471 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:23:14.869493 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:14.871718 26244 tablet_bootstrap.cc:492] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:14.871577 26193 tablet_service.cc:1511] Processing CreateTablet for tablet 8c8d039bdedc4e1db6d66b58d4b56c4f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:14.872440 26244 ts_tablet_manager.cc:1403] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.000s
I20260709 12:23:14.873522 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8d039bdedc4e1db6d66b58d4b56c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.877918 26249 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:14.878589 26040 tablet_service.cc:1511] Processing CreateTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:14.879163 26244 raft_consensus.cc:359] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.880483 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ddbca55e34c47f2b2d10e46e4779bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.884914 26243 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:14.892128 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:14.880013 26244 raft_consensus.cc:385] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.893355 26228 heartbeater.cc:499] Master 127.24.71.190:42395 was elected leader, sending a full tablet report...
I20260709 12:23:14.893694 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ddbca55e34c47f2b2d10e46e4779bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.892109 26243 tablet_bootstrap.cc:654] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.893371 26244 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.897986 26244 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.900178 26244 ts_tablet_manager.cc:1434] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 12:23:14.901151 26244 tablet_bootstrap.cc:492] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:14.902693 26245 ts_tablet_manager.cc:1434] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.070s	user 0.019s	sys 0.037s
I20260709 12:23:14.903735 26245 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:14.910177 26245 tablet_bootstrap.cc:654] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.911259 26249 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.916846 26249 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.919381 26249 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:14.920778 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:14.921623 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:14.923650 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:14.925216 26255 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.925289 26249 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.925767 26255 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.926571 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:14.927162 26203 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:14.927729 26249 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.928294 26249 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.932654 26255 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:14.949148 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:14.950961 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ddbca55e34c47f2b2d10e46e4779bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.916123 26244 tablet_bootstrap.cc:654] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.957755 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 0f85110b9e6d45578bf2f39e2e9ad47c (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:14.959712 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f85110b9e6d45578bf2f39e2e9ad47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.967849 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:14.947554 26249 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.969837 26054 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:14.973511 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:14.974182 26117 tablet_service.cc:1511] Processing CreateTablet for tablet 0f85110b9e6d45578bf2f39e2e9ad47c (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:14.975751 26117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f85110b9e6d45578bf2f39e2e9ad47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.974184 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:14.978055 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:14.979249 26255 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.979624 26255 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.979964 26255 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.981696 26249 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:14.982360 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:14.982969 26129 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.987614 26255 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:14.990849 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:14.991741 26054 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:14.993830 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:14.994740 26203 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:14.994735 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:14.996682 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:14.997453 26128 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:14.997860 26245 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:14.998369 26245 ts_tablet_manager.cc:1403] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.095s	user 0.018s	sys 0.060s
I20260709 12:23:14.998652 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:14.998883 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:14.999821 26249 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.001641 26245 raft_consensus.cc:359] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.003997 26245 raft_consensus.cc:385] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.004443 26245 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.004817 26255 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.006506 26255 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.008754 26245 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.011741 26245 ts_tablet_manager.cc:1434] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:23:15.012108 26249 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:15.013021 26245 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.013147 26249 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.021173 26245 tablet_bootstrap.cc:654] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.022483 26244 tablet_bootstrap.cc:492] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:15.023082 26244 ts_tablet_manager.cc:1403] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.122s	user 0.004s	sys 0.007s
I20260709 12:23:15.024998 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 0f85110b9e6d45578bf2f39e2e9ad47c (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:15.026340 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f85110b9e6d45578bf2f39e2e9ad47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.026288 26244 raft_consensus.cc:359] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.027105 26244 raft_consensus.cc:385] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.027398 26244 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.033005 26244 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.057796 26043 tablet_service.cc:1511] Processing CreateTablet for tablet a34019ceec5a4e4e877bed7351d1d91d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:15.059772 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34019ceec5a4e4e877bed7351d1d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.065574 26119 tablet_service.cc:1511] Processing CreateTablet for tablet a34019ceec5a4e4e877bed7351d1d91d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:15.067747 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34019ceec5a4e4e877bed7351d1d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.084743 26193 tablet_service.cc:1511] Processing CreateTablet for tablet a34019ceec5a4e4e877bed7351d1d91d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:15.086184 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34019ceec5a4e4e877bed7351d1d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.085132 26248 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.088593 26248 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.090059 26244 ts_tablet_manager.cc:1434] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 12:23:15.090543 26245 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:15.091033 26245 ts_tablet_manager.cc:1403] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.078s	user 0.028s	sys 0.027s
I20260709 12:23:15.091154 26244 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:15.091332 26248 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.090917 25952 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:15.094216 26245 raft_consensus.cc:359] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.095091 26245 raft_consensus.cc:385] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.095382 26245 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.096688 26243 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:15.097842 26243 ts_tablet_manager.cc:1403] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.213s	user 0.044s	sys 0.062s
I20260709 12:23:15.096223 26245 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.103294 26245 ts_tablet_manager.cc:1434] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:15.104667 26245 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.105957 26244 tablet_bootstrap.cc:654] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.109531 26040 tablet_service.cc:1511] Processing CreateTablet for tablet 8bede0bf927e45a7b18a1bf2ad0bb594 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:15.111773 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bede0bf927e45a7b18a1bf2ad0bb594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.112037 26245 tablet_bootstrap.cc:654] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.116608 26249 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.117296 26249 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.121830 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:15.123816 26020 leader_election.cc:343] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.126170 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:15.128034 26019 leader_election.cc:343] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.128759 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:15.129850 26263 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.130817 26249 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.101375 26243 raft_consensus.cc:359] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.132095 26243 raft_consensus.cc:385] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.132474 26243 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.133210 26243 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.134824 26243 ts_tablet_manager.cc:1434] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.001s
I20260709 12:23:15.138036 26244 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:15.138820 26244 ts_tablet_manager.cc:1403] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.018s
I20260709 12:23:15.143189 26244 raft_consensus.cc:359] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.144373 26244 raft_consensus.cc:385] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.144873 26244 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.144973 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 8bede0bf927e45a7b18a1bf2ad0bb594 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:15.148353 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bede0bf927e45a7b18a1bf2ad0bb594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.148130 26244 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.152767 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 40485ee0f2b44c1290c368cf529eade1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:15.154703 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40485ee0f2b44c1290c368cf529eade1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.157163 26243 tablet_bootstrap.cc:492] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:15.161162 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 8bede0bf927e45a7b18a1bf2ad0bb594 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:15.167554 26244 ts_tablet_manager.cc:1434] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:23:15.177054 26190 tablet_service.cc:1511] Processing CreateTablet for tablet 40485ee0f2b44c1290c368cf529eade1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:15.178086 26244 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:15.178398 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40485ee0f2b44c1290c368cf529eade1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.182276 26249 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.182929 26249 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.186784 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:15.187737 26244 tablet_bootstrap.cc:654] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:15.188617 26019 leader_election.cc:343] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.189774 26043 tablet_service.cc:1511] Processing CreateTablet for tablet c0ff2e29d14b44128ecab4f62a77317f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:15.190423 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:15.191109 26128 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:15.191342 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ff2e29d14b44128ecab4f62a77317f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.192631 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:15.193785 26263 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.194249 26263 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.194666 26263 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.197233 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bede0bf927e45a7b18a1bf2ad0bb594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.187855 26249 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.202044 26117 tablet_service.cc:1511] Processing CreateTablet for tablet 40485ee0f2b44c1290c368cf529eade1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:15.203826 26117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40485ee0f2b44c1290c368cf529eade1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.203399 26263 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.206869 26245 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:15.207355 26245 ts_tablet_manager.cc:1403] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.031s
I20260709 12:23:15.207916 26243 tablet_bootstrap.cc:654] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.207998 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:15.209326 26116 tablet_service.cc:1511] Processing CreateTablet for tablet c0ff2e29d14b44128ecab4f62a77317f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:15.214689 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ff2e29d14b44128ecab4f62a77317f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.214596 26128 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.219621 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:15.220597 26054 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:15.220511 26245 raft_consensus.cc:359] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.221381 26245 raft_consensus.cc:385] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.221694 26245 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.221930 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:15.222440 26245 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.223860 26203 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:15.230748 26245 ts_tablet_manager.cc:1434] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 12:23:15.231894 26245 tablet_bootstrap.cc:492] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.236821 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:15.238545 26245 tablet_bootstrap.cc:654] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.242620 26044 tablet_service.cc:1511] Processing CreateTablet for tablet bec84548f93742a096e72c1bd887809a (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:15.244431 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec84548f93742a096e72c1bd887809a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.249990 26245 tablet_bootstrap.cc:492] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:15.250578 26245 ts_tablet_manager.cc:1403] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:23:15.254346 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:15.255249 26203 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.261801 26193 tablet_service.cc:1511] Processing CreateTablet for tablet c0ff2e29d14b44128ecab4f62a77317f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:15.263751 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ff2e29d14b44128ecab4f62a77317f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.268150 26245 raft_consensus.cc:359] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.269016 26245 raft_consensus.cc:385] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.269384 26245 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.270218 26245 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.271229 26244 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:15.271727 26244 ts_tablet_manager.cc:1403] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.094s	user 0.012s	sys 0.014s
I20260709 12:23:15.272451 26245 ts_tablet_manager.cc:1434] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:23:15.273104 26190 tablet_service.cc:1511] Processing CreateTablet for tablet bec84548f93742a096e72c1bd887809a (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:15.265986 26263 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.274585 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec84548f93742a096e72c1bd887809a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.274718 26244 raft_consensus.cc:359] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.275456 26244 raft_consensus.cc:385] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.275760 26244 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.273547 26245 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.277987 26203 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:15.279297 26019 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:15.280534 26263 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.281052 26263 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:15.281929 26263 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.282382 26043 tablet_service.cc:1511] Processing CreateTablet for tablet e0673bc676154f04907934c1d3a60bce (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:15.283579 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0673bc676154f04907934c1d3a60bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.289261 26245 tablet_bootstrap.cc:654] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.291575 26244 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.297837 26245 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:15.298218 26244 ts_tablet_manager.cc:1434] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.007s
I20260709 12:23:15.298638 26245 ts_tablet_manager.cc:1403] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.005s
I20260709 12:23:15.304028 26116 tablet_service.cc:1511] Processing CreateTablet for tablet bec84548f93742a096e72c1bd887809a (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:15.314785 26188 tablet_service.cc:1511] Processing CreateTablet for tablet e0673bc676154f04907934c1d3a60bce (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:15.317107 26043 tablet_service.cc:1511] Processing CreateTablet for tablet f5a510044c5b459abfd55896cc2d21eb (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:15.320014 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec84548f93742a096e72c1bd887809a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.320973 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0673bc676154f04907934c1d3a60bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.322698 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a510044c5b459abfd55896cc2d21eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.325215 25952 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:15.322712 26245 raft_consensus.cc:359] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.326506 26245 raft_consensus.cc:385] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.326893 26245 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.321552 26128 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:15.333834 26244 tablet_bootstrap.cc:492] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:15.334293 26119 tablet_service.cc:1511] Processing CreateTablet for tablet e0673bc676154f04907934c1d3a60bce (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:15.335485 26248 raft_consensus.cc:2764] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:23:15.335314 26118 tablet_service.cc:1511] Processing CreateTablet for tablet f5a510044c5b459abfd55896cc2d21eb (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:15.335705 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0673bc676154f04907934c1d3a60bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.338825 26190 tablet_service.cc:1511] Processing CreateTablet for tablet f5a510044c5b459abfd55896cc2d21eb (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:15.327759 26245 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.340606 26244 tablet_bootstrap.cc:654] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.348757 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a510044c5b459abfd55896cc2d21eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.349702 26245 ts_tablet_manager.cc:1434] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 12:23:15.350700 26245 tablet_bootstrap.cc:492] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.340756 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a510044c5b459abfd55896cc2d21eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.355842 26044 tablet_service.cc:1511] Processing CreateTablet for tablet 86798289f90549ed8084d260f08cb204 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:15.357401 26245 tablet_bootstrap.cc:654] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.361937 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86798289f90549ed8084d260f08cb204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.347661 26243 tablet_bootstrap.cc:492] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:15.364610 26243 ts_tablet_manager.cc:1403] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.208s	user 0.028s	sys 0.024s
I20260709 12:23:15.368494 26243 raft_consensus.cc:359] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.371129 26262 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.376448 26117 tablet_service.cc:1511] Processing CreateTablet for tablet 86798289f90549ed8084d260f08cb204 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:15.393805 26262 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.394490 26117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86798289f90549ed8084d260f08cb204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.396054 26262 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.396723 26243 raft_consensus.cc:385] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.396848 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:15.397161 26243 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.381870 26119 tablet_service.cc:1511] Processing CreateTablet for tablet ef606226bb434b95911632eec3ede435 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:15.385566 26244 tablet_bootstrap.cc:492] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:15.397512 26054 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:15.398089 26244 ts_tablet_manager.cc:1403] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.064s	user 0.009s	sys 0.003s
I20260709 12:23:15.398339 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef606226bb434b95911632eec3ede435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.398590 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:15.399243 26128 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:15.380565 26040 tablet_service.cc:1511] Processing CreateTablet for tablet ef606226bb434b95911632eec3ede435 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:15.400640 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef606226bb434b95911632eec3ede435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.401907 26244 raft_consensus.cc:359] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.402626 26244 raft_consensus.cc:385] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.402926 26244 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.403141 26118 tablet_service.cc:1511] Processing CreateTablet for tablet f720b2e4986f496694c7578beef67cd4 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:15.403421 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:15.404575 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f720b2e4986f496694c7578beef67cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.386849 26191 tablet_service.cc:1511] Processing CreateTablet for tablet 549b49961cbb430d9ece6acd81aede77 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:15.404213 26244 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.386061 26116 tablet_service.cc:1511] Processing CreateTablet for tablet 549b49961cbb430d9ece6acd81aede77 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:15.405578 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 549b49961cbb430d9ece6acd81aede77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.398023 26243 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.417428 26044 tablet_service.cc:1511] Processing CreateTablet for tablet f720b2e4986f496694c7578beef67cd4 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:15.378275 26192 tablet_service.cc:1511] Processing CreateTablet for tablet 86798289f90549ed8084d260f08cb204 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:15.406378 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 549b49961cbb430d9ece6acd81aede77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.419193 26243 ts_tablet_manager.cc:1434] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.002s
I20260709 12:23:15.397032 26245 tablet_bootstrap.cc:492] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:15.419795 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86798289f90549ed8084d260f08cb204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.420189 26245 ts_tablet_manager.cc:1403] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.000s
I20260709 12:23:15.392591 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 549b49961cbb430d9ece6acd81aede77 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:15.422398 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 549b49961cbb430d9ece6acd81aede77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.423252 26245 raft_consensus.cc:359] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.427168 26243 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:15.407130 26262 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.428546 26262 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.429060 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f720b2e4986f496694c7578beef67cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.418517 26255 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.391048 26190 tablet_service.cc:1511] Processing CreateTablet for tablet ef606226bb434b95911632eec3ede435 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:15.429574 26255 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.430231 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef606226bb434b95911632eec3ede435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.431972 26255 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.433880 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:15.429008 26262 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:15.435530 26169 leader_election.cc:343] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.406782 26244 ts_tablet_manager.cc:1434] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:15.437383 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:15.438043 26054 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:15.439599 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:15.440867 26255 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.441237 26255 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.441589 26255 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.441797 26243 tablet_bootstrap.cc:654] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.445832 26193 tablet_service.cc:1511] Processing CreateTablet for tablet f720b2e4986f496694c7578beef67cd4 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:15.456439 26244 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:15.456486 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f720b2e4986f496694c7578beef67cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.457088 26041 tablet_service.cc:1511] Processing CreateTablet for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:15.458709 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.448038 26039 tablet_service.cc:1511] Processing CreateTablet for tablet eac7044e09a64594b8fcd962f1d387b7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:15.424028 26245 raft_consensus.cc:385] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.460820 26245 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.461308 26039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac7044e09a64594b8fcd962f1d387b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.462725 26245 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.449633 26115 tablet_service.cc:1511] Processing CreateTablet for tablet eac7044e09a64594b8fcd962f1d387b7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:15.465380 26115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac7044e09a64594b8fcd962f1d387b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.453271 26188 tablet_service.cc:1511] Processing CreateTablet for tablet eac7044e09a64594b8fcd962f1d387b7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:15.469123 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac7044e09a64594b8fcd962f1d387b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.455842 26255 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.469801 26189 tablet_service.cc:1511] Processing CreateTablet for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:15.471275 26189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.473541 26263 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.474013 26042 tablet_service.cc:1511] Processing CreateTablet for tablet c47092de97434584b0810190b58e7623 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:15.473826 26114 tablet_service.cc:1511] Processing CreateTablet for tablet c47092de97434584b0810190b58e7623 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:15.474117 26263 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.475463 26042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47092de97434584b0810190b58e7623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.475554 26114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47092de97434584b0810190b58e7623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.481173 26282 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.481662 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:15.484397 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:15.485033 26203 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
W20260709 12:23:15.485551 26020 leader_election.cc:343] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.482970 26263 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.483165 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:15.493747 26187 tablet_service.cc:1511] Processing CreateTablet for tablet c47092de97434584b0810190b58e7623 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:15.495194 26187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47092de97434584b0810190b58e7623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.496553 26054 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.509142 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:15.518111 26262 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.484076 26255 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.521874 26263 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.524127 26244 tablet_bootstrap.cc:654] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.526428 26262 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.529109 26263 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.532457 26245 ts_tablet_manager.cc:1434] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.000s
I20260709 12:23:15.535868 26243 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:15.536563 26113 tablet_service.cc:1511] Processing CreateTablet for tablet 93ec87f552a04ce49312a2ef65f862ce (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:15.537817 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:15.538806 26192 tablet_service.cc:1511] Processing CreateTablet for tablet 93ec87f552a04ce49312a2ef65f862ce (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:15.539319 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:15.575793 26129 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.577272 26116 tablet_service.cc:1511] Processing CreateTablet for tablet adcc76aa423a40eb95d31d98a8c8776f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:15.539992 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:15.543371 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:15.580331 26053 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.580716 26245 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.581826 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ec87f552a04ce49312a2ef65f862ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.547024 26249 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.586021 26054 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
W20260709 12:23:15.587224 26169 leader_election.cc:343] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.587847 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:15.492151 26282 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.556183 26044 tablet_service.cc:1511] Processing CreateTablet for tablet 93ec87f552a04ce49312a2ef65f862ce (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:15.570505 26248 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.572618 26263 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.591228 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ec87f552a04ce49312a2ef65f862ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.574939 26113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ec87f552a04ce49312a2ef65f862ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.589586 26243 ts_tablet_manager.cc:1403] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.163s	user 0.009s	sys 0.009s
I20260709 12:23:15.590637 26248 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.596494 26282 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.598150 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:15.599673 26169 leader_election.cc:343] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.601461 26041 tablet_service.cc:1511] Processing CreateTablet for tablet 2fee7aee965145c6b43db63c2b3ecca8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:15.601379 26187 tablet_service.cc:1511] Processing CreateTablet for tablet 2fee7aee965145c6b43db63c2b3ecca8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:15.602170 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:15.602739 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fee7aee965145c6b43db63c2b3ecca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.602898 26054 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:15.604162 26114 tablet_service.cc:1511] Processing CreateTablet for tablet 2fee7aee965145c6b43db63c2b3ecca8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:15.604595 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:15.605983 26282 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.606411 26282 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.606737 26248 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.606809 26282 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.608887 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:15.609570 26202 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:15.611203 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:15.611866 26052 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:15.602739 26187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fee7aee965145c6b43db63c2b3ecca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.613397 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.614296 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:15.616564 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcc76aa423a40eb95d31d98a8c8776f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.616823 26245 tablet_bootstrap.cc:654] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.589323 26262 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.619884 26262 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:15.620854 26262 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.624632 26249 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.624889 26191 tablet_service.cc:1511] Processing CreateTablet for tablet adcc76aa423a40eb95d31d98a8c8776f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:15.626284 26114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fee7aee965145c6b43db63c2b3ecca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.630581 26248 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.630990 26248 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.631330 26248 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.626309 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcc76aa423a40eb95d31d98a8c8776f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.599701 26263 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.628554 26282 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.652837 26244 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:15.653396 26244 ts_tablet_manager.cc:1403] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.216s	user 0.018s	sys 0.008s
I20260709 12:23:15.656258 26244 raft_consensus.cc:359] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.657030 26244 raft_consensus.cc:385] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.657328 26244 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.657681 26039 tablet_service.cc:1511] Processing CreateTablet for tablet adcc76aa423a40eb95d31d98a8c8776f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:15.658041 26244 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.659013 26039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcc76aa423a40eb95d31d98a8c8776f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.661127 26129 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:15.662494 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:15.662717 26282 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:15.663028 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:15.663394 26282 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.663676 26054 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.664099 26282 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:15.665040 26282 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.667181 26263 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.668789 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:15.668896 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:15.669737 26244 ts_tablet_manager.cc:1434] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:23:15.671747 26053 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:15.671983 26115 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee8b7d54fee44329721d2da7c8f62ad (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:15.658718 26041 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee8b7d54fee44329721d2da7c8f62ad (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:15.672915 26248 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.673388 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:15.673338 26054 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:15.661571 26243 raft_consensus.cc:359] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.679873 26244 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:15.682183 26202 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.682457 26263 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.681325 26243 raft_consensus.cc:385] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.682880 26115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee8b7d54fee44329721d2da7c8f62ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.683493 26263 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:15.694648 26020 leader_election.cc:343] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.694788 26117 tablet_service.cc:1511] Processing CreateTablet for tablet d3b95127d51d4e5db31f5dbda1a935c0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:15.696223 26117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3b95127d51d4e5db31f5dbda1a935c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.696856 26263 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.697906 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:15.698683 26042 tablet_service.cc:1511] Processing CreateTablet for tablet d3b95127d51d4e5db31f5dbda1a935c0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
W20260709 12:23:15.699331 26020 leader_election.cc:343] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.685616 26243 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
W20260709 12:23:15.690757 26169 leader_election.cc:343] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.708806 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:15.709849 26262 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.709729 26193 tablet_service.cc:1511] Processing CreateTablet for tablet d3b95127d51d4e5db31f5dbda1a935c0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:15.709957 26040 tablet_service.cc:1511] Processing CreateTablet for tablet 0d096e6da5114d91b7c4dd46412d2a6a (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:15.710507 26262 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:15.711145 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3b95127d51d4e5db31f5dbda1a935c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.711340 26262 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.713896 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee8b7d54fee44329721d2da7c8f62ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.716785 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 0d096e6da5114d91b7c4dd46412d2a6a (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:15.717787 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:15.718104 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d096e6da5114d91b7c4dd46412d2a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.720800 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d096e6da5114d91b7c4dd46412d2a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.725548 26042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3b95127d51d4e5db31f5dbda1a935c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.725878 26202 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:15.711205 26243 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.729941 26243 ts_tablet_manager.cc:1434] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.135s	user 0.006s	sys 0.001s
I20260709 12:23:15.731153 26243 tablet_bootstrap.cc:492] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
W20260709 12:23:15.737650 26019 leader_election.cc:343] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.738274 26019 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:15.739056 26263 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.739264 26244 tablet_bootstrap.cc:654] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.739528 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:15.740535 26263 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.741981 26248 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:15.744094 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:15.743970 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:15.744781 26203 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.744889 26054 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.754027 26203 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:15.754228 26192 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee8b7d54fee44329721d2da7c8f62ad (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:15.756768 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee8b7d54fee44329721d2da7c8f62ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.755524 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:15.779922 26245 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:15.780407 26245 ts_tablet_manager.cc:1403] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.200s	user 0.015s	sys 0.006s
I20260709 12:23:15.784106 26248 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.784693 26248 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:15.785547 26113 tablet_service.cc:1511] Processing CreateTablet for tablet 55aa4b8b33f544088007e1987c55617e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:15.785641 26191 tablet_service.cc:1511] Processing CreateTablet for tablet 0d096e6da5114d91b7c4dd46412d2a6a (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:15.787217 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d096e6da5114d91b7c4dd46412d2a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.785548 26248 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.792013 26116 tablet_service.cc:1511] Processing CreateTablet for tablet 4db91b54f05348e6855f89c508d0be01 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:15.794267 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 4db91b54f05348e6855f89c508d0be01 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:15.800973 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db91b54f05348e6855f89c508d0be01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.794396 26038 tablet_service.cc:1511] Processing CreateTablet for tablet 55aa4b8b33f544088007e1987c55617e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:15.804525 26038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aa4b8b33f544088007e1987c55617e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.805754 26245 raft_consensus.cc:359] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.777532 26054 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:15.787212 26113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aa4b8b33f544088007e1987c55617e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.796204 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 4db91b54f05348e6855f89c508d0be01 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:15.809332 26245 raft_consensus.cc:385] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.809674 26245 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.810351 26245 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.777273 26263 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:15.818199 26245 ts_tablet_manager.cc:1434] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:23:15.818939 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db91b54f05348e6855f89c508d0be01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.822830 26187 tablet_service.cc:1511] Processing CreateTablet for tablet 55aa4b8b33f544088007e1987c55617e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:15.828994 26039 tablet_service.cc:1511] Processing CreateTablet for tablet 46aa595bcb23423fb51ff268d34a3bc7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:15.833375 25952 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.834637 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db91b54f05348e6855f89c508d0be01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.836688 26190 tablet_service.cc:1511] Processing CreateTablet for tablet 46aa595bcb23423fb51ff268d34a3bc7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:15.844873 25952 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.849150 25952 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:15.872078 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:15.874131 26245 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:15.878671 26263 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.881630 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 94c3d95514eb45178f2e5aa92e868e36 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:15.875128 26187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aa4b8b33f544088007e1987c55617e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.891014 26243 tablet_bootstrap.cc:654] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.896901 26249 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.886483 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 46aa595bcb23423fb51ff268d34a3bc7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:15.901840 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46aa595bcb23423fb51ff268d34a3bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.900681 26039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46aa595bcb23423fb51ff268d34a3bc7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:15.908483 26155 tablet.cc:2406] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:15.916635 26189 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe81c3162f24fc0b8716b2b42f0676f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:15.919541 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe81c3162f24fc0b8716b2b42f0676f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:15.919931 26114 tablet_service.cc:1511] Processing CreateTablet for tablet 94c3d95514eb45178f2e5aa92e868e36 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:15.920892 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe81c3162f24fc0b8716b2b42f0676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.929703 26249 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.933380 26041 tablet_service.cc:1511] Processing CreateTablet for tablet ec9017ccb0d849c49a75c9b1dce1547e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:15.935221 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9017ccb0d849c49a75c9b1dce1547e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.936062 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c3d95514eb45178f2e5aa92e868e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.941262 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 94c3d95514eb45178f2e5aa92e868e36 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:15.943917 26244 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:15.946806 26244 ts_tablet_manager.cc:1403] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.267s	user 0.015s	sys 0.013s
I20260709 12:23:15.944654 26114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c3d95514eb45178f2e5aa92e868e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.947311 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46aa595bcb23423fb51ff268d34a3bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.955335 26244 raft_consensus.cc:359] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:15.958890 26244 raft_consensus.cc:385] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.970472 26040 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe81c3162f24fc0b8716b2b42f0676f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:15.972028 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe81c3162f24fc0b8716b2b42f0676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.972563 26189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe81c3162f24fc0b8716b2b42f0676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.998339 26119 tablet_service.cc:1511] Processing CreateTablet for tablet ddc5e875aa744c50b603a5072bc7305b (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:15.978816 26118 tablet_service.cc:1511] Processing CreateTablet for tablet ec9017ccb0d849c49a75c9b1dce1547e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:15.999497 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddc5e875aa744c50b603a5072bc7305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.000164 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9017ccb0d849c49a75c9b1dce1547e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.983703 26193 tablet_service.cc:1511] Processing CreateTablet for tablet ec9017ccb0d849c49a75c9b1dce1547e (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:15.993916 26192 tablet_service.cc:1511] Processing CreateTablet for tablet 955755ce63f743b9a01d5871aff94f6c (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:16.005031 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9017ccb0d849c49a75c9b1dce1547e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.005903 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955755ce63f743b9a01d5871aff94f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.994145 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 955755ce63f743b9a01d5871aff94f6c (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:16.025910 26042 tablet_service.cc:1511] Processing CreateTablet for tablet ddc5e875aa744c50b603a5072bc7305b (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:16.026844 26187 tablet_service.cc:1511] Processing CreateTablet for tablet ddc5e875aa744c50b603a5072bc7305b (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:16.028170 26187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddc5e875aa744c50b603a5072bc7305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.029173 26245 tablet_bootstrap.cc:654] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.051071 26191 tablet_service.cc:1511] Processing CreateTablet for tablet 407574b4cda94da99be372d249d21fd5 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:16.037473 26113 tablet_service.cc:1511] Processing CreateTablet for tablet 955755ce63f743b9a01d5871aff94f6c (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:16.055015 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 407574b4cda94da99be372d249d21fd5 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:16.036702 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c3d95514eb45178f2e5aa92e868e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.959235 26244 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.051734 26044 tablet_service.cc:1511] Processing CreateTablet for tablet ecbcf3556b554f8a9c1c00b01dce7f5f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:16.059522 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 407574b4cda94da99be372d249d21fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.060544 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 407574b4cda94da99be372d249d21fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.068116 26113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955755ce63f743b9a01d5871aff94f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.038030 26038 tablet_service.cc:1511] Processing CreateTablet for tablet 407574b4cda94da99be372d249d21fd5 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:16.075631 26042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddc5e875aa744c50b603a5072bc7305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.076426 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955755ce63f743b9a01d5871aff94f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.077797 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbcf3556b554f8a9c1c00b01dce7f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.080050 26282 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.088588 26038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 407574b4cda94da99be372d249d21fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.112494 26244 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.117363 26244 ts_tablet_manager.cc:1434] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.170s	user 0.005s	sys 0.002s
I20260709 12:23:16.122246 26115 tablet_service.cc:1511] Processing CreateTablet for tablet ecbcf3556b554f8a9c1c00b01dce7f5f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:16.123696 26115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbcf3556b554f8a9c1c00b01dce7f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.126299 26243 tablet_bootstrap.cc:492] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:16.126863 26243 ts_tablet_manager.cc:1403] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.396s	user 0.015s	sys 0.024s
I20260709 12:23:16.128050 25951 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.134387 26245 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:16.134891 26245 ts_tablet_manager.cc:1403] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.261s	user 0.031s	sys 0.015s
I20260709 12:23:16.136950 26188 tablet_service.cc:1511] Processing CreateTablet for tablet ecbcf3556b554f8a9c1c00b01dce7f5f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:16.137616 26243 raft_consensus.cc:359] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.138360 26243 raft_consensus.cc:385] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.138720 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbcf3556b554f8a9c1c00b01dce7f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.143383 26244 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:16.138657 26243 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.144196 26243 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.146788 26243 ts_tablet_manager.cc:1434] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:23:16.147884 26243 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:16.139655 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 323fa2c24ef84fd39b5fe37060615814 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:16.150111 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 323fa2c24ef84fd39b5fe37060615814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.154351 26192 tablet_service.cc:1511] Processing CreateTablet for tablet 323fa2c24ef84fd39b5fe37060615814 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:16.155768 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 323fa2c24ef84fd39b5fe37060615814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.160633 26245 raft_consensus.cc:359] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.161273 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 689b9b7393b349afbf302f0f99710830 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:16.162117 26039 tablet_service.cc:1511] Processing CreateTablet for tablet 689b9b7393b349afbf302f0f99710830 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:16.163419 26039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 689b9b7393b349afbf302f0f99710830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.161368 26245 raft_consensus.cc:385] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.164290 26245 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.167634 26193 tablet_service.cc:1511] Processing CreateTablet for tablet 689b9b7393b349afbf302f0f99710830 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:16.168893 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 689b9b7393b349afbf302f0f99710830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.168339 26245 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:16.169173 26169 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:16.174940 26113 tablet_service.cc:1511] Processing CreateTablet for tablet 323fa2c24ef84fd39b5fe37060615814 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:16.176272 26044 tablet_service.cc:1511] Processing CreateTablet for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:16.177671 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.176353 26245 ts_tablet_manager.cc:1434] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.041s	user 0.010s	sys 0.000s
I20260709 12:23:16.185227 26245 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:16.186437 26244 tablet_bootstrap.cc:654] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.190516 26248 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.191090 26248 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.193605 26248 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:16.194591 26129 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.194916 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 689b9b7393b349afbf302f0f99710830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.200300 26113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 323fa2c24ef84fd39b5fe37060615814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.202143 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:16.205137 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:16.209159 26094 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:16.212688 26248 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.216614 26286 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.218309 26245 tablet_bootstrap.cc:654] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.217970 26255 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.222687 26244 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:16.223284 26244 ts_tablet_manager.cc:1403] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.003s
I20260709 12:23:16.224869 26191 tablet_service.cc:1511] Processing CreateTablet for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:16.226233 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.226402 26244 raft_consensus.cc:359] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.227162 26244 raft_consensus.cc:385] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.227452 26244 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.228183 26244 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.232995 26244 ts_tablet_manager.cc:1434] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:23:16.233958 26244 tablet_bootstrap.cc:492] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:16.259091 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:16.260887 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.274168 26192 tablet_service.cc:1511] Processing CreateTablet for tablet 824b6757fe414215b1cf6aa5ba238004 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:16.281747 26114 tablet_service.cc:1511] Processing CreateTablet for tablet 824b6757fe414215b1cf6aa5ba238004 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:16.284277 26255 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.284637 26114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824b6757fe414215b1cf6aa5ba238004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.289388 26118 tablet_service.cc:1511] Processing CreateTablet for tablet dc3c86db3d4e4643a77b0fe0b0e687e1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:16.290823 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3c86db3d4e4643a77b0fe0b0e687e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.293179 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824b6757fe414215b1cf6aa5ba238004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.298977 26248 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.306270 26282 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.306782 26282 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.310985 26282 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:16.312188 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:16.312381 26187 tablet_service.cc:1511] Processing CreateTablet for tablet dc3c86db3d4e4643a77b0fe0b0e687e1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:16.313721 26187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3c86db3d4e4643a77b0fe0b0e687e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:16.313777 26169 leader_election.cc:343] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.316835 26127 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.317291 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:16.317999 26053 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:16.319249 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:16.320590 26282 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.320979 26282 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.321344 26282 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.303344 26255 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:23:16.332639 25950 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.337860 26244 tablet_bootstrap.cc:654] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.338806 26255 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.339643 26255 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.340551 26303 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.349004 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 824b6757fe414215b1cf6aa5ba238004 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:16.350241 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824b6757fe414215b1cf6aa5ba238004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.373202 26243 tablet_bootstrap.cc:654] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.383569 26262 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:16.392513 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 526fbe6496c445dc95df29efcbeb8cd0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:16.394130 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526fbe6496c445dc95df29efcbeb8cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.398531 26193 tablet_service.cc:1511] Processing CreateTablet for tablet 49f4a1a801de461182f8315a884b9889 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:16.399685 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f4a1a801de461182f8315a884b9889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.399380 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 49f4a1a801de461182f8315a884b9889 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:16.399740 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 2d132ed1efbf42c38f5a384134455ca3 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:16.400722 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f4a1a801de461182f8315a884b9889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.401257 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d132ed1efbf42c38f5a384134455ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.411453 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 2d132ed1efbf42c38f5a384134455ca3 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:16.412828 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d132ed1efbf42c38f5a384134455ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.416262 26245 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:16.416914 26245 ts_tablet_manager.cc:1403] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.234s	user 0.034s	sys 0.079s
I20260709 12:23:16.422526 26255 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:16.423894 26311 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:23:16.424165 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:16.424189 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:16.425052 26052 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:16.434765 26189 tablet_service.cc:1511] Processing CreateTablet for tablet c7ad94af29be4656bf3ce8a023d19f7d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:16.436131 26189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ad94af29be4656bf3ce8a023d19f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.444605 26039 tablet_service.cc:1511] Processing CreateTablet for tablet dc3c86db3d4e4643a77b0fe0b0e687e1 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:16.445976 26039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3c86db3d4e4643a77b0fe0b0e687e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.450296 26116 tablet_service.cc:1511] Processing CreateTablet for tablet c7ad94af29be4656bf3ce8a023d19f7d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:16.426596 26115 tablet_service.cc:1511] Processing CreateTablet for tablet 2d132ed1efbf42c38f5a384134455ca3 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:16.451421 26244 tablet_bootstrap.cc:492] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:16.451745 26116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ad94af29be4656bf3ce8a023d19f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.451916 26244 ts_tablet_manager.cc:1403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.218s	user 0.011s	sys 0.014s
I20260709 12:23:16.450421 26190 tablet_service.cc:1511] Processing CreateTablet for tablet 526fbe6496c445dc95df29efcbeb8cd0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:16.453013 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526fbe6496c445dc95df29efcbeb8cd0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:16.425794 26169 leader_election.cc:343] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.468758 26115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d132ed1efbf42c38f5a384134455ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.396186 26282 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.479349 26044 tablet_service.cc:1511] Processing CreateTablet for tablet c7ad94af29be4656bf3ce8a023d19f7d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:16.435900 26245 raft_consensus.cc:359] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.486480 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ad94af29be4656bf3ce8a023d19f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.486472 26245 raft_consensus.cc:385] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.487041 26245 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.487995 26244 raft_consensus.cc:359] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.488663 26244 raft_consensus.cc:385] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.488888 26244 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.491165 26245 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.495486 26244 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.470918 26038 tablet_service.cc:1511] Processing CreateTablet for tablet 526fbe6496c445dc95df29efcbeb8cd0 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:16.510691 26038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526fbe6496c445dc95df29efcbeb8cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.518929 26244 ts_tablet_manager.cc:1434] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.000s
I20260709 12:23:16.530531 26244 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:16.526876 26190 tablet_service.cc:1511] Processing CreateTablet for tablet 47b65a35d65c481985263a8a2546d39d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:16.539950 26303 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.540721 26303 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.462044 26040 tablet_service.cc:1511] Processing CreateTablet for tablet 49f4a1a801de461182f8315a884b9889 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:16.548573 26193 tablet_service.cc:1511] Processing CreateTablet for tablet 891138e164dc40a7af7618323d52397f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:16.549436 26040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f4a1a801de461182f8315a884b9889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.550271 26190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b65a35d65c481985263a8a2546d39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.484637 26168 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:16.543654 26303 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:16.553478 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 83915.9us
I20260709 12:23:16.555070 26315 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.555615 26315 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.556061 26315 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.558561 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:16.560822 26303 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.569118 26282 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:16.575472 26119 tablet_service.cc:1511] Processing CreateTablet for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:16.576135 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 47b65a35d65c481985263a8a2546d39d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:16.576828 26119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.577775 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b65a35d65c481985263a8a2546d39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.578291 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:16.579320 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.529096 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:16.581408 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.584347 26041 tablet_service.cc:1511] Processing CreateTablet for tablet 891138e164dc40a7af7618323d52397f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:16.576170 26113 tablet_service.cc:1511] Processing CreateTablet for tablet 891138e164dc40a7af7618323d52397f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:16.589639 26113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891138e164dc40a7af7618323d52397f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.591961 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:16.593314 26042 tablet_service.cc:1511] Processing CreateTablet for tablet 47b65a35d65c481985263a8a2546d39d (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 12:23:16.595135 26020 leader_election.cc:343] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.585520 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891138e164dc40a7af7618323d52397f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.619710 26193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891138e164dc40a7af7618323d52397f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:16.611392 26019 leader_election.cc:343] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.616364 26315 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.628369 26245 ts_tablet_manager.cc:1434] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.211s	user 0.046s	sys 0.062s
I20260709 12:23:16.631937 26245 tablet_bootstrap.cc:492] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:16.609835 26243 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:16.632624 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:23:16.582926 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 120972us
I20260709 12:23:16.618193 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:16.653949 26243 ts_tablet_manager.cc:1403] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.506s	user 0.026s	sys 0.006s
I20260709 12:23:16.667023 26042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b65a35d65c481985263a8a2546d39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.677888 26245 tablet_bootstrap.cc:654] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.669935 26263 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.665665 26052 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.728384 26249 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.728894 26249 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.732919 26249 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:16.735698 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:16.737277 26169 leader_election.cc:343] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.731525 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 1e8b57c1b80240d483c7ca4e95d74673 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:16.748381 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 1e8b57c1b80240d483c7ca4e95d74673 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:16.749625 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e8b57c1b80240d483c7ca4e95d74673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.752943 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e8b57c1b80240d483c7ca4e95d74673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.766698 26244 tablet_bootstrap.cc:654] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.771353 26043 tablet_service.cc:1511] Processing CreateTablet for tablet 6389d613e36a42958d24b00099843b93 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:16.773002 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6389d613e36a42958d24b00099843b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.787245 26044 tablet_service.cc:1511] Processing CreateTablet for tablet 1e8b57c1b80240d483c7ca4e95d74673 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:16.791205 26044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e8b57c1b80240d483c7ca4e95d74673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.819116 26052 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
W20260709 12:23:16.796020 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 146549us
I20260709 12:23:16.829365 26115 tablet_service.cc:1511] Processing CreateTablet for tablet 6389d613e36a42958d24b00099843b93 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:16.844208 26115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6389d613e36a42958d24b00099843b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.848557 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:16.854020 26315 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:16.869590 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:16.869570 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:16.870502 26201 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:16.832037 26243 raft_consensus.cc:359] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.871985 26243 raft_consensus.cc:385] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.872380 26243 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.940828 26245 tablet_bootstrap.cc:492] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
W20260709 12:23:16.908535 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60165.3us
I20260709 12:23:16.942106 26245 ts_tablet_manager.cc:1403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.310s	user 0.029s	sys 0.059s
I20260709 12:23:16.942724 26243 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:16.943661 26020 leader_election.cc:343] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.944461 26019 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:16.944834 26245 raft_consensus.cc:359] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.945439 26325 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.945592 26245 raft_consensus.cc:385] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.946463 26245 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.946798 26325 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.947249 26325 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.947129 26245 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.949671 26245 ts_tablet_manager.cc:1434] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
W20260709 12:23:16.939476 25938 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 150151us
W20260709 12:23:16.939697 25939 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 150218us
I20260709 12:23:16.878427 26315 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.957418 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:16.958094 26054 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.958745 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:16.960477 26192 tablet_service.cc:1511] Processing CreateTablet for tablet d2b3e794c66a4dc19b6590272eae81f7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:16.962260 26113 tablet_service.cc:1511] Processing CreateTablet for tablet d2b3e794c66a4dc19b6590272eae81f7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:16.963583 26113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b3e794c66a4dc19b6590272eae81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.963990 26054 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:16.965294 26188 tablet_service.cc:1511] Processing CreateTablet for tablet 6389d613e36a42958d24b00099843b93 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:16.969027 26043 tablet_service.cc:1511] Processing CreateTablet for tablet d2b3e794c66a4dc19b6590272eae81f7 (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:16.970235 26043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b3e794c66a4dc19b6590272eae81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.965245 26315 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:23:16.970166 26169 leader_election.cc:343] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.972663 26042 tablet_service.cc:1511] Processing CreateTablet for tablet 0bba37d4e722416198d4e0aa03fc3d5f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:16.973284 26118 tablet_service.cc:1511] Processing CreateTablet for tablet 0bba37d4e722416198d4e0aa03fc3d5f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:16.974012 26042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bba37d4e722416198d4e0aa03fc3d5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.974325 26118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bba37d4e722416198d4e0aa03fc3d5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.977651 26244 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:16.978313 26244 ts_tablet_manager.cc:1403] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.448s	user 0.012s	sys 0.013s
I20260709 12:23:16.980944 26244 raft_consensus.cc:359] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.981595 26244 raft_consensus.cc:385] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.981851 26244 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.982465 26244 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.985149 26325 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:16.985955 26244 ts_tablet_manager.cc:1434] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:16.986966 26244 tablet_bootstrap.cc:492] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:16.992367 26168 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:16.994455 26315 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:16.988710 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:17.006171 26282 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.006946 26282 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:17.008993 26243 ts_tablet_manager.cc:1434] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.338s	user 0.016s	sys 0.012s
I20260709 12:23:17.008985 26282 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.010706 26243 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.012410 26325 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.014389 26245 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:17.016323 26192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b3e794c66a4dc19b6590272eae81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.018826 26188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6389d613e36a42958d24b00099843b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.025462 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:17.026969 26020 leader_election.cc:343] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.028694 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:17.029479 26203 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.034677 26191 tablet_service.cc:1511] Processing CreateTablet for tablet 0bba37d4e722416198d4e0aa03fc3d5f (DEFAULT_TABLE table=test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:17.036445 26191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bba37d4e722416198d4e0aa03fc3d5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.042662 26203 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:17.046198 26019 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.047354 26263 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.047828 26263 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:17.048705 26263 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.051808 26245 tablet_bootstrap.cc:654] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.070104 26243 tablet_bootstrap.cc:654] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.074983 26244 tablet_bootstrap.cc:654] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.082285 25951 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.099826 26245 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.100564 26245 ts_tablet_manager.cc:1403] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.007s
I20260709 12:23:17.110623 26243 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.111444 26243 ts_tablet_manager.cc:1403] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.101s	user 0.008s	sys 0.008s
I20260709 12:23:17.124495 26245 raft_consensus.cc:359] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.125295 26245 raft_consensus.cc:385] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.125612 26245 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.126358 26245 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.134017 26245 ts_tablet_manager.cc:1434] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 12:23:17.135074 26245 tablet_bootstrap.cc:492] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:17.135668 26244 tablet_bootstrap.cc:492] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.136175 26244 ts_tablet_manager.cc:1403] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.149s	user 0.020s	sys 0.022s
I20260709 12:23:17.138893 26244 raft_consensus.cc:359] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.139549 26244 raft_consensus.cc:385] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.139813 26244 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.140332 26243 raft_consensus.cc:359] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.140988 26243 raft_consensus.cc:385] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.140690 26244 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.141263 26243 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.142033 26243 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.142802 26245 tablet_bootstrap.cc:654] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.143229 26244 ts_tablet_manager.cc:1434] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:23:17.144136 26244 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.146963 26243 ts_tablet_manager.cc:1434] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 12:23:17.148008 26243 tablet_bootstrap.cc:492] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.150272 26244 tablet_bootstrap.cc:654] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.154642 26243 tablet_bootstrap.cc:654] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.154702 26248 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.155520 26248 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.160017 26245 tablet_bootstrap.cc:492] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.160605 26248 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.161234 26245 ts_tablet_manager.cc:1403] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.009s
I20260709 12:23:17.164402 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.167204 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.168025 26054 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:17.168395 26244 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.168890 26244 ts_tablet_manager.cc:1403] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.007s
I20260709 12:23:17.169193 26094 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.170542 26245 raft_consensus.cc:359] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.171363 26245 raft_consensus.cc:385] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.171686 26245 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.172411 26243 tablet_bootstrap.cc:492] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.173050 26243 ts_tablet_manager.cc:1403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 12:23:17.172425 26245 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.173532 26244 raft_consensus.cc:359] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.173990 26248 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.174224 26244 raft_consensus.cc:385] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.174511 26244 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.175387 26244 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.176126 26243 raft_consensus.cc:359] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.177139 26245 ts_tablet_manager.cc:1434] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:23:17.177887 26244 ts_tablet_manager.cc:1434] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:23:17.178829 26245 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:17.179003 26244 tablet_bootstrap.cc:492] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.177281 26243 raft_consensus.cc:385] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.180193 26243 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.181267 26243 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.186162 26244 tablet_bootstrap.cc:654] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.186162 26245 tablet_bootstrap.cc:654] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.186654 26243 ts_tablet_manager.cc:1434] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:23:17.187673 26243 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.195851 26243 tablet_bootstrap.cc:654] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.198736 26263 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.199298 26263 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.201520 26263 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.209786 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.210748 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.211424 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:17.211658 26248 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:17.211758 26020 leader_election.cc:343] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.213665 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.214860 26263 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.215258 26263 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.215603 26263 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.222419 25952 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.224303 26263 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.224303 25952 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.227540 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:17.227658 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.228135 26203 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:17.229591 26020 leader_election.cc:343] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.234172 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:17.235574 26263 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.235606 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.237926 26263 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.238447 26315 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.238976 26315 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.239324 26263 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:17.241281 26244 tablet_bootstrap.cc:492] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.241837 26244 ts_tablet_manager.cc:1403] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.010s
I20260709 12:23:17.242187 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.242695 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.243541 26126 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:17.240337 26263 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.244930 26244 raft_consensus.cc:359] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.245687 26244 raft_consensus.cc:385] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.241310 26315 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.242870 26054 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:17.245342 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:17.246088 26244 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.247869 26315 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.248325 26315 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.248772 26315 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.248030 26244 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.250770 26243 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.251027 26244 ts_tablet_manager.cc:1434] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:23:17.251232 26243 ts_tablet_manager.cc:1403] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.023s
I20260709 12:23:17.252038 26244 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.253901 26243 raft_consensus.cc:359] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.254542 26243 raft_consensus.cc:385] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.254804 26243 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.255554 26243 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.256124 26245 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.255435 25952 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.256574 26245 ts_tablet_manager.cc:1403] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.033s
I20260709 12:23:17.259632 26244 tablet_bootstrap.cc:654] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.261427 26315 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.262879 26243 ts_tablet_manager.cc:1434] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:23:17.264379 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:17.259310 26245 raft_consensus.cc:359] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.265035 26054 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.265210 26245 raft_consensus.cc:385] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.265588 26245 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.266234 26315 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.266934 26245 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.269164 26245 ts_tablet_manager.cc:1434] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 12:23:17.273042 26243 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.273471 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.274055 26126 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.274551 26054 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:17.275826 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:17.276750 26329 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.277251 26329 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:17.278084 26329 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.279971 26243 tablet_bootstrap.cc:654] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.281750 26126 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:17.284351 26245 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:17.291818 26244 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.292191 26245 tablet_bootstrap.cc:654] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.292376 26244 ts_tablet_manager.cc:1403] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.005s
I20260709 12:23:17.294368 26243 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.294896 26243 ts_tablet_manager.cc:1403] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.002s
I20260709 12:23:17.294871 25952 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.295085 26244 raft_consensus.cc:359] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.296371 26244 raft_consensus.cc:385] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.296693 26244 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.297525 26244 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.299772 26244 ts_tablet_manager.cc:1434] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:17.299598 26243 raft_consensus.cc:359] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.303939 26243 raft_consensus.cc:385] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.304555 26263 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.308323 26243 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.308439 26263 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.309571 26243 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.310593 26263 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.311846 26243 ts_tablet_manager.cc:1434] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:23:17.312992 26243 tablet_bootstrap.cc:492] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.313489 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.314146 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.314826 26203 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:17.305084 26244 tablet_bootstrap.cc:492] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
W20260709 12:23:17.315758 26020 leader_election.cc:343] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.316517 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.319967 26263 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.320164 26243 tablet_bootstrap.cc:654] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.320438 26263 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.320875 26263 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.320437 26245 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.324811 26245 ts_tablet_manager.cc:1403] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.012s
I20260709 12:23:17.327478 26245 raft_consensus.cc:359] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.328226 26245 raft_consensus.cc:385] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.328006 26263 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.328539 26245 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.329210 26245 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.331175 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.332114 26245 ts_tablet_manager.cc:1434] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:17.333118 26245 tablet_bootstrap.cc:492] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
W20260709 12:23:17.336724 26020 leader_election.cc:343] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.339460 26245 tablet_bootstrap.cc:654] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.342990 26244 tablet_bootstrap.cc:654] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.350008 26243 tablet_bootstrap.cc:492] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.350656 26243 ts_tablet_manager.cc:1403] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.015s
I20260709 12:23:17.351748 26244 tablet_bootstrap.cc:492] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.352330 26244 ts_tablet_manager.cc:1403] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.007s
I20260709 12:23:17.353809 26243 raft_consensus.cc:359] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.354562 26243 raft_consensus.cc:385] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.354822 26243 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.355603 26243 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.357786 26243 ts_tablet_manager.cc:1434] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:17.358729 26243 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.360949 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:17.361801 26203 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.365214 26243 tablet_bootstrap.cc:654] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.367653 26244 raft_consensus.cc:359] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.372624 26244 raft_consensus.cc:385] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.373149 26244 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.374369 26244 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.371843 26263 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.378149 26244 ts_tablet_manager.cc:1434] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:23:17.379069 26203 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:17.380735 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.381989 26244 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.382494 26263 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.382969 26263 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:17.384138 26245 tablet_bootstrap.cc:492] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.383796 26263 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.385033 26245 ts_tablet_manager.cc:1403] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.052s	user 0.034s	sys 0.013s
I20260709 12:23:17.388017 26243 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.387863 26245 raft_consensus.cc:359] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.388487 26243 ts_tablet_manager.cc:1403] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.012s
I20260709 12:23:17.388715 26244 tablet_bootstrap.cc:654] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.388760 26245 raft_consensus.cc:385] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.389297 26245 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.390010 26245 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.391144 26243 raft_consensus.cc:359] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.391796 26243 raft_consensus.cc:385] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.392169 26243 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.392524 26245 ts_tablet_manager.cc:1434] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:23:17.393180 26243 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.393707 26245 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:17.399341 26243 ts_tablet_manager.cc:1434] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:23:17.400564 26243 tablet_bootstrap.cc:492] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.403265 26245 tablet_bootstrap.cc:654] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.407327 26243 tablet_bootstrap.cc:654] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.415143 26244 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.415222 26243 tablet_bootstrap.cc:492] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.415726 26244 ts_tablet_manager.cc:1403] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.034s	user 0.027s	sys 0.003s
I20260709 12:23:17.415772 26243 ts_tablet_manager.cc:1403] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:23:17.417623 26248 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.418396 26248 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.419422 26263 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.419833 26245 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.419893 26263 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.420432 26245 ts_tablet_manager.cc:1403] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.002s
I20260709 12:23:17.421617 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.422415 26203 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:17.422154 26244 raft_consensus.cc:359] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.422842 26244 raft_consensus.cc:385] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.423142 26244 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.423244 26263 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.420492 26248 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.423496 26245 raft_consensus.cc:359] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.424345 26245 raft_consensus.cc:385] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.424540 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.424700 26245 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.424714 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.424942 26243 raft_consensus.cc:359] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.425546 26245 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.426246 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.425757 26243 raft_consensus.cc:385] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.427054 26243 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.427258 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.427842 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:17.427713 26243 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.428054 26245 ts_tablet_manager.cc:1434] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:23:17.429021 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:17.429768 26263 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.429909 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:17.430167 26263 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.430569 26263 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.433133 26244 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.434499 26248 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.437232 26244 ts_tablet_manager.cc:1434] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 12:23:17.438330 26244 tablet_bootstrap.cc:492] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.438267 26263 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.440474 26263 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.441078 26243 ts_tablet_manager.cc:1434] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:23:17.442020 26243 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.442091 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:17.443182 26201 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.446506 26244 tablet_bootstrap.cc:654] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.448520 26243 tablet_bootstrap.cc:654] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.429227 26245 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:17.442625 26248 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.452965 26312 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.453444 26312 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.455035 25952 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.457877 26312 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.459570 26201 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:17.461367 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:17.462538 26263 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.464463 26325 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.465090 26325 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:17.466214 26325 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.469215 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.469830 26126 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.471666 26315 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.476574 26315 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.478813 26315 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.473589 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.480858 26054 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:17.481388 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.483688 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.484138 26127 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:17.487452 26169 leader_election.cc:343] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.488469 26303 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.490063 26245 tablet_bootstrap.cc:654] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.490128 26054 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:17.491843 26168 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.493687 26282 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.494187 26282 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.494652 26282 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.499603 26243 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.500275 26243 ts_tablet_manager.cc:1403] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.058s	user 0.019s	sys 0.021s
I20260709 12:23:17.508220 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.509086 26244 tablet_bootstrap.cc:492] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.509822 26244 ts_tablet_manager.cc:1403] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.000s
I20260709 12:23:17.491029 26315 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.515877 26243 raft_consensus.cc:359] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.516695 26243 raft_consensus.cc:385] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.517011 26243 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.517575 26243 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.519690 26243 ts_tablet_manager.cc:1434] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 12:23:17.521281 26329 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.523501 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:17.526562 26244 raft_consensus.cc:359] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.527292 26244 raft_consensus.cc:385] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.527585 26244 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.527679 26282 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.528910 26244 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.515094 26315 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.531478 26244 ts_tablet_manager.cc:1434] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.002s
I20260709 12:23:17.532464 26244 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.533481 26363 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:17.537176 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.537807 26052 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:17.535383 26248 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.534389 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:17.529134 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.544677 26315 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.530227 26329 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:23:17.546273 26248 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.548430 26248 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.548955 26054 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.551460 26303 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.551959 26303 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.559269 26243 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.542187 26282 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.550760 26248 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.553618 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.571125 26129 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.571278 26245 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.571846 26245 ts_tablet_manager.cc:1403] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.143s	user 0.016s	sys 0.004s
I20260709 12:23:17.572192 26303 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.574649 26245 raft_consensus.cc:359] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.575423 26245 raft_consensus.cc:385] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.575716 26245 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.569468 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.566560 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.576718 26245 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.577534 26128 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.578990 26245 ts_tablet_manager.cc:1434] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:17.579923 26245 tablet_bootstrap.cc:492] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
W20260709 12:23:17.561532 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43551.8us
I20260709 12:23:17.582298 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.582597 26168 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:23:17.585170 26020 leader_election.cc:343] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.588332 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:23:17.589465 26169 leader_election.cc:343] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.630439 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.643766 26054 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:17.656935 26366 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.657361 26366 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.657702 26168 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:23:17.655741 26019 leader_election.cc:343] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.658375 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.658531 26282 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.660545 26315 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.657698 26366 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.655400 26329 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.671666 26329 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.659049 26282 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:17.676872 26244 tablet_bootstrap.cc:654] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.656216 26094 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.679618 26248 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.680070 26248 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.669234 26325 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.694155 25951 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.697702 26282 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.725343 26303 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.725710 26329 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.725912 26303 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.727945 26303 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.728508 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:17.758385 26169 leader_election.cc:343] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.765496 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.765708 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.766549 26052 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:17.750164 26128 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
W20260709 12:23:17.773640 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 112179us
I20260709 12:23:17.775045 26263 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:17.776296 26020 leader_election.cc:343] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.767884 26123 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.788751 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:17.781478 26263 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.795511 26243 tablet_bootstrap.cc:654] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.785640 26248 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.796589 26315 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:17.828572 26364 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.005s
I20260709 12:23:17.813709 26315 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:17.841706 26368 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.146s	user 0.003s	sys 0.000s
I20260709 12:23:17.844892 26129 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.848762 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:17.854458 26244 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:17.855219 26244 ts_tablet_manager.cc:1403] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.323s	user 0.021s	sys 0.016s
I20260709 12:23:17.856688 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.845017 26315 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:17.858793 26020 leader_election.cc:343] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.844949 26312 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.861984 26263 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.864006 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.866743 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.867563 26203 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:17.869069 26245 tablet_bootstrap.cc:654] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.871258 26366 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.871881 26325 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.877147 26244 raft_consensus.cc:359] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.877871 26244 raft_consensus.cc:385] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.878176 26244 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.879325 26244 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.881170 26366 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.881752 26366 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.884298 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.885888 26383 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.886235 26243 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:17.886667 26243 ts_tablet_manager.cc:1403] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.328s	user 0.012s	sys 0.008s
I20260709 12:23:17.880039 26248 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.891927 26244 ts_tablet_manager.cc:1434] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 12:23:17.889293 26325 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:17.892935 26244 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:17.893332 26248 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.901254 26366 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.902104 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.902925 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:17.908735 26245 tablet_bootstrap.cc:492] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:17.909476 26245 ts_tablet_manager.cc:1403] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.330s	user 0.014s	sys 0.007s
I20260709 12:23:17.905412 25952 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.913669 26263 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.896690 26315 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.917176 26201 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:17.918087 26315 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:17.904862 26054 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.923382 26369 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:23:17.928576 26243 raft_consensus.cc:359] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.929366 26243 raft_consensus.cc:385] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.929680 26243 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.924692 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.930384 26243 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.932511 26243 ts_tablet_manager.cc:1434] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
W20260709 12:23:17.933034 26019 leader_election.cc:343] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.933534 26243 tablet_bootstrap.cc:492] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:17.933596 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.935544 26303 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.938624 26123 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.939399 26325 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.945922 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:17.956015 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:17.957289 26053 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.957289 26248 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:17.959861 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:17.964766 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:17.965102 26054 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:17.967546 26244 tablet_bootstrap.cc:654] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.968719 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.968947 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:17.970422 26248 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:17.968854 26123 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
W20260709 12:23:17.976514 26169 leader_election.cc:343] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.976791 26245 raft_consensus.cc:359] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.977546 26245 raft_consensus.cc:385] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.977864 26245 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.978986 26245 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:17.986752 26053 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:17.987358 26303 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.988093 26245 ts_tablet_manager.cc:1434] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.078s	user 0.004s	sys 0.003s
I20260709 12:23:17.992398 26168 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:17.999341 26245 tablet_bootstrap.cc:492] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.002179 26243 tablet_bootstrap.cc:654] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.000623 26315 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.013583 26315 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:18.014802 26315 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.021504 26366 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.027981 26366 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.032684 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.033736 26366 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:18.042045 26169 leader_election.cc:343] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.048110 26282 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.048842 26282 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.051602 26282 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.054317 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:18.056553 26169 leader_election.cc:343] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.115401 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.118628 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.119582 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:18.125658 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.129125 26053 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:18.149250 26244 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.149799 26244 ts_tablet_manager.cc:1403] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.257s	user 0.045s	sys 0.031s
W20260709 12:23:18.150656 26395 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 12:23:18.152659 26245 tablet_bootstrap.cc:654] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.153520 26244 raft_consensus.cc:359] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.154327 26244 raft_consensus.cc:385] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.154615 26244 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.155411 26244 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.145150 26366 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.159544 26366 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:18.162614 26244 ts_tablet_manager.cc:1434] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:23:18.163661 26244 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.166525 26366 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.178822 26244 tablet_bootstrap.cc:654] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.169853 26315 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:23:18.191524 26263 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:18.192637 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:18.192952 26303 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.192368 26263 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.196708 26263 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.197552 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:18.199416 26020 leader_election.cc:343] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.202437 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.213416 26315 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.213961 26315 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.214408 26315 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.215430 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.216305 26282 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.216717 26282 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.217137 26282 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.226316 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.225836 25951 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:18.227758 26019 leader_election.cc:343] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.228276 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.228385 25951 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.229499 26263 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.237434 26243 tablet_bootstrap.cc:492] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:18.237962 26243 ts_tablet_manager.cc:1403] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.305s	user 0.007s	sys 0.009s
I20260709 12:23:18.241988 26244 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.242476 26244 ts_tablet_manager.cc:1403] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.000s
I20260709 12:23:18.246760 26244 raft_consensus.cc:359] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.251614 26244 raft_consensus.cc:385] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.252039 26244 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.252449 26245 tablet_bootstrap.cc:492] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.252910 26245 ts_tablet_manager.cc:1403] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.254s	user 0.018s	sys 0.009s
I20260709 12:23:18.254870 26244 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.255867 26245 raft_consensus.cc:359] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.256613 26245 raft_consensus.cc:385] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.256839 26245 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.257402 26244 ts_tablet_manager.cc:1434] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:23:18.257361 26245 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.258430 26244 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.248683 26243 raft_consensus.cc:359] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.260363 26243 raft_consensus.cc:385] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.263489 26243 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.264343 26244 tablet_bootstrap.cc:654] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.264370 26243 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.266724 26243 ts_tablet_manager.cc:1434] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.001s
I20260709 12:23:18.267462 26366 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.268918 26245 ts_tablet_manager.cc:1434] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:23:18.269918 26245 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.246832 26282 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.250253 26315 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.273345 26244 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.273885 26244 ts_tablet_manager.cc:1403] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.016s	user 0.002s	sys 0.009s
I20260709 12:23:18.274085 26315 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.274726 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:18.275334 26053 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.276324 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:18.278154 26169 leader_election.cc:343] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.279373 26244 raft_consensus.cc:359] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.280131 26244 raft_consensus.cc:385] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.280561 26244 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.281294 26244 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.283321 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:18.283717 26244 ts_tablet_manager.cc:1434] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.006s
I20260709 12:23:18.284832 26244 tablet_bootstrap.cc:492] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.284869 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.286299 26315 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.286955 26315 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:18.287335 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:18.287839 26315 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.289260 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:18.290140 26050 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.291934 26244 tablet_bootstrap.cc:654] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.289266 26169 leader_election.cc:343] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.307221 26244 tablet_bootstrap.cc:492] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.307811 26244 ts_tablet_manager.cc:1403] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.007s
I20260709 12:23:18.309553 26243 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:18.308030 25951 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.311223 26244 raft_consensus.cc:359] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.311964 26244 raft_consensus.cc:385] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.312309 26244 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.315524 26243 tablet_bootstrap.cc:654] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.313199 26244 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.316332 26282 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.318284 26244 ts_tablet_manager.cc:1434] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:23:18.319305 26244 tablet_bootstrap.cc:492] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.322655 26050 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:18.324047 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.325243 26282 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.325847 26244 tablet_bootstrap.cc:654] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.325876 26282 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:18.327445 26282 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.333560 26245 tablet_bootstrap.cc:654] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.363045 25951 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.365190 26244 tablet_bootstrap.cc:492] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.365724 26244 ts_tablet_manager.cc:1403] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.047s	user 0.024s	sys 0.000s
I20260709 12:23:18.368664 26245 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.369127 26245 ts_tablet_manager.cc:1403] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.099s	user 0.006s	sys 0.021s
I20260709 12:23:18.369345 26244 raft_consensus.cc:359] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.370126 26244 raft_consensus.cc:385] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.370388 26244 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.370558 26243 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:18.371053 26243 ts_tablet_manager.cc:1403] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.014s
I20260709 12:23:18.371162 26244 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.371873 26245 raft_consensus.cc:359] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.372805 26245 raft_consensus.cc:385] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.373064 26245 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.373291 26244 ts_tablet_manager.cc:1434] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:18.373663 26243 raft_consensus.cc:359] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.374205 26244 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.373809 26245 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.374341 26243 raft_consensus.cc:385] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.374689 26243 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.375403 26243 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.376477 26245 ts_tablet_manager.cc:1434] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:23:18.377465 26245 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.377630 26243 ts_tablet_manager.cc:1434] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:18.379057 26243 tablet_bootstrap.cc:492] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:18.383476 26245 tablet_bootstrap.cc:654] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.385073 26243 tablet_bootstrap.cc:654] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.400816 26244 tablet_bootstrap.cc:654] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.420909 26245 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.421546 26245 ts_tablet_manager.cc:1403] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.013s
I20260709 12:23:18.425119 26245 raft_consensus.cc:359] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.426061 26245 raft_consensus.cc:385] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.426405 26245 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.427656 26245 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.431155 26245 ts_tablet_manager.cc:1434] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:18.433231 26245 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.444176 26263 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.444736 26263 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.448050 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.448304 26263 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:18.450291 26020 leader_election.cc:343] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.450750 26245 tablet_bootstrap.cc:654] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.451587 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:18.454294 26019 leader_election.cc:343] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.454751 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.456501 26263 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.464568 26244 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.465011 26244 ts_tablet_manager.cc:1403] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.091s	user 0.029s	sys 0.031s
I20260709 12:23:18.465399 26243 tablet_bootstrap.cc:492] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:18.465865 26243 ts_tablet_manager.cc:1403] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.087s	user 0.030s	sys 0.053s
I20260709 12:23:18.467602 26244 raft_consensus.cc:359] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.468294 26244 raft_consensus.cc:385] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.468660 26244 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.468307 26243 raft_consensus.cc:359] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.469015 26243 raft_consensus.cc:385] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.469285 26243 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.469395 26244 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.471472 26244 ts_tablet_manager.cc:1434] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:18.469936 26243 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.473932 26243 ts_tablet_manager.cc:1434] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:23:18.474717 26245 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.475190 26244 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.474972 26282 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.475209 26245 ts_tablet_manager.cc:1403] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.004s
I20260709 12:23:18.479146 26245 raft_consensus.cc:359] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.479995 26245 raft_consensus.cc:385] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.480372 26245 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.481317 26245 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.482272 26244 tablet_bootstrap.cc:654] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.482563 26303 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.483050 26303 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.484526 26243 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:18.485890 26303 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.487396 26248 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.488067 26248 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.490861 26245 ts_tablet_manager.cc:1434] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.007s
I20260709 12:23:18.490916 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.487383 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.492208 26245 tablet_bootstrap.cc:492] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
W20260709 12:23:18.495357 26019 leader_election.cc:343] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:18.496583 26020 leader_election.cc:343] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.496675 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.497201 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.497506 26050 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:18.498122 26245 tablet_bootstrap.cc:654] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.498600 26303 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.499156 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.490751 26248 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.501281 26094 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.506091 26248 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.507493 26282 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.512137 26243 tablet_bootstrap.cc:654] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.514482 26248 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.546319 26245 tablet_bootstrap.cc:492] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.546789 26245 ts_tablet_manager.cc:1403] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.032s
I20260709 12:23:18.553722 26245 raft_consensus.cc:359] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.559239 26245 raft_consensus.cc:385] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.559670 26245 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.560544 26245 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.563345 26245 ts_tablet_manager.cc:1434] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:18.564692 26245 tablet_bootstrap.cc:492] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.570427 26244 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.570904 26244 ts_tablet_manager.cc:1403] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.096s	user 0.022s	sys 0.021s
I20260709 12:23:18.571254 26243 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:18.571713 26243 ts_tablet_manager.cc:1403] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.018s
I20260709 12:23:18.572381 26245 tablet_bootstrap.cc:654] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.573450 26244 raft_consensus.cc:359] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.574113 26244 raft_consensus.cc:385] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.574494 26244 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.574131 26243 raft_consensus.cc:359] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.574863 26243 raft_consensus.cc:385] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.575132 26243 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.575776 26243 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.575153 26244 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.577718 26243 ts_tablet_manager.cc:1434] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:23:18.578591 26243 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:18.584539 26243 tablet_bootstrap.cc:654] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.587448 26244 ts_tablet_manager.cc:1434] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 12:23:18.589308 26248 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.590258 26244 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.589941 26248 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.593919 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.594776 26052 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:18.595876 26243 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:18.596860 26243 ts_tablet_manager.cc:1403] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 12:23:18.599674 26248 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.601336 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.602903 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.603945 26248 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.605225 26243 raft_consensus.cc:359] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.605882 26245 tablet_bootstrap.cc:492] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.605897 26243 raft_consensus.cc:385] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.607061 26243 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.606600 26245 ts_tablet_manager.cc:1403] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.042s	user 0.028s	sys 0.011s
I20260709 12:23:18.607903 26243 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.610745 26248 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.610642 26245 raft_consensus.cc:359] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.611295 26245 raft_consensus.cc:385] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.611557 26245 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.607332 26244 tablet_bootstrap.cc:654] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.612192 26245 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.613075 26243 ts_tablet_manager.cc:1434] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:18.613983 26243 tablet_bootstrap.cc:492] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:18.614267 26245 ts_tablet_manager.cc:1434] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:18.618161 26245 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.624177 26245 tablet_bootstrap.cc:654] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.629426 26263 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.630019 26263 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.633117 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.635416 26244 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
W20260709 12:23:18.635399 26020 leader_election.cc:343] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.636049 26244 ts_tablet_manager.cc:1403] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.007s
I20260709 12:23:18.637989 26329 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.638814 26244 raft_consensus.cc:359] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.639500 26244 raft_consensus.cc:385] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.639770 26244 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.641453 26248 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.641907 26248 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.642582 26243 tablet_bootstrap.cc:654] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.642215 26244 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.644083 26248 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.644526 26244 ts_tablet_manager.cc:1434] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:23:18.645452 26244 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.645351 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.646287 26203 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:18.648433 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:18.649953 26248 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.651032 26248 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.651499 26248 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.657087 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.657857 26050 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:18.658341 26329 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.659029 26329 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.652806 26244 tablet_bootstrap.cc:654] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.663333 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.663750 26248 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.664006 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:18.666498 26366 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.666610 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:18.667244 26053 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.667192 26366 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.667388 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:18.669621 26383 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.670221 26383 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.671370 26244 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.671830 26244 ts_tablet_manager.cc:1403] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.002s
I20260709 12:23:18.672216 26315 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.672441 26383 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.672670 26315 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.673080 26315 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.674521 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.674577 26244 raft_consensus.cc:359] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.675475 26244 raft_consensus.cc:385] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.675833 26244 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.676083 26303 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.676501 26282 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.676606 26303 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.676994 26282 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.676642 26244 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.679447 26245 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.679000 26303 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.681806 26245 ts_tablet_manager.cc:1403] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.023s
I20260709 12:23:18.682296 26248 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.681368 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.681161 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.685099 26282 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.684854 26245 raft_consensus.cc:359] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:18.682300 26094 leader_election.cc:343] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.676083 26329 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.686934 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:18.685268 26050 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:18.685542 26245 raft_consensus.cc:385] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.687237 26244 ts_tablet_manager.cc:1434] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:23:18.687526 26203 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.688347 26244 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
W20260709 12:23:18.688835 26020 leader_election.cc:343] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.672672 26366 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.692170 26245 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.692929 26245 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.697800 26245 ts_tablet_manager.cc:1434] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:23:18.698518 26245 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:18.696635 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.700862 26053 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:18.714880 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.700438 26263 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.718187 26243 tablet_bootstrap.cc:492] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:18.720896 26243 ts_tablet_manager.cc:1403] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.001s
I20260709 12:23:18.704159 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.720038 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.721654 26202 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:18.721408 26050 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:18.724519 26243 raft_consensus.cc:359] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.725263 26243 raft_consensus.cc:385] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.725575 26243 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.701200 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.726289 26243 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.727138 26248 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.727380 26127 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:18.727556 26248 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.727902 26248 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:18.728785 26169 leader_election.cc:343] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.731503 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.732131 26202 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:18.735085 26243 ts_tablet_manager.cc:1434] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 12:23:18.735061 26248 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.736001 26329 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.736388 26243 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:18.737409 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.738585 26263 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.738952 26263 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.739289 26263 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.740381 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:18.750763 26244 tablet_bootstrap.cc:654] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.745754 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.750329 26315 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.751003 26263 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:18.747067 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50466.7us
I20260709 12:23:18.754516 26263 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.755527 26383 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.757472 26383 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.758131 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:18.759306 26245 tablet_bootstrap.cc:654] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.759446 26020 leader_election.cc:343] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.755921 26303 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.768146 26303 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.768699 26303 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.766877 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:18.781213 26248 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.782065 26248 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.782652 26248 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:18.784703 26315 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:18.759485 26383 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:18.755787 26169 leader_election.cc:343] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.809645 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:18.810287 26054 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.810269 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.811213 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
W20260709 12:23:18.812734 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52859.3us
I20260709 12:23:18.825604 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.825793 26203 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:18.827200 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:18.828081 26050 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:18.827927 26303 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:18.816149 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63400.1us
I20260709 12:23:18.830030 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.831235 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:18.833772 26050 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.834290 26315 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.834803 26315 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.835139 26315 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.834890 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:18.836427 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:18.836498 26282 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.841454 26303 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.839671 26125 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.845993 26050 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:18.847165 26054 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:18.838569 26366 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.823608 26248 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.848569 26366 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.848946 26366 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.852205 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:18.858605 26282 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.859434 26282 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:18.861681 26369 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.862259 26369 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.862792 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:18.863595 26203 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.870947 26244 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:18.874174 26243 tablet_bootstrap.cc:654] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.874737 26244 ts_tablet_manager.cc:1403] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.187s	user 0.020s	sys 0.018s
I20260709 12:23:18.867461 26366 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.871447 26282 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.886832 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:18.886325 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:18.899523 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:18.900161 26201 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.900401 26094 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:18.889897 26125 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:18.901455 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:18.901772 26245 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:18.902027 26202 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.902231 26245 ts_tablet_manager.cc:1403] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.204s	user 0.015s	sys 0.003s
I20260709 12:23:18.906793 26245 raft_consensus.cc:359] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.907711 26245 raft_consensus.cc:385] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.908070 26245 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.910835 26245 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.909564 26431 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.913976 26431 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:18.914955 26431 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.912575 26248 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.918387 26432 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.919499 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:18.920140 26200 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:18.922483 26245 ts_tablet_manager.cc:1434] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:23:18.923439 26245 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
W20260709 12:23:18.910260 26428 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:23:18.922017 26244 raft_consensus.cc:359] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.899060 26369 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:18.929085 26244 raft_consensus.cc:385] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.929651 26244 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.932219 26244 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:18.934378 26248 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.938908 25952 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.954145 26244 ts_tablet_manager.cc:1434] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.001s
I20260709 12:23:18.956202 26244 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:18.981225 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:18.944087 26201 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:18.982028 26052 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.987970 26019 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:23:18.999965 26020 leader_election.cc:343] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.001919 26429 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.002473 26429 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.002914 26429 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.997231 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:19.009352 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.010929 26369 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.011664 26366 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.000128 26315 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.009399 26436 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:23:19.030109 26369 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:19.045514 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.045830 26369 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.034368 26052 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.048205 26202 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:19.052588 26244 tablet_bootstrap.cc:654] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.061597 26303 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.062235 26303 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.075551 26303 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.082479 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.083745 26303 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:19.083971 26437 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.071s	user 0.003s	sys 0.000s
W20260709 12:23:19.084465 26020 leader_election.cc:343] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.084381 26303 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.086416 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.086663 26263 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.086745 26303 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.087285 26303 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.087106 26263 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.087841 26303 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:19.087972 26436 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.088913 26263 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.088721 26303 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.118343 26315 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.118871 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:19.120221 26020 leader_election.cc:343] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.120898 26245 tablet_bootstrap.cc:654] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.118662 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:19.121043 26429 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.122037 26315 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.122629 26315 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:19.124051 26429 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.128844 25950 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.130169 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:19.132169 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:19.133445 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:19.134030 26054 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.145180 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.133697 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:19.162093 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:19.146265 26169 leader_election.cc:343] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.124137 26315 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.172544 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.170753 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:19.179250 26203 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.183079 26243 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.183605 26243 ts_tablet_manager.cc:1403] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.447s	user 0.013s	sys 0.015s
I20260709 12:23:19.193236 26244 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:19.193763 26244 ts_tablet_manager.cc:1403] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.238s	user 0.009s	sys 0.014s
W20260709 12:23:19.201501 26169 leader_election.cc:343] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.202633 26054 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.202591 26243 raft_consensus.cc:359] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:19.203039 26020 leader_election.cc:343] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.203464 26243 raft_consensus.cc:385] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.203754 26243 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.196869 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.204702 26243 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.207415 26243 ts_tablet_manager.cc:1434] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 12:23:19.209076 26243 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
W20260709 12:23:19.206040 26019 leader_election.cc:343] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.211975 26019 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:23:19.217490 26019 leader_election.cc:343] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.218005 26019 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:19.219012 26369 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.219638 26369 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:19.220696 26019 leader_election.cc:343] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:19.222508 26020 leader_election.cc:343] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.223102 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:19.219805 26244 raft_consensus.cc:359] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.224537 26244 raft_consensus.cc:385] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.224926 26244 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.225751 26244 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.222831 25950 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.232450 26244 ts_tablet_manager.cc:1434] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.038s	user 0.011s	sys 0.000s
I20260709 12:23:19.233830 26244 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:19.231451 26429 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.236656 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:23:19.235163 26020 leader_election.cc:343] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.239367 26435 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.240024 26435 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:19.251114 25951 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.254104 25951 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.263888 26203 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:19.276492 26243 tablet_bootstrap.cc:654] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.283911 26248 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.284529 26248 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.285693 26019 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.287416 26429 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.287912 26429 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:19.288774 26429 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.289357 26435 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.294958 26244 tablet_bootstrap.cc:654] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.314486 26248 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:19.318214 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:19.320459 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:19.321659 26052 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:19.325585 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:19.327572 26094 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:19.328466 26248 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.331279 26366 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.338304 26245 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.339015 26245 ts_tablet_manager.cc:1403] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.416s	user 0.033s	sys 0.028s
I20260709 12:23:19.345609 26303 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.346261 26303 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.348486 26303 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.350725 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.350893 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:19.353473 26019 leader_election.cc:343] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:19.354431 26020 leader_election.cc:343] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.354962 26020 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:19.356446 26369 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.364552 26248 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.364398 26315 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.370685 26431 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.374993 26243 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.375528 26243 ts_tablet_manager.cc:1403] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.167s	user 0.005s	sys 0.023s
I20260709 12:23:19.375638 26245 raft_consensus.cc:359] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.376645 26245 raft_consensus.cc:385] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.377044 26245 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.377790 25951 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.388015 26243 raft_consensus.cc:359] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.389063 26243 raft_consensus.cc:385] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.389426 26243 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.377897 26245 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.392531 26245 ts_tablet_manager.cc:1434] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.004s
I20260709 12:23:19.394583 26245 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.399572 26243 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.399052 26244 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:19.401017 26244 ts_tablet_manager.cc:1403] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.168s	user 0.032s	sys 0.028s
I20260709 12:23:19.401616 26243 ts_tablet_manager.cc:1434] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.005s
I20260709 12:23:19.402395 26243 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:19.402861 26248 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.404085 26244 raft_consensus.cc:359] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.407335 26244 raft_consensus.cc:385] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.407629 26244 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.408528 26243 tablet_bootstrap.cc:654] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.408296 26244 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.410650 26245 tablet_bootstrap.cc:654] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.413609 26244 ts_tablet_manager.cc:1434] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:23:19.414644 26244 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:19.414079 25950 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.420713 26243 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.421393 26243 ts_tablet_manager.cc:1403] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 12:23:19.425202 25950 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.425520 26243 raft_consensus.cc:359] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.427670 26243 raft_consensus.cc:385] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.429100 26245 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.429596 26245 ts_tablet_manager.cc:1403] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.035s	user 0.025s	sys 0.000s
I20260709 12:23:19.427961 26243 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.430586 26243 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.431548 26244 tablet_bootstrap.cc:654] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.432649 26243 ts_tablet_manager.cc:1434] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:23:19.432682 26245 raft_consensus.cc:359] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.433539 26245 raft_consensus.cc:385] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.433698 26243 tablet_bootstrap.cc:492] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:19.433845 26245 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.434594 26245 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.436945 26245 ts_tablet_manager.cc:1434] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:19.437994 26245 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.438580 26128 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.440042 26243 tablet_bootstrap.cc:654] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.444555 26245 tablet_bootstrap.cc:654] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.447336 26369 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.453285 26243 tablet_bootstrap.cc:492] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.453926 26243 ts_tablet_manager.cc:1403] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 12:23:19.456328 26248 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.458578 26243 raft_consensus.cc:359] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.459434 26243 raft_consensus.cc:385] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.459774 26243 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.462024 26243 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.463904 26245 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.466427 26245 ts_tablet_manager.cc:1403] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.005s
I20260709 12:23:19.464391 26243 ts_tablet_manager.cc:1434] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:23:19.472458 26245 raft_consensus.cc:359] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.473146 26243 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:19.473232 26245 raft_consensus.cc:385] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.473690 26245 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.474308 26444 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.474432 26245 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.477078 26245 ts_tablet_manager.cc:1434] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:19.478041 26245 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.480993 26243 tablet_bootstrap.cc:654] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.485260 26245 tablet_bootstrap.cc:654] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.493000 26243 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.493546 26243 ts_tablet_manager.cc:1403] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.010s
I20260709 12:23:19.495663 26245 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.496196 26245 ts_tablet_manager.cc:1403] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 12:23:19.496225 26243 raft_consensus.cc:359] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.497051 26243 raft_consensus.cc:385] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.497344 26243 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.497956 26243 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.499193 26245 raft_consensus.cc:359] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.499887 26243 ts_tablet_manager.cc:1434] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:19.499868 26245 raft_consensus.cc:385] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.500392 26245 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.500923 26243 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:19.501257 26245 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.504429 26245 ts_tablet_manager.cc:1434] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:23:19.506839 26245 tablet_bootstrap.cc:492] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.511515 26248 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.534613 26244 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:19.535144 26244 ts_tablet_manager.cc:1403] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.121s	user 0.025s	sys 0.008s
I20260709 12:23:19.537750 26244 raft_consensus.cc:359] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.538455 26244 raft_consensus.cc:385] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.538740 26244 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.539955 26244 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.540975 26243 tablet_bootstrap.cc:654] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.542232 26244 ts_tablet_manager.cc:1434] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:23:19.543160 26244 tablet_bootstrap.cc:492] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:19.563086 26245 tablet_bootstrap.cc:654] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.566529 26435 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.567481 26435 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:19.569423 26020 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:19.571573 26435 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.574424 26244 tablet_bootstrap.cc:654] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.574206 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:19.576977 26169 leader_election.cc:343] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.581630 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:19.582954 26050 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:19.584753 26168 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.585790 26315 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.586231 26315 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.586647 26315 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.624938 26315 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.639674 26303 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.645320 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:19.646512 26050 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.647235 26243 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.647854 26243 ts_tablet_manager.cc:1403] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.147s	user 0.026s	sys 0.049s
I20260709 12:23:19.657152 26315 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.658087 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.662251 26243 raft_consensus.cc:359] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.663204 26243 raft_consensus.cc:385] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.663540 26243 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.664633 26243 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.668656 26243 ts_tablet_manager.cc:1434] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.003s
I20260709 12:23:19.669219 26127 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.669512 26244 tablet_bootstrap.cc:492] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:19.669977 26244 ts_tablet_manager.cc:1403] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.127s	user 0.030s	sys 0.030s
I20260709 12:23:19.670434 26243 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:19.673712 26244 raft_consensus.cc:359] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.674888 26244 raft_consensus.cc:385] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.675254 26244 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.679502 26244 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.681950 26436 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.686748 26244 ts_tablet_manager.cc:1434] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:23:19.687731 26244 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:19.698426 26050 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.704895 26168 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:19.705178 26243 tablet_bootstrap.cc:654] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.706986 26435 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.707491 26435 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:19.709151 26127 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.708384 26435 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.718974 26245 tablet_bootstrap.cc:492] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.719416 26245 ts_tablet_manager.cc:1403] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.214s	user 0.032s	sys 0.060s
I20260709 12:23:19.718685 26431 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.720383 26431 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.722141 26245 raft_consensus.cc:359] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.722872 26245 raft_consensus.cc:385] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.722563 26431 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.723148 26245 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.723218 26244 tablet_bootstrap.cc:654] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.723672 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:19.723855 26245 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.724130 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:19.725183 26054 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:19.726969 26202 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:19.727903 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:19.730516 26245 ts_tablet_manager.cc:1434] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:23:19.731058 26431 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.731604 26431 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.731778 26245 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.731931 26431 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.738511 25952 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.745016 26245 tablet_bootstrap.cc:654] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.744980 26282 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.749727 26127 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.753458 26282 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.753913 26282 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.756888 26282 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.757221 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:19.758328 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.759883 26125 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:19.761143 26052 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:19.761001 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:19.762585 26245 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.762053 26282 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.763082 26245 ts_tablet_manager.cc:1403] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 12:23:19.763051 26282 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.763684 26282 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.765087 26244 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:19.765532 26244 ts_tablet_manager.cc:1403] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.003s
I20260709 12:23:19.766265 26245 raft_consensus.cc:359] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.766961 26245 raft_consensus.cc:385] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.750903 26431 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.767413 26245 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.768146 26245 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.769050 26244 raft_consensus.cc:359] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.769712 26244 raft_consensus.cc:385] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.770679 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:19.770871 26436 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.770871 26435 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.771488 26436 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.773656 26436 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.774298 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:19.774947 26054 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:19.775472 26202 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.775797 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.776460 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:23:19.780817 26169 leader_election.cc:343] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.781661 26435 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.782068 26435 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.782433 26435 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.769065 26431 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.769987 26244 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.784260 26244 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.786540 26436 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.787159 26243 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.787653 26243 ts_tablet_manager.cc:1403] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.006s
I20260709 12:23:19.786852 26244 ts_tablet_manager.cc:1434] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:23:19.784838 26202 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:19.788877 26429 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.789434 26244 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:19.789811 26429 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.791844 26429 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.781677 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:19.793227 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:19.793844 26020 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:19.793855 26203 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:19.794986 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.795668 26244 tablet_bootstrap.cc:654] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.796008 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:23:19.796586 26020 leader_election.cc:343] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.796895 26429 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.793373 26052 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.797806 26429 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.797336 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:19.798208 26429 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.798949 26431 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.799419 26431 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:19.800333 26431 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.805702 26244 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:19.777060 26245 ts_tablet_manager.cc:1434] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:23:19.800328 26435 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.811353 26435 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.813724 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:19.814349 26053 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.814379 26243 raft_consensus.cc:359] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.815209 26243 raft_consensus.cc:385] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.815528 26243 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.815845 26052 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:19.819020 26429 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.820748 26282 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.821893 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.821502 26243 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:19.823058 26169 leader_election.cc:343] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.823892 26244 ts_tablet_manager.cc:1403] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.002s
I20260709 12:23:19.824503 26245 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.825637 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:19.826135 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.826296 26050 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:19.827428 26020 leader_election.cc:343] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.831524 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:19.831837 26429 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.832176 26053 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.832366 26202 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.833359 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.834110 26366 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.834590 26366 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:19.835417 26366 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.831317 26245 tablet_bootstrap.cc:654] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.837720 26243 ts_tablet_manager.cc:1434] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.050s	user 0.001s	sys 0.005s
I20260709 12:23:19.839066 26243 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:19.826462 26244 raft_consensus.cc:359] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.841130 26244 raft_consensus.cc:385] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.841526 26244 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.848372 26244 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.868400 26244 ts_tablet_manager.cc:1434] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:23:19.869324 26244 tablet_bootstrap.cc:492] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:19.869643 26243 tablet_bootstrap.cc:654] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.880839 26282 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.885370 25950 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.886340 26202 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:19.888157 26444 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.888697 26444 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.889894 26435 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.890379 26435 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.890528 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.891258 26125 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.894896 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:19.895254 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.895555 26053 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:19.896062 26429 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.896574 26429 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:19.896773 26050 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.901382 26444 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:19.902729 26245 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:19.904881 26245 ts_tablet_manager.cc:1403] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.000s
I20260709 12:23:19.893049 26435 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.897428 26429 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.907930 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.909642 26245 raft_consensus.cc:359] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.910310 26245 raft_consensus.cc:385] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.910565 26245 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.911188 26245 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.915644 25952 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.920938 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.923841 26263 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:19.926683 26169 leader_election.cc:343] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.930600 26245 ts_tablet_manager.cc:1434] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:23:19.931509 26245 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:19.932950 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:19.933590 26201 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:19.941231 26168 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.942847 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:19.944602 26435 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.945063 26435 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.945479 26435 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.946952 26244 tablet_bootstrap.cc:654] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:19.950779 26020 leader_election.cc:343] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.951597 26125 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:19.952921 26366 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.953672 26366 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:19.954896 26366 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.963703 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:19.966354 26263 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.966730 26263 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.967043 26263 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.969907 26435 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.971658 26435 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:19.974443 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.974534 26263 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.979209 26243 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:19.979388 26429 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.979717 26243 ts_tablet_manager.cc:1403] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.141s	user 0.030s	sys 0.037s
I20260709 12:23:19.979848 26429 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.978724 25952 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.985833 26245 tablet_bootstrap.cc:654] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.988214 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:19.988641 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:19.989270 26050 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.992046 26263 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:19.984174 26169 leader_election.cc:343] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.993168 26243 raft_consensus.cc:359] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:19.994519 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:19.994746 26243 raft_consensus.cc:385] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:19.994067 26020 leader_election.cc:343] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.995613 26243 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.997936 26243 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.000981 26127 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.010821 26125 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.011152 26050 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:20.012530 26168 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:20.013038 26243 ts_tablet_manager.cc:1434] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.004s
I20260709 12:23:20.013373 26127 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:20.013717 26432 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.014065 26243 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:20.014283 26432 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:20.015281 26432 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.018288 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:20.019155 26202 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.028352 26244 tablet_bootstrap.cc:492] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.028900 26244 ts_tablet_manager.cc:1403] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.160s	user 0.014s	sys 0.009s
I20260709 12:23:20.032537 26429 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.035068 26202 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:20.038688 26244 raft_consensus.cc:359] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.039541 26244 raft_consensus.cc:385] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.039851 26244 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.041095 26244 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.042943 26244 ts_tablet_manager.cc:1434] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:23:20.043728 26244 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.045048 26020 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:20.046023 26303 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.046566 26303 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:20.047605 26303 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.050526 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:20.055198 26019 leader_election.cc:343] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.056051 26019 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.056838 26429 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.055641 25951 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.057914 26243 tablet_bootstrap.cc:654] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.061403 26245 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.061800 26245 ts_tablet_manager.cc:1403] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.003s
I20260709 12:23:20.067193 26245 raft_consensus.cc:359] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.067804 26244 tablet_bootstrap.cc:654] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.067919 26245 raft_consensus.cc:385] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.068526 26245 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.069594 26245 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.072028 26245 ts_tablet_manager.cc:1434] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:20.072949 26245 tablet_bootstrap.cc:492] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.090678 26244 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.091312 26244 ts_tablet_manager.cc:1403] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.004s
I20260709 12:23:20.093859 26244 raft_consensus.cc:359] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.094508 26244 raft_consensus.cc:385] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.094755 26244 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.095348 26244 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.097679 26244 ts_tablet_manager.cc:1434] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:20.098580 26244 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.099103 26245 tablet_bootstrap.cc:654] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.102733 26243 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:20.103318 26243 ts_tablet_manager.cc:1403] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.005s
I20260709 12:23:20.102690 25950 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:20.105759 26244 tablet_bootstrap.cc:654] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.105944 26243 raft_consensus.cc:359] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.106611 26243 raft_consensus.cc:385] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.106859 26243 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.108821 26366 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.109293 26366 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.116137 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:20.117864 26169 leader_election.cc:343] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.111382 26366 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.119427 25951 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.122856 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.123472 26050 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:20.124763 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:20.125489 26435 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.125979 26435 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.125959 26243 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.126612 26435 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.128537 26383 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.128993 26383 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.130980 26383 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.134878 26243 ts_tablet_manager.cc:1434] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 12:23:20.136286 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:20.137185 26243 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:20.135349 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.135578 26435 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.140558 26435 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.141230 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:20.141789 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:20.143450 26169 leader_election.cc:343] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.144164 26050 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.146176 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.147310 26383 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.153515 26432 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.155759 26303 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.156383 26303 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.158699 26303 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.159731 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:20.161490 26020 leader_election.cc:343] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.164211 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:20.165665 26019 leader_election.cc:343] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.166072 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.166653 26303 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.176805 26366 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:20.193430 26245 tablet_bootstrap.cc:492] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.193812 26245 ts_tablet_manager.cc:1403] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.121s	user 0.023s	sys 0.012s
I20260709 12:23:20.195611 26245 raft_consensus.cc:359] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.196419 26245 raft_consensus.cc:385] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.196707 26245 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.197359 26245 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.199929 26245 ts_tablet_manager.cc:1434] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:23:20.201261 26245 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.203600 26243 tablet_bootstrap.cc:654] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.208288 26245 tablet_bootstrap.cc:654] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.206674 26050 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:20.213749 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:20.214843 26435 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.215389 26435 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:20.221560 26435 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.231127 26243 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:20.231693 26243 ts_tablet_manager.cc:1403] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.000s
I20260709 12:23:20.232295 26245 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.232609 26244 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.232774 26245 ts_tablet_manager.cc:1403] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.007s
I20260709 12:23:20.233053 26244 ts_tablet_manager.cc:1403] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.135s	user 0.036s	sys 0.061s
I20260709 12:23:20.234622 26243 raft_consensus.cc:359] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:20.235347 26169 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.235423 26244 raft_consensus.cc:359] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.236330 26244 raft_consensus.cc:385] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.235581 26245 raft_consensus.cc:359] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.236707 26244 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.236865 26245 raft_consensus.cc:385] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.237352 26245 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.238361 26245 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.238699 26244 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.240473 26244 ts_tablet_manager.cc:1434] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:20.241436 26244 tablet_bootstrap.cc:492] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.241623 26245 ts_tablet_manager.cc:1434] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:20.243285 26245 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.235405 26243 raft_consensus.cc:385] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.243888 26243 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.244908 26243 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.247701 26243 ts_tablet_manager.cc:1434] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.002s
I20260709 12:23:20.249189 26244 tablet_bootstrap.cc:654] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.251479 26243 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:20.256594 25950 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.267689 26125 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.269150 26245 tablet_bootstrap.cc:654] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.279366 26243 tablet_bootstrap.cc:654] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.280169 26244 tablet_bootstrap.cc:492] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.280678 26244 ts_tablet_manager.cc:1403] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.009s
I20260709 12:23:20.282032 26383 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.283532 26244 raft_consensus.cc:359] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.284323 26244 raft_consensus.cc:385] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.284679 26244 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.285333 26244 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.288986 26244 ts_tablet_manager.cc:1434] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:23:20.290107 26244 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.302980 26435 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.304113 26245 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.304775 26245 ts_tablet_manager.cc:1403] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.006s
I20260709 12:23:20.308836 26245 raft_consensus.cc:359] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.309517 26245 raft_consensus.cc:385] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.309868 26245 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.311304 26244 tablet_bootstrap.cc:654] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.310657 26245 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.314191 26383 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.319344 26245 ts_tablet_manager.cc:1434] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:23:20.320353 26245 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
W20260709 12:23:20.328895 26020 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.331470 26303 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:20.328138 26245 tablet_bootstrap.cc:654] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.349614 26244 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.368145 26244 ts_tablet_manager.cc:1403] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.003s
I20260709 12:23:20.383776 26243 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:20.384374 26243 ts_tablet_manager.cc:1403] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.133s	user 0.039s	sys 0.036s
I20260709 12:23:20.384866 26244 raft_consensus.cc:359] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.386834 26244 raft_consensus.cc:385] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.387095 26244 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.387923 26244 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.388499 26243 raft_consensus.cc:359] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.388903 26366 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.389164 26243 raft_consensus.cc:385] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.391341 26243 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.392009 26243 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.393361 26244 ts_tablet_manager.cc:1434] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:23:20.394184 26244 tablet_bootstrap.cc:492] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.391875 26366 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.399748 26243 ts_tablet_manager.cc:1434] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:23:20.400787 26243 tablet_bootstrap.cc:492] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:20.402217 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.402882 26050 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:20.406265 26168 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:20.408191 26366 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.408728 26366 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.409060 26366 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.409376 26366 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.429944 26282 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.430519 26282 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.434530 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:20.437371 26436 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.435978 26366 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.447921 26244 tablet_bootstrap.cc:654] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.453411 26282 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.458346 26243 tablet_bootstrap.cc:654] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.458652 26366 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.465023 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.465736 26053 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
W20260709 12:23:20.466281 26169 leader_election.cc:343] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.467602 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:20.468322 26245 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.468809 26245 ts_tablet_manager.cc:1403] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.149s	user 0.019s	sys 0.018s
I20260709 12:23:20.468624 26053 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.468595 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:20.471901 26517 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.477274 26168 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:20.481099 26436 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.487996 26436 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.488533 26436 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.476212 26517 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:20.501566 26169 consensus_peers.cc:597] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.519260 26263 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.520730 26245 raft_consensus.cc:359] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.522879 26245 raft_consensus.cc:385] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.523169 26245 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
W20260709 12:23:20.523108 26169 leader_election.cc:343] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.525408 26263 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.527513 26245 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.529455 26244 tablet_bootstrap.cc:492] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.528451 26243 tablet_bootstrap.cc:492] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:20.529913 26244 ts_tablet_manager.cc:1403] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.136s	user 0.015s	sys 0.009s
I20260709 12:23:20.530449 26243 ts_tablet_manager.cc:1403] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.130s	user 0.007s	sys 0.031s
I20260709 12:23:20.517917 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:20.533802 26366 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.534861 26263 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.539948 26245 ts_tablet_manager.cc:1434] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.003s
I20260709 12:23:20.542717 26244 raft_consensus.cc:359] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.543541 26244 raft_consensus.cc:385] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.543779 26244 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.544035 26245 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.544587 26244 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.528642 26429 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:23:20.547431 26053 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:20.547616 26244 ts_tablet_manager.cc:1434] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:23:20.548830 26244 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.550087 26436 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.546528 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.554767 26517 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.555191 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:20.556885 26094 leader_election.cc:343] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:20.557672 26020 leader_election.cc:343] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.555840 26432 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:23:20.559120 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:20.554786 26169 leader_election.cc:343] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.559813 26199 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:20.560971 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:20.578872 26429 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.583945 26168 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:20.581291 26383 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.588016 26436 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.588168 26383 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.588652 26383 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.589437 26243 raft_consensus.cc:359] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.590129 26243 raft_consensus.cc:385] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.590384 26243 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.591042 26243 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.581318 26429 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.595595 26429 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.607721 26243 ts_tablet_manager.cc:1434] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.077s	user 0.005s	sys 0.003s
I20260709 12:23:20.588045 26522 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.610440 26243 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:20.616708 26522 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:20.628206 26522 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.632534 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:20.615849 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:20.636926 26019 leader_election.cc:343] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.637347 26019 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.638113 26263 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:20.638671 26019 leader_election.cc:343] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.643896 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:20.645601 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:23:20.646004 26020 leader_election.cc:343] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.646279 26054 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.646456 26020 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.647643 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:20.647970 26429 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.655517 26244 tablet_bootstrap.cc:654] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.663269 26245 tablet_bootstrap.cc:654] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.663125 26383 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:20.664871 26169 leader_election.cc:343] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:20.664918 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:20.671885 26383 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.671584 26054 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:20.675441 26517 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.686168 26444 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:20.687431 26517 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.697896 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:20.700879 26202 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.701294 26522 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.701838 26522 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.704092 26522 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.706230 25951 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.695250 26517 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.710294 26383 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.722155 26383 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.707626 26444 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.724346 26444 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.725520 26383 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.719241 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.726271 26051 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:20.726552 26444 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.721081 26168 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:20.710193 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:20.732610 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.734836 26523 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:23:20.754216 26094 leader_election.cc:343] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.741613 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:20.740085 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:20.761479 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:20.740723 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:20.759687 26523 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:20.763561 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:20.763952 26523 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.742153 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:20.764837 26432 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.766871 26432 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.767266 26432 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.771968 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:20.774529 26094 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.773882 26244 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.776010 26244 ts_tablet_manager.cc:1403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.227s	user 0.019s	sys 0.024s
I20260709 12:23:20.776201 26383 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.782572 26243 tablet_bootstrap.cc:654] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.785668 26202 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:20.786216 26245 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.786072 26432 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.786685 26245 ts_tablet_manager.cc:1403] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.243s	user 0.022s	sys 0.023s
I20260709 12:23:20.787418 26201 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:20.788354 26125 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:20.789496 26020 leader_election.cc:343] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.791333 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.796223 26244 raft_consensus.cc:359] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.796943 26244 raft_consensus.cc:385] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.797204 26244 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
W20260709 12:23:20.797845 26169 leader_election.cc:343] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.799220 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:20.799819 26054 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.801669 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:20.794706 26432 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:20.802963 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.805061 26432 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:20.803085 26169 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.803359 26244 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.807850 26383 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.811409 26383 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:20.812382 26383 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.819056 26244 ts_tablet_manager.cc:1434] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:23:20.824266 26244 tablet_bootstrap.cc:492] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.827944 26245 raft_consensus.cc:359] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.831629 26245 raft_consensus.cc:385] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.832531 26245 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.833464 26245 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.831626 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:20.842901 26169 leader_election.cc:343] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.846575 26244 tablet_bootstrap.cc:654] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.858274 26123 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.838920 26245 ts_tablet_manager.cc:1434] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:23:20.842751 26054 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:20.862440 26444 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.863476 26245 tablet_bootstrap.cc:492] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.864363 26431 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.866262 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:20.867583 26522 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.868121 26429 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.868090 26522 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:20.871104 26522 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.871881 26245 tablet_bootstrap.cc:654] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.871829 25950 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:20.868629 26429 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.877801 26429 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:20.883369 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:20.885502 26019 leader_election.cc:343] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.892782 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:20.899645 26020 leader_election.cc:343] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.900097 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:20.901257 26263 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.907955 26244 tablet_bootstrap.cc:492] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.907841 25951 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.908610 26244 ts_tablet_manager.cc:1403] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.087s	user 0.009s	sys 0.003s
I20260709 12:23:20.912196 26244 raft_consensus.cc:359] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.913218 26244 raft_consensus.cc:385] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.913506 26244 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.914371 26244 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.917197 26244 ts_tablet_manager.cc:1434] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:23:20.919274 26244 tablet_bootstrap.cc:492] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.919775 26243 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:20.920295 26243 ts_tablet_manager.cc:1403] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.311s	user 0.021s	sys 0.009s
I20260709 12:23:20.922793 26243 raft_consensus.cc:359] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.923470 26243 raft_consensus.cc:385] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.923739 26243 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.927886 26244 tablet_bootstrap.cc:654] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.933230 26245 tablet_bootstrap.cc:492] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.933770 26245 ts_tablet_manager.cc:1403] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.071s	user 0.022s	sys 0.021s
I20260709 12:23:20.936599 26245 raft_consensus.cc:359] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.937352 26245 raft_consensus.cc:385] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.937718 26245 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.937451 26243 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.938477 26245 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.940529 26243 ts_tablet_manager.cc:1434] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.005s
I20260709 12:23:20.941103 26245 ts_tablet_manager.cc:1434] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:20.941807 26243 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:20.941942 26244 tablet_bootstrap.cc:492] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.942314 26245 tablet_bootstrap.cc:492] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.942417 26244 ts_tablet_manager.cc:1403] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:23:20.945222 26244 raft_consensus.cc:359] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.945899 26244 raft_consensus.cc:385] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.946162 26244 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.946924 26244 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.948587 26245 tablet_bootstrap.cc:654] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.948827 26244 ts_tablet_manager.cc:1434] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:23:20.949653 26244 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.955466 26244 tablet_bootstrap.cc:654] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.962152 26243 tablet_bootstrap.cc:654] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.966229 26245 tablet_bootstrap.cc:492] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:20.966758 26245 ts_tablet_manager.cc:1403] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.005s
I20260709 12:23:20.967492 26244 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:20.967923 26244 ts_tablet_manager.cc:1403] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.018s	user 0.003s	sys 0.010s
I20260709 12:23:20.969343 26245 raft_consensus.cc:359] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.970733 26245 raft_consensus.cc:385] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.971068 26245 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.970821 26244 raft_consensus.cc:359] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.971531 26244 raft_consensus.cc:385] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.971804 26244 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.971953 26245 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.972527 26244 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.974619 26244 ts_tablet_manager.cc:1434] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:20.974673 26245 ts_tablet_manager.cc:1434] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:20.975657 26244 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:20.975713 26245 tablet_bootstrap.cc:492] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:20.976833 25952 catalog_manager.cc:5697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.982352 26244 tablet_bootstrap.cc:654] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.986603 26243 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:20.987056 26243 ts_tablet_manager.cc:1403] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.004s
I20260709 12:23:20.989727 26243 raft_consensus.cc:359] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.990450 26243 raft_consensus.cc:385] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.990720 26243 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.991910 26245 tablet_bootstrap.cc:654] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.991611 26243 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:20.998809 26243 ts_tablet_manager.cc:1434] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:23:20.999933 26243 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:21.006705 26243 tablet_bootstrap.cc:654] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.011725 26244 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:21.012193 26244 ts_tablet_manager.cc:1403] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.015s
I20260709 12:23:21.014817 26244 raft_consensus.cc:359] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.015482 26244 raft_consensus.cc:385] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.015761 26244 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.016566 26244 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.018563 26244 ts_tablet_manager.cc:1434] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 12:23:21.020447 26244 tablet_bootstrap.cc:492] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:21.026165 26244 tablet_bootstrap.cc:654] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.029812 26243 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:21.034075 26243 ts_tablet_manager.cc:1403] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.004s
I20260709 12:23:21.034992 26245 tablet_bootstrap.cc:492] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:21.035434 26245 ts_tablet_manager.cc:1403] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.060s	user 0.008s	sys 0.003s
I20260709 12:23:21.039139 26245 raft_consensus.cc:359] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.049779 26245 raft_consensus.cc:385] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.050077 26245 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.048930 26243 raft_consensus.cc:359] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.050837 26243 raft_consensus.cc:385] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.054459 26243 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.056597 26245 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.058022 26243 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.058856 26245 ts_tablet_manager.cc:1434] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 12:23:21.059777 26245 tablet_bootstrap.cc:492] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:21.061725 26243 ts_tablet_manager.cc:1434] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.003s
I20260709 12:23:21.062652 26243 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:21.073170 26244 tablet_bootstrap.cc:492] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:21.078085 26244 ts_tablet_manager.cc:1403] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.019s
I20260709 12:23:21.080113 26245 tablet_bootstrap.cc:654] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.080817 26244 raft_consensus.cc:359] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.081561 26244 raft_consensus.cc:385] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.081844 26244 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.082074 26383 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.082635 26244 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.088138 26244 ts_tablet_manager.cc:1434] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:23:21.089219 26244 tablet_bootstrap.cc:492] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:21.082568 26383 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.091578 26383 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.093880 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:21.095726 26244 tablet_bootstrap.cc:654] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.096247 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.098189 26263 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.098692 26263 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.099125 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.100826 26263 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.100993 26383 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.102244 26369 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.102907 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.104571 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.105180 26202 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:21.106735 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:21.107187 26245 tablet_bootstrap.cc:492] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:21.107795 26245 ts_tablet_manager.cc:1403] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.014s
I20260709 12:23:21.108073 26429 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.108353 26243 tablet_bootstrap.cc:654] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.108489 26429 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.108856 26429 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.111263 26245 raft_consensus.cc:359] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.112169 26245 raft_consensus.cc:385] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.112530 26245 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
W20260709 12:23:21.112622 26020 leader_election.cc:343] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.102702 26369 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.113469 26245 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.115540 26369 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.117336 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.118418 26244 tablet_bootstrap.cc:492] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:21.118885 26244 ts_tablet_manager.cc:1403] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.007s
I20260709 12:23:21.119467 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.119390 26429 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.121500 26245 ts_tablet_manager.cc:1434] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 12:23:21.121524 26244 raft_consensus.cc:359] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.122143 26429 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.123196 26245 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:21.122489 26244 raft_consensus.cc:385] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.124053 26244 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
W20260709 12:23:21.125102 26019 leader_election.cc:343] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.126346 26243 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:21.126535 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:23:21.126741 26020 leader_election.cc:343] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.127096 26202 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.127223 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.128930 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:21.130162 26020 leader_election.cc:343] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.131004 26245 tablet_bootstrap.cc:654] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.126830 26243 ts_tablet_manager.cc:1403] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.015s
I20260709 12:23:21.133345 26369 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.134212 26522 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.134598 26202 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:21.124742 26244 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.134681 26522 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.137212 26243 raft_consensus.cc:359] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.138052 26243 raft_consensus.cc:385] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.138408 26243 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.139225 26243 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.142138 26243 ts_tablet_manager.cc:1434] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:21.142791 26244 ts_tablet_manager.cc:1434] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:23:21.143508 26243 tablet_bootstrap.cc:492] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:21.144027 26244 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:21.150259 26522 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:21.150556 26243 tablet_bootstrap.cc:654] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.142824 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:21.151827 26244 tablet_bootstrap.cc:654] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.152397 26429 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.152969 26429 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:21.153980 26429 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:21.168780 26169 consensus_peers.cc:597] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:21.170847 26383 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.171494 26383 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.172575 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.174026 26383 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.176208 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.177044 26202 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:21.179004 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:21.179710 26054 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:21.182220 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:21.187541 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:21.188519 26054 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:21.189121 26383 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.189674 26383 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.190147 26383 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:21.192654 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49918.6us
I20260709 12:23:21.194125 26263 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.194706 26263 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.197423 26263 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.200536 26431 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.201413 26431 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.202322 26432 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.206003 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.204205 26431 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.212095 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:21.214178 26094 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.215618 26431 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.218909 26431 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.219559 26431 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:21.223978 26169 leader_election.cc:343] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.226624 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:21.227571 26052 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:21.227643 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.229242 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:21.230202 26517 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.230688 26517 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.231060 26517 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.236204 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:21.238943 26020 leader_election.cc:343] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.242257 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:21.244400 26094 leader_election.cc:343] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.245378 26431 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.259162 26369 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.260334 26369 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.257021 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:21.279814 26263 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.280550 26263 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.290792 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.291721 26202 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:21.297745 26431 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:21.295030 26369 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.323122 26129 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.279366 26436 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.324510 26436 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.324877 26436 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.326349 26263 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.329869 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.331545 26383 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:21.332788 26019 leader_election.cc:343] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.333204 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.334976 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:23:21.335377 26020 leader_election.cc:343] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.335644 26054 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.340764 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:21.336447 26444 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.338075 26245 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:21.345845 26245 ts_tablet_manager.cc:1403] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.223s	user 0.030s	sys 0.058s
I20260709 12:23:21.339387 26517 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.346046 26303 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.346446 26303 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.346781 26303 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.300940 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.386096 26243 tablet_bootstrap.cc:492] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:21.386837 26243 ts_tablet_manager.cc:1403] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.244s	user 0.035s	sys 0.072s
I20260709 12:23:21.415418 26244 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:21.416172 26244 ts_tablet_manager.cc:1403] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.272s	user 0.057s	sys 0.127s
W20260709 12:23:21.409093 26559 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.143s	user 0.007s	sys 0.000s
I20260709 12:23:21.387710 26383 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.458149 26263 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.458762 26263 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.460865 26052 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.459725 26245 raft_consensus.cc:359] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.479688 26245 raft_consensus.cc:385] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.480077 26245 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
W20260709 12:23:21.486549 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 145833us
I20260709 12:23:21.480842 26245 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:21.484004 26020 leader_election.cc:343] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.490841 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.484638 26263 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.512745 26243 raft_consensus.cc:359] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.513759 26243 raft_consensus.cc:385] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.514106 26243 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.515012 26243 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.480194 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:21.520224 26054 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:21.521886 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:21.522871 26051 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.481464 26517 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.523521 26244 raft_consensus.cc:359] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.527525 26244 raft_consensus.cc:385] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.527839 26244 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.528048 26243 ts_tablet_manager.cc:1434] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.141s	user 0.008s	sys 0.000s
I20260709 12:23:21.528542 26244 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.530392 26245 ts_tablet_manager.cc:1434] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.184s	user 0.008s	sys 0.020s
I20260709 12:23:21.532374 26245 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:21.523536 26563 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.529515 26383 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:23:21.535486 26169 consensus_peers.cc:597] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:21.557824 26303 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.559902 26303 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.555929 26436 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.543157 26563 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.562400 26563 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.563136 26383 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:21.564064 26383 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.565023 26436 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:21.573585 25950 catalog_manager.cc:5697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.580986 26051 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:21.582705 26123 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.584444 26244 ts_tablet_manager.cc:1434] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.168s	user 0.007s	sys 0.000s
I20260709 12:23:21.585404 26244 tablet_bootstrap.cc:492] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:21.615942 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.617368 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.613341 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:21.622799 26432 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.630265 26245 tablet_bootstrap.cc:654] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.619638 26369 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.623540 26124 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:21.637379 26169 leader_election.cc:343] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.646108 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:21.646893 26195 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.647877 26244 tablet_bootstrap.cc:654] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.650897 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:21.635251 26369 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.652200 26200 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.633469 26432 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.655750 26369 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:21.666706 26432 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:21.683223 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:21.701148 26569 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.702414 26569 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:21.693800 26431 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:23:21.709883 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:21.710578 26203 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:21.719987 26019 leader_election.cc:343] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.721112 26019 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.723795 26263 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.725314 26019 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.723935 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.720463 26200 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:21.733328 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:21.740450 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:21.740891 26169 leader_election.cc:343] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.740888 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.743513 26572 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.746629 26282 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:21.747913 26020 leader_election.cc:343] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.738523 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:21.746657 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:21.752571 26050 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.756857 26244 tablet_bootstrap.cc:492] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:21.757328 26244 ts_tablet_manager.cc:1403] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.172s	user 0.033s	sys 0.022s
W20260709 12:23:21.755637 26019 leader_election.cc:343] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:21.758838 26020 leader_election.cc:343] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:21.759495 26020 leader_election.cc:343] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.759873 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.761863 26245 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:21.762346 26245 ts_tablet_manager.cc:1403] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.230s	user 0.011s	sys 0.004s
I20260709 12:23:21.709889 26569 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.775600 26562 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.796416 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:21.816946 26203 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:21.823931 26050 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:21.838856 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:21.839864 26048 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:21.845239 26245 raft_consensus.cc:359] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.845760 26245 raft_consensus.cc:385] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.846029 26245 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.843524 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:21.846685 26245 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.873962 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:21.881420 26245 ts_tablet_manager.cc:1434] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.119s	user 0.004s	sys 0.007s
W20260709 12:23:21.888893 26019 leader_election.cc:343] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.889680 26019 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:21.893390 26568 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:23:21.898305 26243 tablet_bootstrap.cc:492] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:21.895826 26444 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.894685 26517 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:21.903949 26244 raft_consensus.cc:359] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.904780 26244 raft_consensus.cc:385] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.905027 26244 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.905678 26244 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.907938 26244 ts_tablet_manager.cc:1434] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.150s	user 0.006s	sys 0.000s
I20260709 12:23:21.889176 26563 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:23:21.910461 26580 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.913928 26444 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:21.918207 26444 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.912209 26517 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.939054 26243 tablet_bootstrap.cc:654] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:21.942905 26020 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:21.941871 26523 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.945422 26523 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:21.961380 26523 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:22.001313 26586 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:23:22.001525 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:22.003690 26169 leader_election.cc:343] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:22.008261 26581 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.126s	user 0.007s	sys 0.003s
W20260709 12:23:22.013163 26583 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:22.016786 26243 tablet_bootstrap.cc:492] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.017252 26243 ts_tablet_manager.cc:1403] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.119s	user 0.018s	sys 0.023s
I20260709 12:23:22.020170 26243 raft_consensus.cc:359] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.020980 26243 raft_consensus.cc:385] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.021313 26243 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.021615 26244 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.022056 26243 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.023118 26245 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:22.033026 26243 ts_tablet_manager.cc:1434] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:23:22.034015 26243 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:22.038290 26568 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:22.039140 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:22.040171 26282 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.040565 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:23:22.041198 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:22.045535 26282 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:22.048053 26564 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.048679 26564 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.049019 26564 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.047317 26282 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.045284 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:22.064654 26245 tablet_bootstrap.cc:654] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.065251 26530 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:22.072623 26168 leader_election.cc:343] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.073086 26168 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:22.062938 26383 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.076442 26383 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.075666 26566 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.097990 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:22.098697 26053 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:22.100793 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:22.101572 26530 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.101940 26530 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.102283 26530 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.114310 26564 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.116339 25952 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.118093 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:22.119385 26564 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:22.118664 25952 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.124863 26517 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.136960 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:22.147265 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.149015 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:22.149778 26050 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.151185 26383 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.190475 26243 tablet_bootstrap.cc:654] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.203495 26244 tablet_bootstrap.cc:654] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.210460 26431 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:22.213413 25950 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.217362 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:22.218432 26569 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:22.232323 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.233028 25951 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.245101 26195 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:22.240895 26530 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.266050 26050 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:22.268316 26562 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.270098 26128 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:23:22.269869 26562 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.288947 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:22.291025 26562 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:22.291189 26020 leader_election.cc:343] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.299299 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:22.300174 26196 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:22.301752 26530 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.303092 26245 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:22.303637 26245 ts_tablet_manager.cc:1403] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.281s	user 0.013s	sys 0.013s
I20260709 12:23:22.306679 26245 raft_consensus.cc:359] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.307397 26245 raft_consensus.cc:385] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.307679 26245 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.308318 26245 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.310618 26245 ts_tablet_manager.cc:1434] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:23:22.303428 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:22.311388 26050 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:22.312757 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:22.313625 26244 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:22.313912 26530 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.314366 26244 ts_tablet_manager.cc:1403] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.293s	user 0.022s	sys 0.000s
I20260709 12:23:22.315519 26243 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.316004 26243 ts_tablet_manager.cc:1403] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.282s	user 0.014s	sys 0.006s
I20260709 12:23:22.317072 26245 tablet_bootstrap.cc:492] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:22.317888 26530 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.319136 26243 raft_consensus.cc:359] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.319839 26243 raft_consensus.cc:385] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.320114 26243 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.321878 26244 raft_consensus.cc:359] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.322633 26244 raft_consensus.cc:385] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.322911 26244 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.322708 26243 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.323536 26244 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.325081 26243 ts_tablet_manager.cc:1434] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:22.325953 26243 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:22.331815 26243 tablet_bootstrap.cc:654] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.332134 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:22.332965 26436 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.333451 26436 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:22.335213 26245 tablet_bootstrap.cc:654] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.334431 26436 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.339486 26244 ts_tablet_manager.cc:1434] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:23:22.342522 26244 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.354308 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:22.369594 26019 leader_election.cc:343] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.370060 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:22.371634 26580 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.387048 26244 tablet_bootstrap.cc:654] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:22.411645 26020 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:22.418171 26431 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:22.425747 25952 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.434051 26245 tablet_bootstrap.cc:492] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:22.434331 26562 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.436451 26243 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.437000 26243 ts_tablet_manager.cc:1403] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.111s	user 0.035s	sys 0.051s
W20260709 12:23:22.439258 26169 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:22.443128 26245 ts_tablet_manager.cc:1403] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.126s	user 0.020s	sys 0.000s
I20260709 12:23:22.446071 26245 raft_consensus.cc:359] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.446734 26245 raft_consensus.cc:385] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.446996 26245 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.447589 26245 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.452642 26243 raft_consensus.cc:359] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.453357 26243 raft_consensus.cc:385] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.453609 26243 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.454224 26243 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.455739 26243 ts_tablet_manager.cc:1434] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.000s
I20260709 12:23:22.456722 26243 tablet_bootstrap.cc:492] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:22.461823 26245 ts_tablet_manager.cc:1434] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:23:22.462827 26245 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:22.464309 26244 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:22.464962 26244 ts_tablet_manager.cc:1403] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.125s	user 0.024s	sys 0.000s
I20260709 12:23:22.476612 26243 tablet_bootstrap.cc:654] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.477985 26530 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.478533 26530 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.480687 26530 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.481873 26245 tablet_bootstrap.cc:654] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.482905 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:22.483327 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.485745 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:22.487797 26530 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.488718 26244 raft_consensus.cc:359] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.490149 26244 raft_consensus.cc:385] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.494047 26244 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.495086 26244 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.498207 26244 ts_tablet_manager.cc:1434] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:23:22.502108 26244 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.511621 26243 tablet_bootstrap.cc:492] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.512157 26243 ts_tablet_manager.cc:1403] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.056s	user 0.016s	sys 0.014s
I20260709 12:23:22.512192 26245 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:22.513763 26245 ts_tablet_manager.cc:1403] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.051s	user 0.006s	sys 0.011s
I20260709 12:23:22.515040 26243 raft_consensus.cc:359] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.515722 26243 raft_consensus.cc:385] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.516055 26243 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.516743 26243 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.518225 26245 raft_consensus.cc:359] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.520364 26245 raft_consensus.cc:385] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.520712 26245 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.520047 26244 tablet_bootstrap.cc:654] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.523692 26243 ts_tablet_manager.cc:1434] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:23:22.524688 26243 tablet_bootstrap.cc:492] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:22.524808 26245 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.527973 26573 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.528523 26573 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.529799 26245 ts_tablet_manager.cc:1434] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.006s
I20260709 12:23:22.530822 26245 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:22.530985 26573 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.533102 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.533903 26201 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:22.535573 26243 tablet_bootstrap.cc:654] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.535879 26019 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:22.537058 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:22.538549 26020 leader_election.cc:343] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.546155 26573 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.546609 26573 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.546969 26573 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.548527 26245 tablet_bootstrap.cc:654] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.559582 26244 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:22.560089 26244 ts_tablet_manager.cc:1403] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.005s
I20260709 12:23:22.562732 26244 raft_consensus.cc:359] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.563438 26244 raft_consensus.cc:385] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.563712 26244 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.564412 26244 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.567018 26244 ts_tablet_manager.cc:1434] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:23:22.568096 26244 tablet_bootstrap.cc:492] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.568534 26573 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.571916 26436 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:22.574697 26244 tablet_bootstrap.cc:654] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.578357 26243 tablet_bootstrap.cc:492] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.578897 26243 ts_tablet_manager.cc:1403] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.054s	user 0.005s	sys 0.011s
I20260709 12:23:22.582472 26243 raft_consensus.cc:359] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.583479 26243 raft_consensus.cc:385] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.583817 26243 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.587960 26243 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.588451 26573 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.589785 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:22.590193 26243 ts_tablet_manager.cc:1434] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:23:22.590400 26201 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.593031 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:22.594399 26020 leader_election.cc:343] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.595506 26243 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:22.596549 26244 tablet_bootstrap.cc:492] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:22.597124 26244 ts_tablet_manager.cc:1403] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.001s
I20260709 12:23:22.600008 26244 raft_consensus.cc:359] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.600843 26244 raft_consensus.cc:385] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.601130 26244 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.601895 26244 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.606912 26243 tablet_bootstrap.cc:654] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.604387 26244 ts_tablet_manager.cc:1434] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:22.610222 26244 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.616808 26244 tablet_bootstrap.cc:654] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.621605 26122 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.633112 26245 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:22.633625 26245 ts_tablet_manager.cc:1403] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.103s	user 0.007s	sys 0.005s
I20260709 12:23:22.640859 26245 raft_consensus.cc:359] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.641590 26245 raft_consensus.cc:385] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.641865 26245 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.642462 26245 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.644007 26245 ts_tablet_manager.cc:1434] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:23:22.649989 26201 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:22.651441 26019 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:22.652954 26244 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:22.653430 26244 ts_tablet_manager.cc:1403] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.003s
I20260709 12:23:22.654691 26573 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.655642 26573 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:22.656508 26573 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.657698 26244 raft_consensus.cc:359] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.658373 26244 raft_consensus.cc:385] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.658710 26244 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.659340 26244 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.661545 26244 ts_tablet_manager.cc:1434] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:23:22.662472 26244 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.665633 26125 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.669507 26243 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.669999 26243 ts_tablet_manager.cc:1403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.075s	user 0.017s	sys 0.000s
I20260709 12:23:22.670785 26128 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.655051 26245 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:22.682145 26243 raft_consensus.cc:359] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.691427 26243 raft_consensus.cc:385] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.691835 26243 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.698815 26243 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.703147 26243 ts_tablet_manager.cc:1434] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.002s
I20260709 12:23:22.704078 26243 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:22.707863 26369 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.708606 26244 tablet_bootstrap.cc:654] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.708587 26369 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.737576 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:22.740303 26020 leader_election.cc:343] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.755127 26562 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.755874 26244 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:22.755684 26562 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.756371 26244 ts_tablet_manager.cc:1403] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.094s	user 0.013s	sys 0.000s
I20260709 12:23:22.757786 26562 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.759661 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:22.760035 26369 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.759090 26244 raft_consensus.cc:359] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.760970 26244 raft_consensus.cc:385] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.761277 26244 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
W20260709 12:23:22.761000 26020 leader_election.cc:343] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.763423 26244 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.782194 26243 tablet_bootstrap.cc:654] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.790903 26244 ts_tablet_manager.cc:1434] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:23:22.799880 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.801950 26244 tablet_bootstrap.cc:492] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:22.803050 26245 tablet_bootstrap.cc:654] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.810710 26566 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.811281 26566 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.820076 26566 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:22.822775 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.825871 26573 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.826727 26573 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.837699 26245 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:22.838034 26573 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.838223 26245 ts_tablet_manager.cc:1403] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.183s	user 0.031s	sys 0.009s
I20260709 12:23:22.840183 26243 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:22.840716 26243 ts_tablet_manager.cc:1403] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.137s	user 0.023s	sys 0.013s
I20260709 12:23:22.847859 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:22.849462 26019 leader_election.cc:343] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.849888 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:22.850824 26243 raft_consensus.cc:359] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:22.851243 26020 leader_election.cc:343] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.851620 26243 raft_consensus.cc:385] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.851778 26244 tablet_bootstrap.cc:654] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.851946 26243 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
W20260709 12:23:22.852654 26019 leader_election.cc:343] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.853077 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:22.853461 26580 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.850804 26530 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.853821 26122 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.852705 26243 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.853932 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:22.854110 26530 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.859465 26243 ts_tablet_manager.cc:1434] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 12:23:22.861325 26530 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:22.856598 26580 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.850730 26567 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:23:22.862851 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:22.865252 26169 leader_election.cc:343] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.867195 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:22.869626 26245 raft_consensus.cc:359] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.870337 26245 raft_consensus.cc:385] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.870599 26245 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.871220 26245 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.873596 26245 ts_tablet_manager.cc:1434] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 12:23:22.874657 26243 tablet_bootstrap.cc:492] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
W20260709 12:23:22.919186 26019 leader_election.cc:343] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.004922 26019 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:23:23.006558 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87790.3us
I20260709 12:23:22.923194 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:22.874707 26245 tablet_bootstrap.cc:492] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:23.008543 26053 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:23.010210 26429 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.955503 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:22.956310 26567 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.924331 26369 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:22.933283 26094 leader_election.cc:343] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.977196 26580 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.978856 26562 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.012418 26562 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.989156 26626 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:23.012374 26369 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:22.997318 26244 tablet_bootstrap.cc:492] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
W20260709 12:23:23.013710 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80783.6us
I20260709 12:23:23.014287 26580 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.017234 26048 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:23.017374 26567 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.027011 26567 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.025631 26369 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.029882 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.031724 26562 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.032531 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.035041 26244 ts_tablet_manager.cc:1403] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.233s	user 0.008s	sys 0.007s
I20260709 12:23:23.035425 26580 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.042783 26597 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.043533 26597 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.044023 26597 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.044984 26168 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.046201 26523 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.046686 26523 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.047009 26523 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.055503 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.057408 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.058736 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.060690 26563 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.063057 26563 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.050679 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.062318 26436 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.064316 26567 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:23.065176 26169 leader_election.cc:343] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.065228 26436 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.065383 26567 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.069746 26563 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.070609 26048 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:23.067900 26436 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:23.073063 26020 leader_election.cc:343] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:23.074208 26020 leader_election.cc:343] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:23.076355 26020 leader_election.cc:343] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.077437 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:23.079034 26169 leader_election.cc:343] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.081342 26567 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.081897 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:23.084735 26020 leader_election.cc:343] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.090508 26244 raft_consensus.cc:359] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.092051 26523 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.093737 26244 raft_consensus.cc:385] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.096757 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.098213 26523 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.100857 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.101604 26245 tablet_bootstrap.cc:654] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.095597 26244 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
W20260709 12:23:23.105544 26169 leader_election.cc:343] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.111891 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:23.119093 26597 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.124168 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.124940 26053 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:23.124971 26169 leader_election.cc:343] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.128772 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.129623 26195 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:23.119829 26244 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.134833 26243 tablet_bootstrap.cc:654] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.139708 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.140897 26201 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:23.149091 26244 ts_tablet_manager.cc:1434] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.109s	user 0.003s	sys 0.005s
I20260709 12:23:23.150282 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.156335 26244 tablet_bootstrap.cc:492] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Bootstrap starting.
I20260709 12:23:23.159379 26053 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:23.165222 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:23.166754 26431 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.167382 26431 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:23.168810 26431 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.180333 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:23:23.183045 26094 leader_election.cc:343] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.188210 25952 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.189971 26597 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:23.201612 26020 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:23.207190 26245 tablet_bootstrap.cc:492] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:23.211808 26245 ts_tablet_manager.cc:1403] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.337s	user 0.013s	sys 0.020s
I20260709 12:23:23.212535 26243 tablet_bootstrap.cc:492] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:23.210196 26244 tablet_bootstrap.cc:654] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.213737 26243 ts_tablet_manager.cc:1403] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.339s	user 0.009s	sys 0.028s
I20260709 12:23:23.226356 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.227217 26053 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
W20260709 12:23:23.228334 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:23:23.231572 26019 leader_election.cc:343] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.232259 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.233177 26562 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.234038 26019 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.235088 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.235679 26050 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.239208 26019 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.234172 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.245177 26048 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:23.247241 26580 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.247792 26580 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.248271 26580 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.250670 26245 raft_consensus.cc:359] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.251439 26245 raft_consensus.cc:385] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.251739 26245 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.235116 26562 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.256196 26562 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.256683 26562 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.225545 26243 raft_consensus.cc:359] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.257651 26243 raft_consensus.cc:385] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.258029 26243 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.258836 26243 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.276909 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.278858 26523 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.279295 26523 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:23.288578 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49509.4us
I20260709 12:23:23.288290 26431 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.289361 26523 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.289664 26431 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:23.294848 26019 leader_election.cc:343] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.295228 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.295256 26019 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.296355 26436 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.296777 26436 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.296674 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.299268 26436 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.292392 26431 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.299901 26050 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:23.300422 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.300364 26245 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.300359 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
W20260709 12:23:23.306571 26094 leader_election.cc:343] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.307284 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.302785 26369 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:23:23.308650 26597 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.309158 26597 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.309592 26597 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.307255 26245 ts_tablet_manager.cc:1434] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.003s
I20260709 12:23:23.301065 26168 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.313902 26565 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.314285 26565 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.314613 26565 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.314718 26168 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.318476 26245 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:23.302464 26429 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.321883 26627 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.322481 26627 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.323951 26566 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:23:23.348457 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48639.5us
I20260709 12:23:23.365310 26243 ts_tablet_manager.cc:1434] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.151s	user 0.015s	sys 0.006s
I20260709 12:23:23.368618 26245 tablet_bootstrap.cc:654] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.377256 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.378178 26050 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:23.379938 26566 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:23.396159 26243 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:23.381157 26627 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.397679 26244 tablet_bootstrap.cc:492] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: No bootstrap required, opened a new log
I20260709 12:23:23.398314 26244 ts_tablet_manager.cc:1403] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent bootstrapping tablet: real 0.244s	user 0.015s	sys 0.021s
I20260709 12:23:23.396904 26562 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.416311 26562 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.396313 26580 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.418783 26580 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:23.434619 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70808.3us
I20260709 12:23:23.430936 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:23.441815 26020 leader_election.cc:343] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.420943 26569 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.434080 26523 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.420142 26436 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:23.465320 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113145us
I20260709 12:23:23.455730 26523 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.465464 26569 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.493532 26243 tablet_bootstrap.cc:654] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.493148 26244 raft_consensus.cc:359] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.494445 26244 raft_consensus.cc:385] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.495499 26244 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.456478 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:23.464196 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:23.488377 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.442385 26597 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.501717 26244 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:23.504889 26020 leader_election.cc:343] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.472131 26565 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.505581 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:23.492008 26566 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.509105 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.514631 26648 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.515323 26569 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.516250 26244 ts_tablet_manager.cc:1434] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent starting tablet: real 0.118s	user 0.005s	sys 0.005s
I20260709 12:23:23.517125 26432 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.517520 26432 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.517856 26432 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:23.520282 26169 leader_election.cc:343] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:23.521049 26169 leader_election.cc:343] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.522890 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:23.515275 26648 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.524279 26436 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:23.524374 26169 leader_election.cc:343] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.528425 26648 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.529604 26597 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.532491 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.532747 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.533121 26053 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.533809 26565 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:23.534044 26020 leader_election.cc:343] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.534168 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.534713 26050 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.544723 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.545573 26047 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.547660 26050 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:23.551403 26432 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.554596 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.555078 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:23.556548 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:23.556761 26196 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.557739 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:23.558513 26195 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:23:23.562386 26169 leader_election.cc:343] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.543779 26129 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.564128 26243 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:23.565711 26243 ts_tablet_manager.cc:1403] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.010s
I20260709 12:23:23.570665 26245 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:23.571190 26245 ts_tablet_manager.cc:1403] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.253s	user 0.012s	sys 0.009s
I20260709 12:23:23.553946 25949 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.596725 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.600476 26648 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.601193 26648 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.611397 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:23.613761 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.618553 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.621704 26053 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:23.623502 26648 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.621186 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:23.626092 26432 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.626248 26597 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.626776 26597 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:23.627568 26597 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.631392 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:23.631575 26019 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.631948 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.632974 26562 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.634320 26196 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
W20260709 12:23:23.639657 26169 leader_election.cc:343] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:23.640846 26020 leader_election.cc:343] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.583688 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:23:23.647035 26094 leader_election.cc:343] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.642609 26243 raft_consensus.cc:359] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.649603 26243 raft_consensus.cc:385] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.649892 26243 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
W20260709 12:23:23.651362 26019 leader_election.cc:343] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.651796 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.654167 26563 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.656735 26243 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.659323 26243 ts_tablet_manager.cc:1434] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.000s
I20260709 12:23:23.661933 26245 raft_consensus.cc:359] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.662700 26245 raft_consensus.cc:385] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.662952 26245 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.665983 26243 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
W20260709 12:23:23.672993 26094 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:23.676807 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.677932 26650 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.678607 26650 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:23.680809 26168 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.663544 26245 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.685601 26432 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.679541 26650 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.690656 26019 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.692392 26648 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.693168 26648 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:23.694213 26648 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.694926 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:23.696727 26053 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.703501 26047 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:23.704344 26245 ts_tablet_manager.cc:1434] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.133s	user 0.011s	sys 0.017s
I20260709 12:23:23.715710 26243 tablet_bootstrap.cc:654] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.720986 26245 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:23.722684 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.732767 26569 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.737145 26569 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:23.749715 26053 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:23.753067 26243 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:23.753584 26243 ts_tablet_manager.cc:1403] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.093s	user 0.009s	sys 0.005s
I20260709 12:23:23.756654 26243 raft_consensus.cc:359] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.757391 26243 raft_consensus.cc:385] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.757709 26243 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.758370 26243 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.759105 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.760007 26243 ts_tablet_manager.cc:1434] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:23.759958 26432 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.760687 26432 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:23.761176 26243 tablet_bootstrap.cc:492] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:23.762176 26432 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.763082 26245 tablet_bootstrap.cc:654] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.764364 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.765389 26650 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.766083 26650 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:23.767201 26650 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.760825 25950 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:23.779017 26019 leader_election.cc:343] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.788401 26243 tablet_bootstrap.cc:654] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.788936 26627 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.789498 26627 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.779554 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.791821 26627 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:23.791992 26429 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.793640 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:23.816212 25952 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.823729 26245 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:23.824460 26245 ts_tablet_manager.cc:1403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.000s
I20260709 12:23:23.827019 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.828104 26047 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:23:23.831991 26168 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:23.833578 26566 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.819092 25952 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.889135 26245 raft_consensus.cc:359] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.890918 26245 raft_consensus.cc:385] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.891227 26245 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.891868 26245 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.898881 26245 ts_tablet_manager.cc:1434] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.074s	user 0.011s	sys 0.001s
I20260709 12:23:23.900022 26245 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:23.901021 25949 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.903195 26243 tablet_bootstrap.cc:492] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:23.903669 26243 ts_tablet_manager.cc:1403] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.143s	user 0.053s	sys 0.066s
I20260709 12:23:23.906279 26243 raft_consensus.cc:359] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.906909 26243 raft_consensus.cc:385] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.907187 26243 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.907775 26243 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.909345 26243 ts_tablet_manager.cc:1434] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:23:23.916440 26243 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:23.918349 26245 tablet_bootstrap.cc:654] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.918581 26383 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.920555 26566 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.919088 26383 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.925839 26243 tablet_bootstrap.cc:654] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.927390 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.928058 26383 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.928256 26200 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:23.929255 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.935989 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.944862 26383 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.961294 26597 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.961871 26597 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.962726 25952 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.963869 26597 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:23.967497 25952 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.970469 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:23.981316 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:23.984576 26383 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.984840 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:23.985774 26383 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.987156 26245 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:23.987614 26245 ts_tablet_manager.cc:1403] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.022s
I20260709 12:23:23.991808 26243 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:23.992640 26243 ts_tablet_manager.cc:1403] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.076s	user 0.027s	sys 0.029s
I20260709 12:23:23.991842 26245 raft_consensus.cc:359] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.993103 26245 raft_consensus.cc:385] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.994143 26245 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.995086 26245 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.995823 26243 raft_consensus.cc:359] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.996518 26243 raft_consensus.cc:385] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.996785 26243 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.997668 26245 ts_tablet_manager.cc:1434] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:23:23.997524 26243 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:23.998761 26245 tablet_bootstrap.cc:492] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:23.999759 26243 ts_tablet_manager.cc:1434] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:23:24.001125 26243 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.007922 26243 tablet_bootstrap.cc:654] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:24.008622 26169 consensus_peers.cc:597] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.011374 26245 tablet_bootstrap.cc:654] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.014792 26243 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:24.015637 26243 ts_tablet_manager.cc:1403] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.005s
I20260709 12:23:24.017903 26243 raft_consensus.cc:359] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.019655 26243 raft_consensus.cc:385] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.019919 26243 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.020547 26243 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.021937 26245 tablet_bootstrap.cc:492] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:24.022467 26245 ts_tablet_manager.cc:1403] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.005s
I20260709 12:23:24.023555 26243 ts_tablet_manager.cc:1434] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:24.024638 26243 tablet_bootstrap.cc:492] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.025274 26245 raft_consensus.cc:359] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.025983 26245 raft_consensus.cc:385] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.026260 26245 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.027022 26245 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.029243 26245 ts_tablet_manager.cc:1434] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:24.030253 26245 tablet_bootstrap.cc:492] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:24.030566 26243 tablet_bootstrap.cc:654] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.035681 26243 tablet_bootstrap.cc:492] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:24.036108 26245 tablet_bootstrap.cc:654] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.036064 26597 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.036717 26597 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.038945 26597 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.041611 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.041803 26245 tablet_bootstrap.cc:492] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:24.041874 26243 ts_tablet_manager.cc:1403] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 12:23:24.042457 26047 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:24.046438 26243 raft_consensus.cc:359] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.047122 26243 raft_consensus.cc:385] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.047379 26243 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.048043 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.048489 26243 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.052067 26243 ts_tablet_manager.cc:1434] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:23:24.053124 26243 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.053370 26566 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.053829 26566 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.042379 26245 ts_tablet_manager.cc:1403] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:23:24.056671 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.057348 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:24.058548 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:24.060163 26432 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.060488 26243 tablet_bootstrap.cc:654] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.060609 26432 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.061046 26432 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.064920 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:23:24.069099 26169 leader_election.cc:343] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:24.058881 26245 raft_consensus.cc:359] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.070343 26245 raft_consensus.cc:385] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.071650 26432 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.072664 26245 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.060526 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:24.073480 26245 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.074309 26597 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.074796 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:24.075383 26047 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.075577 26245 ts_tablet_manager.cc:1434] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:23:24.076474 26432 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.077836 26245 tablet_bootstrap.cc:492] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:24.055917 26566 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.083153 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:23:24.084324 26169 leader_election.cc:343] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:24.084784 26245 tablet_bootstrap.cc:654] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.085736 26597 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.091729 26243 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:24.092250 26243 ts_tablet_manager.cc:1403] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.001s
I20260709 12:23:24.096815 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:24.097930 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:24.098699 26243 raft_consensus.cc:359] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.099717 26243 raft_consensus.cc:385] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.099989 26243 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.100427 26121 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.100999 26243 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.103269 26243 ts_tablet_manager.cc:1434] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:23:24.104277 26243 tablet_bootstrap.cc:492] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.104836 26566 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:24.105386 26566 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:24.106395 26566 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.108084 26245 tablet_bootstrap.cc:492] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:24.108564 26245 ts_tablet_manager.cc:1403] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.004s
I20260709 12:23:24.111136 26243 tablet_bootstrap.cc:654] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.114310 26245 raft_consensus.cc:359] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.115054 26245 raft_consensus.cc:385] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.115306 26245 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.116058 26245 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.118963 26649 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:24.122747 26243 tablet_bootstrap.cc:492] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:24.123230 26243 ts_tablet_manager.cc:1403] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:23:24.124500 26432 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.125847 26243 raft_consensus.cc:359] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.125041 26432 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.126533 26243 raft_consensus.cc:385] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.127138 26597 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.127135 26243 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.127871 26597 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.128212 26243 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.130649 26243 ts_tablet_manager.cc:1434] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:24.130766 26597 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.154716 26569 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:23:24.147528 26580 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:23:24.139676 26245 ts_tablet_manager.cc:1434] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.031s	user 0.002s	sys 0.004s
I20260709 12:23:24.133036 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.169626 26245 tablet_bootstrap.cc:492] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Bootstrap starting.
I20260709 12:23:24.169921 26199 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.180742 26569 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.182018 26569 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.185432 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.187566 26569 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.191804 25949 catalog_manager.cc:5697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:24.184403 26432 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.187654 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:24.188408 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.202919 26050 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:24.203691 26169 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:24.131822 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.191499 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.134413 26129 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.145942 26125 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.206467 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:24.183269 26243 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.208602 26569 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.200757 26649 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:24.183203 26245 tablet_bootstrap.cc:654] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.224184 26563 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:24.230844 26649 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:23:24.225807 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:24.239017 26383 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.239424 26597 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:24.247494 26383 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.272940 26647 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:23:24.303277 26243 tablet_bootstrap.cc:654] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.308991 26523 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:23:24.341444 26383 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.341579 26245 tablet_bootstrap.cc:492] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: No bootstrap required, opened a new log
I20260709 12:23:24.342229 26245 ts_tablet_manager.cc:1403] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent bootstrapping tablet: real 0.173s	user 0.012s	sys 0.005s
I20260709 12:23:24.342278 26523 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:24.362697 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:24.342011 26383 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.366154 26523 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.369202 26432 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.389690 26243 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:24.390276 26243 ts_tablet_manager.cc:1403] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.207s	user 0.018s	sys 0.015s
I20260709 12:23:24.392913 26563 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.401933 26383 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.401664 26563 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.403026 26243 raft_consensus.cc:359] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.406791 26243 raft_consensus.cc:385] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.407323 26243 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.407974 26243 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.416172 26245 raft_consensus.cc:359] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.416913 26245 raft_consensus.cc:385] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.417192 26245 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.419703 26243 ts_tablet_manager.cc:1434] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:23:24.419495 26245 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.409511 26647 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.418743 26650 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.437880 26243 tablet_bootstrap.cc:492] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.439682 26563 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.431015 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.429509 26647 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.441391 26523 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.463482 26383 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.465538 26383 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.436966 26650 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.476822 26383 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:24.556221 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68748.4us
I20260709 12:23:24.580118 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.572842 26432 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:23:24.580317 26689 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
I20260709 12:23:24.637655 26565 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.625718 26047 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:24.638244 26565 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.675905 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:24.676645 26125 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:24.693233 26647 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.701267 26650 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:24.717739 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 337918us
I20260709 12:23:24.728084 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.752592 26201 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:24.755059 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.755903 26053 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 0.
I20260709 12:23:24.766943 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.770395 26245 ts_tablet_manager.cc:1434] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent starting tablet: real 0.427s	user 0.022s	sys 0.008s
I20260709 12:23:24.775914 26243 tablet_bootstrap.cc:654] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.776556 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:24.779378 26122 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.791047 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:24.792321 26565 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:24.802726 26020 leader_election.cc:343] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:24.802593 26704 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.812824 26704 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.813316 26704 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.832602 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.833390 26048 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:24.842495 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.843310 26054 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:24.845762 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.846745 26196 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 0.
I20260709 12:23:24.846994 26169 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:24.848354 26432 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.848755 26432 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.849120 26432 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.860942 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:24.862841 26597 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:24.863144 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:24.866962 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:24.866252 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:24.882092 26566 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.887881 26566 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.888363 26566 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.888449 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:24.885136 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:24.885934 26698 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:24.855667 26704 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.899932 26692 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.902199 26692 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.894711 26436 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.908746 26436 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.914031 26436 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.932586 26704 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.933117 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:24.935866 26702 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.936318 26702 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.936686 26702 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.931797 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.946817 26045 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 0.
I20260709 12:23:24.945946 26692 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:24.960870 26243 tablet_bootstrap.cc:492] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:24.960723 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:24.961344 26243 ts_tablet_manager.cc:1403] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.531s	user 0.026s	sys 0.019s
I20260709 12:23:24.962118 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:24.963104 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:24.963490 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:24.963943 26201 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:24.961346 26052 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.965960 26048 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:24.973863 26243 raft_consensus.cc:359] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.974649 26243 raft_consensus.cc:385] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.974974 26243 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.975596 26243 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.971323 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:24.976791 26432 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.979563 26052 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:24.979761 26200 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:24.982848 26243 ts_tablet_manager.cc:1434] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:23:24.984687 26243 tablet_bootstrap.cc:492] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:24.991003 26432 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.991981 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:24.985838 26702 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.992383 26692 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.967049 26566 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.995368 26566 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:24.995885 26688 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:24.996443 26688 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:24.997309 26688 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:24.998631 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:25.001556 26707 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.999327 26702 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:25.003247 26169 leader_election.cc:343] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:25.002152 26707 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.007647 26707 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:25.012784 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:25.014788 26128 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:25.032670 26692 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:25.041343 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:25.042402 26051 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:25.044767 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:24.996438 26436 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.051590 26048 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:23:25.053779 26436 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:25.064291 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:25.065454 26201 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:25.066949 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:25.067803 26052 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:25.070050 26128 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:25.072085 26051 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:25.074574 26700 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:23:25.076155 26707 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.078995 26707 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:25.095647 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:25.097694 26701 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:25.098261 26701 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:23:25.085799 26169 leader_election.cc:343] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:25.100567 26169 leader_election.cc:343] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:25.099180 26701 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.113790 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:25.117393 26700 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:25.117892 26700 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:25.118719 26700 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.120466 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:25.121125 26050 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:25.126837 26243 tablet_bootstrap.cc:654] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:25.135409 26050 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:25.140393 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:25.149134 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:25.150031 26707 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:25.150596 26707 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:25.151460 26707 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.152566 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:25.159960 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:25.160919 26122 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 1: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:23:25.175872 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:25.178174 26169 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:23:25.181362 26169 leader_election.cc:343] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:25.185563 26700 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:25.191744 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:25.192534 26650 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:25.194628 26243 tablet_bootstrap.cc:492] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:25.195127 26243 ts_tablet_manager.cc:1403] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.211s	user 0.014s	sys 0.003s
I20260709 12:23:25.195663 25952 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:25.198916 26169 leader_election.cc:343] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:25.206102 26243 raft_consensus.cc:359] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.206631 26243 raft_consensus.cc:385] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:25.206931 26243 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:25.207949 26243 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.217481 26243 ts_tablet_manager.cc:1434] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:23:25.218636 26243 tablet_bootstrap.cc:492] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Bootstrap starting.
I20260709 12:23:25.225274 26243 tablet_bootstrap.cc:654] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:25.227329 26704 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.227850 26704 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.229925 26704 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:25.232650 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:25.233052 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:25.233837 26047 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 1: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:23:25.235039 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:25.236083 26704 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:25.263720 25949 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:25.289330 26704 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:25.296872 26243 tablet_bootstrap.cc:492] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: No bootstrap required, opened a new log
I20260709 12:23:25.297365 26243 ts_tablet_manager.cc:1403] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent bootstrapping tablet: real 0.079s	user 0.028s	sys 0.027s
I20260709 12:23:25.300398 26243 raft_consensus.cc:359] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.301348 26243 raft_consensus.cc:385] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:25.301630 26243 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Initialized, Role: FOLLOWER
I20260709 12:23:25.302279 26243 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.304512 26243 ts_tablet_manager.cc:1434] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:25.353708 25950 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:25.356712 25950 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 0 to 1, leader changed from <none> to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:25.413812 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:25.534112 26700 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.550668 26650 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.570537 26704 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.588546 26704 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.598238 26705 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.605938 26705 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.606455 26705 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.608587 26705 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:25.610904 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:25.611799 26196 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:25.610961 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:25.614029 26122 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:25.614312 26019 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:25.612432 26566 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.615180 26705 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:25.615518 26705 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.615840 26705 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.627308 26705 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.631772 26705 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:25.637566 26128 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:25.639529 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:25.640184 26127 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.649326 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:25.649960 26195 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.656445 26701 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:25.658274 26195 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:25.647883 26127 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:25.662446 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:25.670306 26705 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:25.670835 26705 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:25.675904 26650 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:25.671885 26705 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:25.714322 25951 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 0 to 2, leader changed from <none> to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:26.209051 26201 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:26.210413 26736 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:26.233460 26125 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:26.234870 26735 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:26.978333 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:27.014818 26745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:27.014823 26748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:27.015336 26746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.017117 24862 server_base.cc:1061] running on GCE node
I20260709 12:23:27.018867 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:27.019065 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:27.019232 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783599807019220 us; error 0 us; skew 500 ppm
I20260709 12:23:27.019840 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:27.026917 24862 webserver.cc:533] Webserver started at http://127.24.71.132:40673/ using document root <none> and password file <none>
I20260709 12:23:27.028180 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:27.028578 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:27.029034 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:27.030766 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-3-root/instance:
uuid: "24e61c63097e4d3dbb149050e6fcc997"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-3zgk"
I20260709 12:23:27.042311 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.011s	user 0.009s	sys 0.003s
I20260709 12:23:27.054445 26753 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.055644 24862 fs_manager.cc:730] Time spent opening block manager: real 0.009s	user 0.002s	sys 0.005s
I20260709 12:23:27.056031 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-3-root
uuid: "24e61c63097e4d3dbb149050e6fcc997"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-3zgk"
I20260709 12:23:27.056509 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:27.072584 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:27.073966 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:27.075562 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:27.078182 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:27.078380 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.078576 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:27.078723 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.140561 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.132:39865
I20260709 12:23:27.140651 26815 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.132:39865 every 8 connection(s)
I20260709 12:23:27.144977 24862 auto_leader_rebalancer-test.cc:439] add a tserver: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:27.165467 26816 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42395
I20260709 12:23:27.165925 26816 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:27.166622 26816 heartbeater.cc:507] Master 127.24.71.190:42395 requested a full tablet report, sending...
I20260709 12:23:27.168663 25950 ts_manager.cc:194] Registered new tserver with Master: 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:27.170034 25950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39330
I20260709 12:23:27.543396 26047 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:27.551833 26201 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.553090 26735 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:27.553324 26125 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.554838 26734 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:27.573809 26018 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d096e6da5114d91b7c4dd46412d2a6a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:27.575400 26827 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.579341 26195 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.593963 26125 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.595850 25950 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.599964 26047 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:27.611872 26125 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.613101 26827 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:27.614924 26018 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 526fbe6496c445dc95df29efcbeb8cd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:27.616508 26195 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.618180 26827 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:27.624408 26734 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.629631 26125 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.662259 25951 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.663913 26195 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.664110 26125 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:27.676702 26047 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.678174 26688 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:27.694046 26201 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.697821 26836 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:27.698829 26841 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.729701 26047 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.739460 25949 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:27.742288 26195 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:27.756592 26093 consensus_peers.cc:597] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 46aa595bcb23423fb51ff268d34a3bc7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:27.787070 26195 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:27.795287 26125 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.796553 26051 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:27.798568 26566 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:27.799393 26689 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:27.809105 26689 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.811180 26125 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:27.815734 26051 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:27.817941 26167 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55aa4b8b33f544088007e1987c55617e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:27.827847 25950 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:28.157979 26852 ts_tablet_manager.cc:933] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:28.158751 26853 ts_tablet_manager.cc:933] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:28.166585 26852 tablet_copy_client.cc:323] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:28.172891 26816 heartbeater.cc:499] Master 127.24.71.190:42395 was elected leader, sending a full tablet report...
I20260709 12:23:28.184365 26853 tablet_copy_client.cc:323] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:28.196171 26064 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 0d096e6da5114d91b7c4dd46412d2a6a from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:28.196485 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 526fbe6496c445dc95df29efcbeb8cd0 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:28.196830 26064 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 0d096e6da5114d91b7c4dd46412d2a6a from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-0d096e6da5114d91b7c4dd46412d2a6a
I20260709 12:23:28.197733 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 526fbe6496c445dc95df29efcbeb8cd0 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-526fbe6496c445dc95df29efcbeb8cd0
I20260709 12:23:28.209704 26063 tablet_copy_source_session.cc:215] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:28.210186 26064 tablet_copy_source_session.cc:215] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:28.213884 26853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526fbe6496c445dc95df29efcbeb8cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.214636 26852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d096e6da5114d91b7c4dd46412d2a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.231202 26852 tablet_copy_client.cc:806] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:28.231194 26853 tablet_copy_client.cc:806] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:28.231916 26852 tablet_copy_client.cc:670] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:28.232195 26853 tablet_copy_client.cc:670] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:28.236717 26853 tablet_copy_client.cc:538] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:28.237017 26852 tablet_copy_client.cc:538] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:28.246433 26852 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:28.246701 26853 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:28.258826 26858 ts_tablet_manager.cc:933] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:28.279767 26858 tablet_copy_client.cc:323] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:28.324146 26852 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:28.326875 26852 tablet_bootstrap.cc:492] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:28.327615 26852 ts_tablet_manager.cc:1403] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.081s	user 0.037s	sys 0.009s
I20260709 12:23:28.330727 26852 raft_consensus.cc:359] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.331995 26852 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:28.332718 26852 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.342070 26852 ts_tablet_manager.cc:1434] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.005s
I20260709 12:23:28.348501 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-0d096e6da5114d91b7c4dd46412d2a6a received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:28.349045 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-0d096e6da5114d91b7c4dd46412d2a6a on tablet 0d096e6da5114d91b7c4dd46412d2a6a with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:28.352013 26853 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:28.353088 26853 tablet_bootstrap.cc:492] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:28.353808 26853 ts_tablet_manager.cc:1403] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.107s	user 0.023s	sys 0.008s
I20260709 12:23:28.358762 26863 ts_tablet_manager.cc:933] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:28.359042 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 55aa4b8b33f544088007e1987c55617e from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:28.359642 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 55aa4b8b33f544088007e1987c55617e from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-55aa4b8b33f544088007e1987c55617e
I20260709 12:23:28.378988 26213 tablet_copy_source_session.cc:215] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:28.382299 26858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aa4b8b33f544088007e1987c55617e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.381852 26853 raft_consensus.cc:359] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.382798 26853 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:28.383467 26853 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.385913 26853 ts_tablet_manager.cc:1434] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:23:28.391286 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-526fbe6496c445dc95df29efcbeb8cd0 received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:28.391848 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-526fbe6496c445dc95df29efcbeb8cd0 on tablet 526fbe6496c445dc95df29efcbeb8cd0 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:28.418107 26863 tablet_copy_client.cc:323] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:28.427884 26858 tablet_copy_client.cc:806] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:28.428603 26858 tablet_copy_client.cc:670] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:28.436546 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 46aa595bcb23423fb51ff268d34a3bc7 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:28.437162 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 46aa595bcb23423fb51ff268d34a3bc7 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-46aa595bcb23423fb51ff268d34a3bc7
I20260709 12:23:28.438127 26858 tablet_copy_client.cc:538] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:28.442927 26139 tablet_copy_source_session.cc:215] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:28.446790 26863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46aa595bcb23423fb51ff268d34a3bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.452064 26858 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:28.460616 26863 tablet_copy_client.cc:806] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:28.461205 26863 tablet_copy_client.cc:670] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:28.470965 26863 tablet_copy_client.cc:538] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:28.481132 26858 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:28.482115 26858 tablet_bootstrap.cc:492] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:28.482813 26858 ts_tablet_manager.cc:1403] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.011s
I20260709 12:23:28.486225 26858 raft_consensus.cc:359] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.487102 26858 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:28.487418 26863 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:28.487768 26858 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.490212 26858 ts_tablet_manager.cc:1434] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:28.493290 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-55aa4b8b33f544088007e1987c55617e received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:28.493788 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-55aa4b8b33f544088007e1987c55617e on tablet 55aa4b8b33f544088007e1987c55617e with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:28.513965 26863 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:28.514699 26863 tablet_bootstrap.cc:492] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:28.515285 26863 ts_tablet_manager.cc:1403] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.010s
I20260709 12:23:28.517632 26863 raft_consensus.cc:359] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.518261 26863 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:28.518750 26863 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:28.520949 26863 ts_tablet_manager.cc:1434] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.001s
I20260709 12:23:28.524081 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-46aa595bcb23423fb51ff268d34a3bc7 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:28.524596 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-46aa595bcb23423fb51ff268d34a3bc7 on tablet 46aa595bcb23423fb51ff268d34a3bc7 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:28.583770 26790 raft_consensus.cc:1217] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:28.633966 26790 raft_consensus.cc:1217] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:28.718402 26790 raft_consensus.cc:1217] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:28.899693 26790 raft_consensus.cc:1217] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:29.169050 26735 raft_consensus.cc:1064] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:29.171217 26735 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.173635 26869 raft_consensus.cc:1064] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:29.176745 26869 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.186918 26790 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.187974 26127 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.190161 26735 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.190578 26195 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.191704 26734 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.197865 26870 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.198745 26195 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.200205 26790 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.205837 26734 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.209188 26125 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.211573 26870 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.219784 26869 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.293316 26869 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.296617 26849 raft_consensus.cc:1064] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:29.312357 26827 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.311995 26121 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.318886 26789 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.330273 26791 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.340031 26849 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.346251 26125 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.362025 25950 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:29.364349 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64718us
I20260709 12:23:29.378602 26789 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.380023 26127 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.393347 26883 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.398857 26047 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.408190 26884 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.413091 26882 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.417975 26195 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.418350 26201 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.458178 25951 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.465790 26872 raft_consensus.cc:1064] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:29.470423 26884 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.472209 26127 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.473094 26872 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.492268 26791 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.493619 26051 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.497594 26898 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.501627 26201 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.503796 26842 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.513545 26047 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.518359 26872 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.522408 26047 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.532198 26789 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.536690 26870 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:29.545933 26789 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.547916 26050 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.554719 26125 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.543656 25952 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.569180 26734 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:29.572530 26734 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.584658 26790 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.597092 26788 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.602571 26200 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.604305 26827 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:29.622359 26889 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:23:29.648095 26195 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.655539 26788 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.657267 26883 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:29.662050 26869 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.683974 25952 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.690389 26125 raft_consensus.cc:2955] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.691116 26790 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.698477 26051 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.702944 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 526fbe6496c445dc95df29efcbeb8cd0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:29.703977 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0d096e6da5114d91b7c4dd46412d2a6a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:29.707720 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 526fbe6496c445dc95df29efcbeb8cd0 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:29.710866 26872 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:29.725077 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61002.9us
I20260709 12:23:29.727929 25952 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:29.729082 26201 raft_consensus.cc:2955] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.737461 26051 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.744822 25951 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.757225 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d096e6da5114d91b7c4dd46412d2a6a with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:29.770026 26903 tablet_replica.cc:333] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:29.771554 26903 raft_consensus.cc:2243] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:29.772493 26903 raft_consensus.cc:2272] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:29.778533 26903 ts_tablet_manager.cc:1916] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:29.779659 26883 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:29.785581 26195 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.795640 26882 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.804427 26903 ts_tablet_manager.cc:1929] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:29.805380 26903 log.cc:1199] T 0d096e6da5114d91b7c4dd46412d2a6a P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/0d096e6da5114d91b7c4dd46412d2a6a
I20260709 12:23:29.808318 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 0d096e6da5114d91b7c4dd46412d2a6a (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:29.816522 26790 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.819576 26788 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.828090 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 55aa4b8b33f544088007e1987c55617e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:29.859663 26902 tablet_replica.cc:333] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:29.860786 26902 raft_consensus.cc:2243] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:29.861541 26902 raft_consensus.cc:2272] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:29.862284 25951 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.868885 26122 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:29.877887 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 55aa4b8b33f544088007e1987c55617e with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:29.878788 26903 tablet_replica.cc:333] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:29.882822 26903 raft_consensus.cc:2243] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:29.883731 26903 raft_consensus.cc:2272] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:29.887408 26903 ts_tablet_manager.cc:1916] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:29.886518 26048 raft_consensus.cc:2955] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.890535 26902 ts_tablet_manager.cc:1916] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:29.912652 26791 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.915412 26195 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:29.919922 26842 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:29.924225 26902 ts_tablet_manager.cc:1929] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:29.924729 26902 log.cc:1199] T 526fbe6496c445dc95df29efcbeb8cd0 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/526fbe6496c445dc95df29efcbeb8cd0
I20260709 12:23:29.926019 26897 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:29.926402 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 526fbe6496c445dc95df29efcbeb8cd0 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:29.934882 26903 ts_tablet_manager.cc:1929] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:29.935494 26903 log.cc:1199] T 55aa4b8b33f544088007e1987c55617e P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/55aa4b8b33f544088007e1987c55617e
I20260709 12:23:29.935882 26898 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.943934 26791 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.974136 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 55aa4b8b33f544088007e1987c55617e (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:29.974078 25951 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.975567 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 46aa595bcb23423fb51ff268d34a3bc7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:29.977502 26200 raft_consensus.cc:2955] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:29.989075 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet 46aa595bcb23423fb51ff268d34a3bc7 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:30.013561 26905 tablet_replica.cc:333] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:30.015479 26905 raft_consensus.cc:2243] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:30.016809 26905 raft_consensus.cc:2272] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:30.022070 26905 ts_tablet_manager.cc:1916] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:30.037053 26905 ts_tablet_manager.cc:1929] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:30.037427 26905 log.cc:1199] T 46aa595bcb23423fb51ff268d34a3bc7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/46aa595bcb23423fb51ff268d34a3bc7
I20260709 12:23:30.039252 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet 46aa595bcb23423fb51ff268d34a3bc7 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:31.125020 26051 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:31.138620 26018 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fee7aee965145c6b43db63c2b3ecca8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.138638 26195 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.139796 26127 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.140993 26907 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.141901 26869 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.151412 26907 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.155085 26195 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.158016 26121 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.168416 25951 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:31.176378 26201 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.189692 26051 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.191304 26849 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:31.192411 26167 consensus_peers.cc:597] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a34019ceec5a4e4e877bed7351d1d91d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.189692 26121 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.199656 26908 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.202091 26884 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.205379 26051 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.219341 26121 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.234987 26201 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.236367 25949 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:31.248983 26167 consensus_peers.cc:597] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bede0bf927e45a7b18a1bf2ad0bb594. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.250418 26127 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.253440 26908 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.254354 26051 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.256487 26884 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.264001 26908 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.266016 26121 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.276858 26051 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.287354 26048 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.291545 25950 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:31.312911 26201 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:31.313557 26018 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 49f4a1a801de461182f8315a884b9889. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.315023 26907 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.316910 26121 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.320151 26907 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.326771 26869 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.330826 26201 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.352306 26121 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:31.360293 25950 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:31.707929 26928 ts_tablet_manager.cc:933] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:31.710162 26928 tablet_copy_client.cc:323] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:31.711946 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 49f4a1a801de461182f8315a884b9889 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:31.712679 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 49f4a1a801de461182f8315a884b9889 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-49f4a1a801de461182f8315a884b9889
I20260709 12:23:31.719558 26063 tablet_copy_source_session.cc:215] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.723290 26928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49f4a1a801de461182f8315a884b9889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.738004 26928 tablet_copy_client.cc:806] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.739475 26928 tablet_copy_client.cc:670] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.755038 26928 tablet_copy_client.cc:538] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.758592 26931 ts_tablet_manager.cc:933] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:31.760878 26932 ts_tablet_manager.cc:933] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:31.769449 26932 tablet_copy_client.cc:323] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:31.771420 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet a34019ceec5a4e4e877bed7351d1d91d from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:31.771960 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet a34019ceec5a4e4e877bed7351d1d91d from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-a34019ceec5a4e4e877bed7351d1d91d
I20260709 12:23:31.772660 26931 tablet_copy_client.cc:323] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:31.772825 26928 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:31.777454 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 2fee7aee965145c6b43db63c2b3ecca8 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:31.778036 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 2fee7aee965145c6b43db63c2b3ecca8 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-2fee7aee965145c6b43db63c2b3ecca8
I20260709 12:23:31.782766 26213 tablet_copy_source_session.cc:215] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.788591 26932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34019ceec5a4e4e877bed7351d1d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.789338 26063 tablet_copy_source_session.cc:215] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.792934 26931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fee7aee965145c6b43db63c2b3ecca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.829052 26931 tablet_copy_client.cc:806] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.829794 26931 tablet_copy_client.cc:670] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.830343 26932 tablet_copy_client.cc:806] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.830930 26932 tablet_copy_client.cc:670] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.835301 26931 tablet_copy_client.cc:538] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.840613 26928 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.841020 26932 tablet_copy_client.cc:538] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.841595 26928 tablet_bootstrap.cc:492] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:31.847173 26928 ts_tablet_manager.cc:1403] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.015s
I20260709 12:23:31.850186 26928 raft_consensus.cc:359] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.851790 26931 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:31.851820 26928 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:31.852254 26935 ts_tablet_manager.cc:933] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:31.852835 26928 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.866086 26928 ts_tablet_manager.cc:1434] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 12:23:31.867762 26932 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:31.881866 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-49f4a1a801de461182f8315a884b9889 received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:31.882344 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-49f4a1a801de461182f8315a884b9889 on tablet 49f4a1a801de461182f8315a884b9889 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:31.885775 26935 tablet_copy_client.cc:323] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:31.887707 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 8bede0bf927e45a7b18a1bf2ad0bb594 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:31.888192 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 8bede0bf927e45a7b18a1bf2ad0bb594 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-8bede0bf927e45a7b18a1bf2ad0bb594
I20260709 12:23:31.894315 26213 tablet_copy_source_session.cc:215] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.895551 26931 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.896587 26931 tablet_bootstrap.cc:492] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:31.897270 26931 ts_tablet_manager.cc:1403] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.006s
I20260709 12:23:31.901357 26935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bede0bf927e45a7b18a1bf2ad0bb594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.904920 26931 raft_consensus.cc:359] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.905826 26931 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:31.906572 26931 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.908916 26931 ts_tablet_manager.cc:1434] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:23:31.911293 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-2fee7aee965145c6b43db63c2b3ecca8 received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:31.911764 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-2fee7aee965145c6b43db63c2b3ecca8 on tablet 2fee7aee965145c6b43db63c2b3ecca8 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:31.918640 26932 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.919842 26932 tablet_bootstrap.cc:492] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:31.920572 26932 ts_tablet_manager.cc:1403] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.011s
I20260709 12:23:31.935545 26932 raft_consensus.cc:359] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.939329 26932 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:31.940217 26932 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.941001 26935 tablet_copy_client.cc:806] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.941686 26935 tablet_copy_client.cc:670] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.943591 26932 ts_tablet_manager.cc:1434] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:23:31.949579 26935 tablet_copy_client.cc:538] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.950758 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-a34019ceec5a4e4e877bed7351d1d91d received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:31.951301 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-a34019ceec5a4e4e877bed7351d1d91d on tablet a34019ceec5a4e4e877bed7351d1d91d with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:31.963768 26935 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:31.980162 26935 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.981081 26935 tablet_bootstrap.cc:492] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:31.981815 26935 ts_tablet_manager.cc:1403] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.014s
I20260709 12:23:31.984694 26935 raft_consensus.cc:359] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.985357 26935 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:31.986023 26935 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:31.988169 26935 ts_tablet_manager.cc:1434] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:31.990476 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-8bede0bf927e45a7b18a1bf2ad0bb594 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:31.990933 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-8bede0bf927e45a7b18a1bf2ad0bb594 on tablet 8bede0bf927e45a7b18a1bf2ad0bb594 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:32.167865 26788 raft_consensus.cc:1217] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.192436 26788 raft_consensus.cc:1217] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.203171 26788 raft_consensus.cc:1217] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.377374 26788 raft_consensus.cc:1217] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.664891 26734 raft_consensus.cc:1064] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:32.667315 26734 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:32.667914 26884 raft_consensus.cc:1064] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:32.670444 26884 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:32.681778 26788 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.683166 26121 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.683564 26734 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.684695 26201 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.686574 26734 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.690011 26788 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.691792 26930 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:32.698392 26127 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.701094 26869 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.702534 26942 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.733736 26869 raft_consensus.cc:1064] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:32.736788 26869 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:32.741678 26907 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.743345 26791 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:32.786511 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72833.1us
I20260709 12:23:32.788708 26051 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.794488 26884 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.797171 26790 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:32.805066 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77300.2us
I20260709 12:23:32.809293 26734 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.805994 25949 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:32.816346 26201 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.820465 26957 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:32.827521 26947 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:23:32.836817 26953 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.839922 26788 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.846398 26048 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.857120 26200 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.862808 26121 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.863430 26122 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.865279 26955 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.869143 26949 raft_consensus.cc:1064] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:32.873452 26949 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:32.926385 26956 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.931497 26200 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.941905 26790 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.943638 26884 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.954694 26125 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.957129 26127 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.975056 26791 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:32.986646 26128 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.988592 26952 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.991240 26048 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.992832 26952 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.002404 26949 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.005852 26122 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.017346 26788 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.067413 26048 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.111933 25950 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.117616 25950 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.225498 25952 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.266711 26052 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:33.268790 26048 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:33.275394 26201 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:33.280664 26195 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:33.280754 26128 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.296761 26734 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.301054 26788 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.303272 26125 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.304857 26734 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.310971 26122 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.312678 26952 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.319303 26788 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.322832 26974 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.322975 26907 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.326208 26788 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.326351 26128 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.324800 26973 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.331166 26788 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.331077 26127 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.337522 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 49f4a1a801de461182f8315a884b9889 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.340302 26908 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.340204 26128 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.342743 26908 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.346629 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fee7aee965145c6b43db63c2b3ecca8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.350014 25949 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.350975 26973 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.355134 26125 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.359994 26791 raft_consensus.cc:2955] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.369983 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 49f4a1a801de461182f8315a884b9889 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:33.379773 26952 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.384945 26974 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.388702 26974 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.388196 25949 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.398164 26790 raft_consensus.cc:2955] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.401023 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8bede0bf927e45a7b18a1bf2ad0bb594 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.400695 26127 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.409145 26976 tablet_replica.cc:333] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:33.410375 26976 raft_consensus.cc:2243] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.412417 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a34019ceec5a4e4e877bed7351d1d91d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.413836 26976 raft_consensus.cc:2272] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.420047 26788 raft_consensus.cc:2955] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.424660 26789 raft_consensus.cc:2955] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.450439 26976 ts_tablet_manager.cc:1916] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.455470 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fee7aee965145c6b43db63c2b3ecca8 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:33.473898 26976 ts_tablet_manager.cc:1929] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.474709 26976 log.cc:1199] T 49f4a1a801de461182f8315a884b9889 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/49f4a1a801de461182f8315a884b9889
I20260709 12:23:33.473846 25952 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.474786 25951 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:33.477067 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 49f4a1a801de461182f8315a884b9889 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:33.478397 26976 tablet_replica.cc:333] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:33.479858 26976 raft_consensus.cc:2243] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.482194 26976 raft_consensus.cc:2272] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.486680 26976 ts_tablet_manager.cc:1916] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.491153 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet a34019ceec5a4e4e877bed7351d1d91d with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:33.495344 26043 tablet_service.cc:1558] Processing DeleteTablet for tablet 8bede0bf927e45a7b18a1bf2ad0bb594 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:33.501948 26978 tablet_replica.cc:333] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:33.506075 26978 raft_consensus.cc:2243] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.507987 26978 raft_consensus.cc:2272] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.509986 26976 ts_tablet_manager.cc:1929] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.510464 26976 log.cc:1199] T 2fee7aee965145c6b43db63c2b3ecca8 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/2fee7aee965145c6b43db63c2b3ecca8
I20260709 12:23:33.512375 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 2fee7aee965145c6b43db63c2b3ecca8 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:33.514184 26978 ts_tablet_manager.cc:1916] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.533627 26978 ts_tablet_manager.cc:1929] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.533970 26978 log.cc:1199] T a34019ceec5a4e4e877bed7351d1d91d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/a34019ceec5a4e4e877bed7351d1d91d
I20260709 12:23:33.535461 26978 tablet_replica.cc:333] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:33.535645 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet a34019ceec5a4e4e877bed7351d1d91d (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:33.549427 26978 raft_consensus.cc:2243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.550207 26978 raft_consensus.cc:2272] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.553179 26978 ts_tablet_manager.cc:1916] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.567996 26978 ts_tablet_manager.cc:1929] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.568450 26978 log.cc:1199] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/8bede0bf927e45a7b18a1bf2ad0bb594
I20260709 12:23:33.570156 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet 8bede0bf927e45a7b18a1bf2ad0bb594 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:34.596410 26201 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:34.603904 26052 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.604672 26127 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.606211 26979 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:34.607605 26167 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86798289f90549ed8084d260f08cb204. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.608314 26980 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.614813 26980 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.616602 26052 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.619287 26127 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.633929 26201 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:34.635360 25952 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:34.649454 26167 consensus_peers.cc:597] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4db91b54f05348e6855f89c508d0be01. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.651726 26127 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.653805 26979 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.654428 26052 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.656060 26979 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:34.672709 26980 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.677870 26127 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.680016 26052 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:34.693488 26980 consensus_queue.cc:1175] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:23:34.695125 26201 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:34.705314 25952 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:34.706979 26167 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4a62528d9404c66b98e87e76ac0f21e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.712150 26121 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.715792 26051 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.713629 26979 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.721539 26952 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.730636 26979 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.732465 26127 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.738148 26051 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.749701 25951 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:34.754503 26127 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:34.771262 26093 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 824b6757fe414215b1cf6aa5ba238004. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.771389 26201 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.771683 26051 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.772796 26975 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:34.774014 26927 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.784962 26927 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.788079 26051 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.787705 26201 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:34.807984 25951 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:35.004053 26998 ts_tablet_manager.cc:933] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:35.006484 26998 tablet_copy_client.cc:323] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:35.009093 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 86798289f90549ed8084d260f08cb204 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:35.009704 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 86798289f90549ed8084d260f08cb204 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-86798289f90549ed8084d260f08cb204
I20260709 12:23:35.015800 26213 tablet_copy_source_session.cc:215] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.018823 26998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86798289f90549ed8084d260f08cb204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.032565 27000 ts_tablet_manager.cc:933] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:35.034875 26998 tablet_copy_client.cc:806] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.035620 26998 tablet_copy_client.cc:670] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.035734 27000 tablet_copy_client.cc:323] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:35.038031 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 4db91b54f05348e6855f89c508d0be01 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:35.038586 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 4db91b54f05348e6855f89c508d0be01 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-4db91b54f05348e6855f89c508d0be01
I20260709 12:23:35.041199 26998 tablet_copy_client.cc:538] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.051779 26998 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:35.053342 26213 tablet_copy_source_session.cc:215] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.056948 27000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db91b54f05348e6855f89c508d0be01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.071882 26998 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.072914 26998 tablet_bootstrap.cc:492] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:35.073652 26998 ts_tablet_manager.cc:1403] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.012s
I20260709 12:23:35.077104 26998 raft_consensus.cc:359] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.078009 26998 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:35.078723 26998 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.080440 27000 tablet_copy_client.cc:806] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.081312 27000 tablet_copy_client.cc:670] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.084785 26998 ts_tablet_manager.cc:1434] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.005s
I20260709 12:23:35.086664 26212 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-86798289f90549ed8084d260f08cb204 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:35.087138 26212 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-86798289f90549ed8084d260f08cb204 on tablet 86798289f90549ed8084d260f08cb204 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:35.092411 27000 tablet_copy_client.cc:538] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.101052 27000 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:35.123956 27000 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.124781 27000 tablet_bootstrap.cc:492] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:35.125349 27000 ts_tablet_manager.cc:1403] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:23:35.128070 27000 raft_consensus.cc:359] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.128925 27000 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:35.129541 27000 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.134296 27000 ts_tablet_manager.cc:1434] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:23:35.136947 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-4db91b54f05348e6855f89c508d0be01 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:35.137356 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-4db91b54f05348e6855f89c508d0be01 on tablet 4db91b54f05348e6855f89c508d0be01 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:35.152411 27000 ts_tablet_manager.cc:933] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:35.155055 27000 tablet_copy_client.cc:323] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:35.157709 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 824b6757fe414215b1cf6aa5ba238004 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:35.158288 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 824b6757fe414215b1cf6aa5ba238004 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-824b6757fe414215b1cf6aa5ba238004
I20260709 12:23:35.166436 26139 tablet_copy_source_session.cc:215] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.172195 27000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824b6757fe414215b1cf6aa5ba238004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.192864 27000 tablet_copy_client.cc:806] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.193583 27000 tablet_copy_client.cc:670] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.198304 27000 tablet_copy_client.cc:538] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.215801 27000 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:35.236562 27000 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.237454 27000 tablet_bootstrap.cc:492] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:35.238165 27000 ts_tablet_manager.cc:1403] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 12:23:35.240705 27000 raft_consensus.cc:359] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.241511 27000 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:35.242388 27000 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.244630 27000 ts_tablet_manager.cc:1434] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:35.246532 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-824b6757fe414215b1cf6aa5ba238004 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:35.246958 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-824b6757fe414215b1cf6aa5ba238004 on tablet 824b6757fe414215b1cf6aa5ba238004 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:35.323208 27000 ts_tablet_manager.cc:933] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:35.325142 27000 tablet_copy_client.cc:323] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:35.326625 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet b4a62528d9404c66b98e87e76ac0f21e from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:35.327073 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet b4a62528d9404c66b98e87e76ac0f21e from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-b4a62528d9404c66b98e87e76ac0f21e
I20260709 12:23:35.333007 26213 tablet_copy_source_session.cc:215] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.336056 27000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a62528d9404c66b98e87e76ac0f21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.352743 27000 tablet_copy_client.cc:806] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.353549 27000 tablet_copy_client.cc:670] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.357368 27000 tablet_copy_client.cc:538] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.364166 27000 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:35.385905 27000 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.386797 27000 tablet_bootstrap.cc:492] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:35.387444 27000 ts_tablet_manager.cc:1403] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.023s	user 0.023s	sys 0.000s
I20260709 12:23:35.389972 27000 raft_consensus.cc:359] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.390682 27000 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:35.391819 27000 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:35.396838 27000 ts_tablet_manager.cc:1434] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:23:35.398452 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-b4a62528d9404c66b98e87e76ac0f21e received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:35.398861 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-b4a62528d9404c66b98e87e76ac0f21e on tablet b4a62528d9404c66b98e87e76ac0f21e with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:35.453552 26790 raft_consensus.cc:1217] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:35.606703 26790 raft_consensus.cc:1217] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:35.622869 26790 raft_consensus.cc:1217] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:35.900451 27009 raft_consensus.cc:1064] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:35.909199 27009 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:35.917021 26790 raft_consensus.cc:1217] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:35.934898 26127 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:35.935897 26790 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:35.934944 26051 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:35.937427 27010 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.939857 27010 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.949224 26980 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.988497 27014 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:35.989270 26121 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.003429 26790 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.020350 26052 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.024881 25950 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.049242 27009 raft_consensus.cc:1064] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:36.051268 26997 raft_consensus.cc:1064] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:36.051941 27009 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.054179 26997 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.184540 26790 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.185460 26127 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.186643 26980 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.196069 26980 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.206542 26051 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:36.239938 27021 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:23:36.269234 27010 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:36.296483 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51883.6us
I20260709 12:23:36.341293 26790 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.342796 26927 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.343498 26200 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.348039 26927 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.375671 26953 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.400010 26047 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.409173 26122 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.411986 26791 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:36.448342 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91205.6us
W20260709 12:23:36.456795 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50327.9us
I20260709 12:23:36.459837 26953 raft_consensus.cc:1064] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:36.461383 27030 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.462970 26953 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.471748 26997 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.477149 26790 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.501760 26052 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.511804 26791 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.512795 26201 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.518965 26980 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.536825 26122 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.539340 26052 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.553169 26047 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.575771 25950 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.578576 27009 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.580576 27009 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.577577 25950 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.581543 26201 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.604223 26127 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.607549 26952 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.612394 26789 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.613394 27014 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.616011 27034 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:36.620297 26125 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.635495 26790 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.632637 27035 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.647490 26127 raft_consensus.cc:2955] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.687233 26122 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.693580 25952 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.700127 26789 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.702713 27030 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.715042 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet 4db91b54f05348e6855f89c508d0be01 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:36.714385 26997 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.715911 26048 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.718598 25950 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.732602 26789 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.739782 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4db91b54f05348e6855f89c508d0be01 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:36.747553 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 824b6757fe414215b1cf6aa5ba238004 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:36.727029 26790 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.753207 26048 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.757329 27030 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.760604 27040 tablet_replica.cc:333] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:36.762396 27040 raft_consensus.cc:2243] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:36.763294 27040 raft_consensus.cc:2272] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:36.771960 26048 raft_consensus.cc:2955] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.773701 27040 ts_tablet_manager.cc:1916] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:36.787834 26200 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.794179 26789 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.795074 26052 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.795220 25951 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:36.796680 26980 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.799852 26953 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.804610 26953 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.813133 26789 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.816584 26052 raft_consensus.cc:2955] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.819881 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 824b6757fe414215b1cf6aa5ba238004 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:36.834801 26200 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:36.849776 26790 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.851253 26980 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.851708 26048 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:36.857481 27041 tablet_replica.cc:333] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:36.858651 27041 raft_consensus.cc:2243] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:36.859436 27041 raft_consensus.cc:2272] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:36.860646 27040 ts_tablet_manager.cc:1929] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:36.861124 27040 log.cc:1199] T 4db91b54f05348e6855f89c508d0be01 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/4db91b54f05348e6855f89c508d0be01
I20260709 12:23:36.863056 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet 4db91b54f05348e6855f89c508d0be01 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:36.865245 27035 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.868005 27034 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.870267 26789 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.877230 27041 ts_tablet_manager.cc:1916] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:36.874722 25952 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.895512 26048 raft_consensus.cc:2955] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.907598 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 86798289f90549ed8084d260f08cb204 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:36.918884 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 86798289f90549ed8084d260f08cb204 with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:36.922209 27041 ts_tablet_manager.cc:1929] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:36.922708 27041 log.cc:1199] T 824b6757fe414215b1cf6aa5ba238004 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/824b6757fe414215b1cf6aa5ba238004
I20260709 12:23:36.924454 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 824b6757fe414215b1cf6aa5ba238004 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:36.926811 25951 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:36.948197 26113 tablet_service.cc:1558] Processing DeleteTablet for tablet b4a62528d9404c66b98e87e76ac0f21e with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:36.957388 27042 tablet_replica.cc:333] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:36.959599 27042 raft_consensus.cc:2243] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:36.960520 27042 raft_consensus.cc:2272] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:36.960253 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b4a62528d9404c66b98e87e76ac0f21e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:36.965566 27042 ts_tablet_manager.cc:1916] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:36.982692 27042 ts_tablet_manager.cc:1929] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:36.983089 27042 log.cc:1199] T 86798289f90549ed8084d260f08cb204 P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/86798289f90549ed8084d260f08cb204
I20260709 12:23:36.984889 27042 tablet_replica.cc:333] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:36.984784 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 86798289f90549ed8084d260f08cb204 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:36.985805 27042 raft_consensus.cc:2243] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:36.986523 27042 raft_consensus.cc:2272] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:36.990916 27042 ts_tablet_manager.cc:1916] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:37.008956 27042 ts_tablet_manager.cc:1929] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:37.009388 27042 log.cc:1199] T b4a62528d9404c66b98e87e76ac0f21e P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/b4a62528d9404c66b98e87e76ac0f21e
I20260709 12:23:37.011189 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet b4a62528d9404c66b98e87e76ac0f21e (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:38.037884 26122 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.044406 26200 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:38.045048 26093 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86fd5767abab4e45ad1c8bd5f6d7a9c8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.046295 26048 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.046792 27046 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.048822 27046 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.055563 27043 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.057875 26200 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.074966 26201 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.077282 26048 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.081125 25950 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:38.086185 26167 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 772287743e884fe8a6dc6c516c3ad901. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.087889 26122 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.088804 26052 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.090222 27034 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.091168 26953 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.101913 27051 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.105899 26122 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.113336 26052 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.132581 26201 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:38.141789 26167 consensus_peers.cc:597] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7ad94af29be4656bf3ce8a023d19f7d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.148887 26125 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.147801 25951 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.151000 26052 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.150957 27058 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.152796 26953 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.161383 27034 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.169057 26052 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.180018 26122 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.181947 26048 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:38.189875 26018 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 549b49961cbb430d9ece6acd81aede77. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.189790 25951 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.192785 26125 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.192509 26201 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.194451 27011 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.195420 27031 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.219871 27044 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.221818 26127 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.245039 25951 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.262578 26201 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:38.527796 27066 ts_tablet_manager.cc:933] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:38.530037 27066 tablet_copy_client.cc:323] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:38.531682 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 772287743e884fe8a6dc6c516c3ad901 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:38.532222 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 772287743e884fe8a6dc6c516c3ad901 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-772287743e884fe8a6dc6c516c3ad901
I20260709 12:23:38.556445 27068 ts_tablet_manager.cc:933] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:38.596415 26213 tablet_copy_source_session.cc:215] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:38.601191 27066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772287743e884fe8a6dc6c516c3ad901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:38.603650 27068 tablet_copy_client.cc:323] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:38.606055 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:38.606634 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-86fd5767abab4e45ad1c8bd5f6d7a9c8
I20260709 12:23:38.616286 26139 tablet_copy_source_session.cc:215] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:38.620123 27068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:38.629719 27070 ts_tablet_manager.cc:933] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:38.639891 27068 tablet_copy_client.cc:806] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:38.640260 27066 tablet_copy_client.cc:806] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:38.640749 27068 tablet_copy_client.cc:670] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:38.640938 27066 tablet_copy_client.cc:670] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:38.641506 27070 tablet_copy_client.cc:323] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:38.644097 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 549b49961cbb430d9ece6acd81aede77 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:38.644672 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 549b49961cbb430d9ece6acd81aede77 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-549b49961cbb430d9ece6acd81aede77
I20260709 12:23:38.649366 27066 tablet_copy_client.cc:538] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:38.650492 27068 tablet_copy_client.cc:538] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:38.658824 26063 tablet_copy_source_session.cc:215] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:38.663326 27070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 549b49961cbb430d9ece6acd81aede77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:38.686302 27066 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:38.696544 27070 tablet_copy_client.cc:806] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:38.697203 27070 tablet_copy_client.cc:670] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:38.692538 27068 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:38.705960 27072 ts_tablet_manager.cc:933] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:38.707408 27070 tablet_copy_client.cc:538] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:38.723623 27070 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:38.728489 27072 tablet_copy_client.cc:323] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:38.731233 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet c7ad94af29be4656bf3ce8a023d19f7d from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:38.731796 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet c7ad94af29be4656bf3ce8a023d19f7d from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-c7ad94af29be4656bf3ce8a023d19f7d
I20260709 12:23:38.742303 26213 tablet_copy_source_session.cc:215] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:38.746078 27072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ad94af29be4656bf3ce8a023d19f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:38.781643 27066 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:38.782248 27068 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:38.794231 27066 tablet_bootstrap.cc:492] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:38.794852 27068 tablet_bootstrap.cc:492] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:38.796578 27072 tablet_copy_client.cc:806] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:38.797199 27072 tablet_copy_client.cc:670] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:38.797989 27068 ts_tablet_manager.cc:1403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.113s	user 0.019s	sys 0.007s
I20260709 12:23:38.802459 27066 ts_tablet_manager.cc:1403] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.116s	user 0.032s	sys 0.021s
I20260709 12:23:38.804854 27072 tablet_copy_client.cc:538] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:38.801620 27068 raft_consensus.cc:359] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.806047 27068 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:38.806774 27068 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.808873 27066 raft_consensus.cc:359] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.810746 27066 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:38.811451 27066 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.830336 27068 ts_tablet_manager.cc:1434] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.006s
I20260709 12:23:38.832155 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-86fd5767abab4e45ad1c8bd5f6d7a9c8 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:38.832621 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-86fd5767abab4e45ad1c8bd5f6d7a9c8 on tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:38.833333 27066 ts_tablet_manager.cc:1434] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.008s
I20260709 12:23:38.835541 27070 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:38.836582 27070 tablet_bootstrap.cc:492] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:38.836793 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-772287743e884fe8a6dc6c516c3ad901 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:38.837342 27070 ts_tablet_manager.cc:1403] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.114s	user 0.028s	sys 0.016s
I20260709 12:23:38.840719 27070 raft_consensus.cc:359] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.837301 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-772287743e884fe8a6dc6c516c3ad901 on tablet 772287743e884fe8a6dc6c516c3ad901 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:38.841651 27070 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:38.842375 27070 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.845304 27070 ts_tablet_manager.cc:1434] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:23:38.851781 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-549b49961cbb430d9ece6acd81aede77 received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:38.852356 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-549b49961cbb430d9ece6acd81aede77 on tablet 549b49961cbb430d9ece6acd81aede77 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:38.862834 27072 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:38.894341 27072 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:38.895078 27072 tablet_bootstrap.cc:492] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:38.895628 27072 ts_tablet_manager.cc:1403] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.007s
I20260709 12:23:38.897876 27072 raft_consensus.cc:359] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.898603 27072 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:38.899240 27072 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:38.903162 27072 ts_tablet_manager.cc:1434] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.008s
I20260709 12:23:38.904848 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-c7ad94af29be4656bf3ce8a023d19f7d received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:38.905292 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-c7ad94af29be4656bf3ce8a023d19f7d on tablet c7ad94af29be4656bf3ce8a023d19f7d with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:39.071031 26789 raft_consensus.cc:1217] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:39.089190 26789 raft_consensus.cc:1217] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:39.144308 26789 raft_consensus.cc:1217] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:39.254637 26789 raft_consensus.cc:1217] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:39.472635 26953 raft_consensus.cc:1064] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:39.475025 26953 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.481637 26127 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.482144 26789 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.483191 26953 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:39.484679 27034 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:39.484997 26048 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.486480 26953 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:39.499629 26953 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.501907 26789 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.502869 26048 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.511230 26122 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.533048 25949 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.569677 26196 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.583942 26789 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.584098 26048 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.586194 27086 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.587030 27051 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.601014 27043 raft_consensus.cc:1064] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:39.603021 27043 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.603092 27051 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.607187 26048 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.607409 26790 raft_consensus.cc:2955] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.616052 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 772287743e884fe8a6dc6c516c3ad901 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:39.636963 25952 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.647492 26790 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.649782 27045 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:39.664152 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 772287743e884fe8a6dc6c516c3ad901 with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:39.671998 26196 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.674198 27044 raft_consensus.cc:1064] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:39.674330 27045 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:39.677182 27044 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.676899 27094 tablet_replica.cc:333] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:39.678984 26048 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.683348 26953 raft_consensus.cc:1064] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:39.685566 26953 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.699680 27094 raft_consensus.cc:2243] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.700685 27094 raft_consensus.cc:2272] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.705308 27094 ts_tablet_manager.cc:1916] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.708384 26791 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.709990 27085 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:39.718408 26125 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.721257 27086 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:39.724336 26788 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.726037 27044 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:39.730024 26125 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:39.730880 27094 ts_tablet_manager.cc:1929] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.733925 27094 log.cc:1199] T 772287743e884fe8a6dc6c516c3ad901 P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/772287743e884fe8a6dc6c516c3ad901
I20260709 12:23:39.735384 27044 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:39.745254 26048 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:39.746871 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 73604.6us
I20260709 12:23:39.753517 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 772287743e884fe8a6dc6c516c3ad901 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:39.758903 26196 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:39.763974 27091 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:23:39.791524 27092 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:39.791599 27045 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:39.795859 27085 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:39.807299 27045 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.825482 26196 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.828466 27034 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.846161 27044 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.863816 25950 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.886572 26790 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.887010 25951 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.866557 26122 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.894425 26791 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.870770 26789 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.879025 26129 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.877316 26047 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.928215 26048 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.929737 26129 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:39.931357 26051 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.934904 26789 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.938117 27045 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.932719 25952 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.952802 27043 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.956314 26196 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.955770 26790 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.965022 26791 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.967717 27063 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.974769 27044 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.981071 26791 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:39.997442 26200 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.003793 27092 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.020503 26051 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.015141 26196 raft_consensus.cc:2955] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:40.025266 26200 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:40.026829 27045 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.028048 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.034080 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 549b49961cbb430d9ece6acd81aede77 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.034732 26789 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.041564 27051 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:40.045586 25950 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.046854 26127 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.037005 26051 raft_consensus.cc:2955] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:40.053514 27085 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:40.055263 25951 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.058777 26789 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:40.064426 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c7ad94af29be4656bf3ce8a023d19f7d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.065757 27104 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:40.072772 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 549b49961cbb430d9ece6acd81aede77 with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:40.073688 27094 tablet_replica.cc:333] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:40.074710 27094 raft_consensus.cc:2243] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.075459 27094 raft_consensus.cc:2272] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:40.076759 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:40.077373 26129 raft_consensus.cc:2955] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:40.079777 27094 ts_tablet_manager.cc:1916] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:40.105320 25949 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:40.109486 27108 tablet_replica.cc:333] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:40.110774 27108 raft_consensus.cc:2243] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.121865 27108 raft_consensus.cc:2272] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:40.123410 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet c7ad94af29be4656bf3ce8a023d19f7d with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:40.128042 27108 ts_tablet_manager.cc:1916] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:40.138803 27109 tablet_replica.cc:333] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:40.139982 27109 raft_consensus.cc:2243] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.141017 27109 raft_consensus.cc:2272] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:40.142849 27094 ts_tablet_manager.cc:1929] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:40.145877 27094 log.cc:1199] T 549b49961cbb430d9ece6acd81aede77 P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/549b49961cbb430d9ece6acd81aede77
I20260709 12:23:40.145402 27109 ts_tablet_manager.cc:1916] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:40.147208 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 549b49961cbb430d9ece6acd81aede77 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:40.148749 27108 ts_tablet_manager.cc:1929] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:40.149260 27108 log.cc:1199] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/86fd5767abab4e45ad1c8bd5f6d7a9c8
I20260709 12:23:40.150849 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 86fd5767abab4e45ad1c8bd5f6d7a9c8 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:40.160959 27109 ts_tablet_manager.cc:1929] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:40.161418 27109 log.cc:1199] T c7ad94af29be4656bf3ce8a023d19f7d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/c7ad94af29be4656bf3ce8a023d19f7d
I20260709 12:23:40.162866 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet c7ad94af29be4656bf3ce8a023d19f7d (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:41.198752 26051 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:41.206184 26203 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:41.206772 26129 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:41.207486 26018 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0d1927cc7714e91945918d6aa1216e5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:41.208150 27110 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:41.209782 27044 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:41.223240 27110 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.225286 26129 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.226761 26203 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.241572 26195 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:41.243009 25950 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:41.252578 26167 consensus_peers.cc:597] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0cbe5295e8db495985835a50d39e4438. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:41.254837 26121 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:41.263098 26050 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:41.265170 27115 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:41.266690 27104 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:41.321808 27051 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.322867 26051 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.349061 26200 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:41.359200 25948 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:41.363554 26121 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.366441 26129 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:41.366907 26167 consensus_peers.cc:597] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 955755ce63f743b9a01d5871aff94f6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:41.369127 26051 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:41.369498 27114 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:41.372185 27085 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:41.386507 27085 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.388669 26121 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.401438 26052 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.406323 26200 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:41.414602 25950 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:41.439884 26167 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bec84548f93742a096e72c1bd887809a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:41.444970 26121 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:41.452474 27115 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:41.461808 27085 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.464742 26052 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:41.464856 26127 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.466786 27104 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:41.480713 26052 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:41.487295 25950 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:41.520802 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08001aaf80 after lost signal to thread 24865
W20260709 12:23:41.521875 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08000362e0 after lost signal to thread 25986
W20260709 12:23:41.522981 25983 debug-util.cc:398] Leaking SignalData structure 0x7b080030cae0 after lost signal to thread 26078
W20260709 12:23:41.523943 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800225900 after lost signal to thread 26153
W20260709 12:23:41.524981 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08003154c0 after lost signal to thread 26227
W20260709 12:23:41.526109 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08003081a0 after lost signal to thread 26815
I20260709 12:23:41.653033 27133 ts_tablet_manager.cc:933] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:41.661787 27133 tablet_copy_client.cc:323] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:41.664121 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet d0d1927cc7714e91945918d6aa1216e5 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:41.664731 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet d0d1927cc7714e91945918d6aa1216e5 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-d0d1927cc7714e91945918d6aa1216e5
I20260709 12:23:41.672894 26063 tablet_copy_source_session.cc:215] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:41.677470 27133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d1927cc7714e91945918d6aa1216e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.708940 27133 tablet_copy_client.cc:806] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.709640 27133 tablet_copy_client.cc:670] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.718377 27133 tablet_copy_client.cc:538] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.761971 27133 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:41.801020 27135 ts_tablet_manager.cc:933] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:41.814949 27135 tablet_copy_client.cc:323] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:41.817415 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 0cbe5295e8db495985835a50d39e4438 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:41.818056 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 0cbe5295e8db495985835a50d39e4438 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-0cbe5295e8db495985835a50d39e4438
I20260709 12:23:41.845285 26213 tablet_copy_source_session.cc:215] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:41.849355 27135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cbe5295e8db495985835a50d39e4438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.946098 27138 ts_tablet_manager.cc:933] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:41.958329 27135 tablet_copy_client.cc:806] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.963200 27135 tablet_copy_client.cc:670] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.960964 27133 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:41.966312 27133 tablet_bootstrap.cc:492] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:41.968736 27133 ts_tablet_manager.cc:1403] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.207s	user 0.019s	sys 0.008s
I20260709 12:23:41.969118 27135 tablet_copy_client.cc:538] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.962373 27139 ts_tablet_manager.cc:933] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:41.973778 27133 raft_consensus.cc:359] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:41.974872 27133 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:41.975831 27133 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:41.984972 27135 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:41.987957 27138 tablet_copy_client.cc:323] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:41.987155 27139 tablet_copy_client.cc:323] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:41.990471 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet bec84548f93742a096e72c1bd887809a from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:41.991067 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet bec84548f93742a096e72c1bd887809a from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-bec84548f93742a096e72c1bd887809a
I20260709 12:23:41.995326 27133 ts_tablet_manager.cc:1434] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:23:42.003227 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-d0d1927cc7714e91945918d6aa1216e5 received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:42.003741 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-d0d1927cc7714e91945918d6aa1216e5 on tablet d0d1927cc7714e91945918d6aa1216e5 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:42.009262 26212 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 955755ce63f743b9a01d5871aff94f6c from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:42.009865 26212 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 955755ce63f743b9a01d5871aff94f6c from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-955755ce63f743b9a01d5871aff94f6c
I20260709 12:23:42.018280 26212 tablet_copy_source_session.cc:215] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:42.019726 26213 tablet_copy_source_session.cc:215] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:42.023008 27138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec84548f93742a096e72c1bd887809a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.030215 27139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955755ce63f743b9a01d5871aff94f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.066563 27139 tablet_copy_client.cc:806] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:42.067180 27139 tablet_copy_client.cc:670] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:42.073568 27139 tablet_copy_client.cc:538] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:42.090720 27138 tablet_copy_client.cc:806] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:42.091364 27138 tablet_copy_client.cc:670] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:42.095292 27138 tablet_copy_client.cc:538] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:42.108438 27138 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:42.120714 27139 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:42.131434 27135 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.132776 27135 tablet_bootstrap.cc:492] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:42.133605 27135 ts_tablet_manager.cc:1403] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.149s	user 0.020s	sys 0.003s
I20260709 12:23:42.143535 26788 raft_consensus.cc:1217] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:42.145781 27138 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.147286 27138 tablet_bootstrap.cc:492] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:42.160104 27138 ts_tablet_manager.cc:1403] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.007s
I20260709 12:23:42.186610 27138 raft_consensus.cc:359] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:42.187487 27138 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:42.188293 27138 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:42.191346 27138 ts_tablet_manager.cc:1434] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:23:42.192051 27139 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.193001 27135 raft_consensus.cc:359] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:42.193926 27135 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:42.194025 26212 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-bec84548f93742a096e72c1bd887809a received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:42.194490 26212 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-bec84548f93742a096e72c1bd887809a on tablet bec84548f93742a096e72c1bd887809a with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:42.194670 27135 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:42.197621 27135 ts_tablet_manager.cc:1434] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 12:23:42.193013 27139 tablet_bootstrap.cc:492] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:42.199267 27139 ts_tablet_manager.cc:1403] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.079s	user 0.024s	sys 0.012s
I20260709 12:23:42.201357 26212 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-0cbe5295e8db495985835a50d39e4438 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:42.201814 26212 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-0cbe5295e8db495985835a50d39e4438 on tablet 0cbe5295e8db495985835a50d39e4438 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:42.208926 27139 raft_consensus.cc:359] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:42.210457 27139 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:42.211293 27139 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:42.215404 27139 ts_tablet_manager.cc:1434] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:42.217393 26212 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-955755ce63f743b9a01d5871aff94f6c received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:42.217907 26212 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-955755ce63f743b9a01d5871aff94f6c on tablet 955755ce63f743b9a01d5871aff94f6c with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:42.246840 26788 raft_consensus.cc:1217] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:42.299919 26788 raft_consensus.cc:1217] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:42.484984 26791 raft_consensus.cc:1217] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:42.691274 27112 raft_consensus.cc:1064] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:42.703425 27112 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:42.769590 27115 raft_consensus.cc:1064] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:42.772539 27115 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:42.777680 27147 raft_consensus.cc:1064] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:42.799546 26788 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.792685 27147 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:42.802021 26121 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.802814 27051 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.808817 26788 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.811782 27114 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.819118 27112 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.821763 26050 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.821501 26052 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.823662 26129 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.828065 27114 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.828790 27051 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.834625 27051 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.861397 26790 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.864893 27044 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.868398 26203 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.873775 27044 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:42.978438 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 66312.5us
I20260709 12:23:42.984184 26122 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.006325 27104 raft_consensus.cc:1064] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:43.009384 27104 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:43.018975 27044 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.027215 27148 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.049000 26201 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.068058 26791 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.073590 26791 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.084296 27112 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:43.092502 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62899.4us
I20260709 12:23:43.091234 27114 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:23:43.106356 27115 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.116261 26129 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.126497 27112 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.126607 25951 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.129599 26124 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.129875 26121 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:43.145506 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72966.9us
I20260709 12:23:43.149626 25948 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.163287 26127 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.180735 26052 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.179461 26050 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.187598 26047 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:43.257196 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 183780us
I20260709 12:23:43.261171 26791 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.266410 26790 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.290549 27104 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 12:23:43.350869 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64136.8us
I20260709 12:23:43.372864 27112 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.481906 25951 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.465600 26790 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:43.520924 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 109133us
I20260709 12:23:43.564592 26129 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.580204 26052 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:43.580400 25951 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.603062 26788 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.608700 26122 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.612823 27148 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.625556 26195 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:43.627192 27147 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.633847 26201 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:43.646519 26790 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.648923 26122 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.649034 27104 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.648432 26050 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.657197 27165 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.672437 27162 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.677070 26791 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.678697 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d0d1927cc7714e91945918d6aa1216e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.685420 26124 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.692435 27112 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.697080 26789 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.705485 27178 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.705562 26788 raft_consensus.cc:2955] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.714432 26122 raft_consensus.cc:2955] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.726363 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0cbe5295e8db495985835a50d39e4438 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.727993 26791 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.731117 27162 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.745057 26051 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.754496 25951 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.754302 25948 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.765669 26203 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:43.773442 26791 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.775522 27171 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.779664 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bec84548f93742a096e72c1bd887809a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.793993 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet 0cbe5295e8db495985835a50d39e4438 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:43.803493 27178 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.811646 27112 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.813882 26791 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.815682 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet d0d1927cc7714e91945918d6aa1216e5 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:43.816365 26122 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.837092 26048 raft_consensus.cc:2955] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.844400 27104 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.850214 26127 raft_consensus.cc:2955] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.855432 25949 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.875510 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet bec84548f93742a096e72c1bd887809a with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:43.891400 27180 tablet_replica.cc:333] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:43.903697 27179 tablet_replica.cc:333] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:43.904959 27179 raft_consensus.cc:2243] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.905886 27179 raft_consensus.cc:2272] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.930267 27182 tablet_replica.cc:333] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:43.935551 27179 ts_tablet_manager.cc:1916] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.932068 27182 raft_consensus.cc:2243] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.957151 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 955755ce63f743b9a01d5871aff94f6c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.963608 25951 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:43.968549 27180 raft_consensus.cc:2243] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.979528 27180 raft_consensus.cc:2272] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.992197 27180 ts_tablet_manager.cc:1916] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:44.009061 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet 955755ce63f743b9a01d5871aff94f6c with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:43.966228 27182 raft_consensus.cc:2272] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:44.013864 27182 ts_tablet_manager.cc:1916] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:44.026889 27180 ts_tablet_manager.cc:1929] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:44.027534 27180 log.cc:1199] T d0d1927cc7714e91945918d6aa1216e5 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/d0d1927cc7714e91945918d6aa1216e5
I20260709 12:23:44.029661 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet d0d1927cc7714e91945918d6aa1216e5 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:44.034584 27179 ts_tablet_manager.cc:1929] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:44.035023 27179 log.cc:1199] T 0cbe5295e8db495985835a50d39e4438 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/0cbe5295e8db495985835a50d39e4438
I20260709 12:23:44.036971 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet 0cbe5295e8db495985835a50d39e4438 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:44.040881 27179 tablet_replica.cc:333] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:44.042270 27179 raft_consensus.cc:2243] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:44.043219 27179 raft_consensus.cc:2272] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:44.045238 27182 ts_tablet_manager.cc:1929] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:44.045758 27182 log.cc:1199] T bec84548f93742a096e72c1bd887809a P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/bec84548f93742a096e72c1bd887809a
I20260709 12:23:44.102545 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet bec84548f93742a096e72c1bd887809a (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:44.102865 27179 ts_tablet_manager.cc:1916] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:44.151124 27179 ts_tablet_manager.cc:1929] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:44.152834 27179 log.cc:1199] T 955755ce63f743b9a01d5871aff94f6c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/955755ce63f743b9a01d5871aff94f6c
I20260709 12:23:44.154556 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet 955755ce63f743b9a01d5871aff94f6c (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:45.102658 26127 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:45.128436 26093 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc3c86db3d4e4643a77b0fe0b0e687e1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.128839 26203 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.131072 26048 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.131829 27095 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.133795 27183 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.146039 27095 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.147781 26203 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.155496 26050 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.166972 25951 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.185724 26203 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:45.206751 26167 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ee8b7d54fee44329721d2da7c8f62ad. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.207109 26127 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.209326 26050 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.210358 27114 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.213053 27114 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.224762 27112 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.238037 26127 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.261271 26050 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.267814 26124 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:45.271055 25951 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:45.300128 26093 consensus_peers.cc:597] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eac7044e09a64594b8fcd962f1d387b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.303960 26203 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.305985 27095 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.306393 26050 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.316835 27095 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.351869 27183 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.371282 26124 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:45.374361 26050 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.376031 26203 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.400941 25948 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.415738 26203 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:45.417164 26093 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3fe81c3162f24fc0b8716b2b42f0676f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.418229 26050 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:45.418310 27198 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.421288 27095 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.434410 27095 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.438254 26050 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.456391 26203 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:45.476761 25948 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.701793 27204 ts_tablet_manager.cc:933] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:45.711436 27204 tablet_copy_client.cc:323] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:45.713901 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet dc3c86db3d4e4643a77b0fe0b0e687e1 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:45.714481 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet dc3c86db3d4e4643a77b0fe0b0e687e1 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-dc3c86db3d4e4643a77b0fe0b0e687e1
I20260709 12:23:45.731796 26139 tablet_copy_source_session.cc:215] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.734915 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3c86db3d4e4643a77b0fe0b0e687e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.776700 27206 ts_tablet_manager.cc:933] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:45.781452 27204 tablet_copy_client.cc:806] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.782218 27204 tablet_copy_client.cc:670] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.788846 27204 tablet_copy_client.cc:538] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.792080 27208 ts_tablet_manager.cc:933] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:45.796640 27208 tablet_copy_client.cc:323] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:45.799656 26212 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 2ee8b7d54fee44329721d2da7c8f62ad from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:45.800308 26212 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 2ee8b7d54fee44329721d2da7c8f62ad from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-2ee8b7d54fee44329721d2da7c8f62ad
I20260709 12:23:45.808288 27204 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:45.810340 26212 tablet_copy_source_session.cc:215] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.821749 27206 tablet_copy_client.cc:323] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:45.825651 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet eac7044e09a64594b8fcd962f1d387b7 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:45.826200 27208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee8b7d54fee44329721d2da7c8f62ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.826248 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet eac7044e09a64594b8fcd962f1d387b7 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-eac7044e09a64594b8fcd962f1d387b7
I20260709 12:23:45.839023 26139 tablet_copy_source_session.cc:215] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.843573 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac7044e09a64594b8fcd962f1d387b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.851433 27204 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.852651 27204 tablet_bootstrap.cc:492] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:45.853480 27204 ts_tablet_manager.cc:1403] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.045s	user 0.029s	sys 0.005s
I20260709 12:23:45.858034 27204 raft_consensus.cc:359] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:45.859283 27204 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:45.860052 27204 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:45.884604 27204 ts_tablet_manager.cc:1434] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.011s
I20260709 12:23:45.890345 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-dc3c86db3d4e4643a77b0fe0b0e687e1 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:45.890916 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-dc3c86db3d4e4643a77b0fe0b0e687e1 on tablet dc3c86db3d4e4643a77b0fe0b0e687e1 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:45.913980 27206 tablet_copy_client.cc:806] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.913996 27208 tablet_copy_client.cc:806] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.914847 27208 tablet_copy_client.cc:670] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.914734 27206 tablet_copy_client.cc:670] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.919214 27208 tablet_copy_client.cc:538] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.920939 27206 tablet_copy_client.cc:538] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.934705 27208 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:45.950152 27206 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:46.035012 27208 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:46.036057 27208 tablet_bootstrap.cc:492] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:46.036746 27208 ts_tablet_manager.cc:1403] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.013s
I20260709 12:23:46.039880 27208 raft_consensus.cc:359] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:46.040797 27208 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:46.041522 27208 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:46.042220 27204 ts_tablet_manager.cc:933] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:46.045598 27208 ts_tablet_manager.cc:1434] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:23:46.049190 27206 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:46.052542 26212 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-2ee8b7d54fee44329721d2da7c8f62ad received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:46.053190 26212 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-2ee8b7d54fee44329721d2da7c8f62ad on tablet 2ee8b7d54fee44329721d2da7c8f62ad with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:46.061163 27204 tablet_copy_client.cc:323] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:46.053246 27206 tablet_bootstrap.cc:492] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:46.065101 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 3fe81c3162f24fc0b8716b2b42f0676f from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:46.065656 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 3fe81c3162f24fc0b8716b2b42f0676f from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-3fe81c3162f24fc0b8716b2b42f0676f
I20260709 12:23:46.073859 26139 tablet_copy_source_session.cc:215] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:46.074496 27206 ts_tablet_manager.cc:1403] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.125s	user 0.031s	sys 0.044s
I20260709 12:23:46.078029 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe81c3162f24fc0b8716b2b42f0676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.079780 27206 raft_consensus.cc:359] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:46.080821 27206 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:46.081666 27206 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:46.084794 27206 ts_tablet_manager.cc:1434] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:23:46.093403 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-eac7044e09a64594b8fcd962f1d387b7 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:46.093871 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-eac7044e09a64594b8fcd962f1d387b7 on tablet eac7044e09a64594b8fcd962f1d387b7 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:46.095758 27204 tablet_copy_client.cc:806] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:46.096432 27204 tablet_copy_client.cc:670] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:46.102337 27204 tablet_copy_client.cc:538] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:46.113556 27204 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:46.139829 27204 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:46.141271 27204 tablet_bootstrap.cc:492] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:46.142062 27204 ts_tablet_manager.cc:1403] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.013s
I20260709 12:23:46.145488 27204 raft_consensus.cc:359] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:46.146427 27204 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:46.147212 27204 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:46.149850 27204 ts_tablet_manager.cc:1434] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:23:46.155139 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-3fe81c3162f24fc0b8716b2b42f0676f received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:46.155833 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-3fe81c3162f24fc0b8716b2b42f0676f on tablet 3fe81c3162f24fc0b8716b2b42f0676f with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:46.204191 26789 raft_consensus.cc:1217] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:46.281649 26791 raft_consensus.cc:1217] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:46.297257 26791 raft_consensus.cc:1217] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:46.499176 26791 raft_consensus.cc:1217] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:46.739444 27216 raft_consensus.cc:1064] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:46.742051 27216 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:46.760890 26203 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.762004 26050 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.763350 27217 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.765312 27216 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.773834 26789 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.775971 27216 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.804735 27095 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.836165 27217 raft_consensus.cc:1064] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:46.845122 27217 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:46.845661 25949 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:46.854260 26048 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.832487 26203 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.819289 26789 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.862282 27193 raft_consensus.cc:1064] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:46.865638 27193 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:46.878389 26791 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.880149 27095 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.885916 26048 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.886806 26788 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.887305 26129 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.888779 27178 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.891144 27193 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.894794 27112 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.911919 26203 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.914134 27112 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.915167 26048 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.916888 26051 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.920390 27219 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.928025 27217 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.939034 26129 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.947896 26790 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.962683 27219 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.975021 26048 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.986960 26790 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:46.993583 26124 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.001206 26201 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.003124 26203 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.004796 27095 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.009588 27216 raft_consensus.cc:1064] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:47.018682 27219 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.024996 25949 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.012022 27216 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.026300 26203 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.027369 26790 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.047086 25950 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.054112 26203 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.058826 27217 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:47.063882 26048 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.064476 27095 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.071338 27095 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:47.086369 26201 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.094828 26790 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.096210 27114 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.098524 26048 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.103624 27112 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.095640 26124 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.127770 27114 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.130403 26789 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.144531 26048 raft_consensus.cc:2955] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.154575 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2ee8b7d54fee44329721d2da7c8f62ad with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:47.182451 26051 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.182309 27246 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.190577 25949 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.191515 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:47.196285 27095 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:47.201481 26051 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.228909 25948 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:23:47.248085 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 201543us
I20260709 12:23:47.249998 26789 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.253671 26790 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.255815 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee8b7d54fee44329721d2da7c8f62ad with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:47.261245 26790 raft_consensus.cc:2955] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.265331 27250 tablet_replica.cc:333] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:47.265906 27217 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.284987 27250 raft_consensus.cc:2243] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:47.277377 27249 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:47.278451 27095 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.259195 25950 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.307588 26201 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.309015 26789 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.313651 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc3c86db3d4e4643a77b0fe0b0e687e1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:47.328711 27250 raft_consensus.cc:2272] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:47.330355 26790 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.303100 26051 raft_consensus.cc:2955] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.347991 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet eac7044e09a64594b8fcd962f1d387b7 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:47.356176 27250 ts_tablet_manager.cc:1916] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:47.358315 25950 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.379115 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eac7044e09a64594b8fcd962f1d387b7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:47.381886 27253 tablet_replica.cc:333] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:47.385145 27253 raft_consensus.cc:2243] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:47.391305 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet dc3c86db3d4e4643a77b0fe0b0e687e1 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:47.403220 27250 ts_tablet_manager.cc:1929] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:47.403872 27250 log.cc:1199] T 2ee8b7d54fee44329721d2da7c8f62ad P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/2ee8b7d54fee44329721d2da7c8f62ad
I20260709 12:23:47.405680 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 2ee8b7d54fee44329721d2da7c8f62ad (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:47.409808 27253 raft_consensus.cc:2272] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:47.450668 27253 ts_tablet_manager.cc:1916] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:47.451963 27254 tablet_replica.cc:333] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:47.491106 27254 raft_consensus.cc:2243] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:47.493784 27254 raft_consensus.cc:2272] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:47.544780 27254 ts_tablet_manager.cc:1916] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:47.564749 26201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:47.565599 26201 raft_consensus.cc:606] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:47.606940 27253 ts_tablet_manager.cc:1929] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:47.607851 27253 log.cc:1199] T eac7044e09a64594b8fcd962f1d387b7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/eac7044e09a64594b8fcd962f1d387b7
I20260709 12:23:47.611593 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet eac7044e09a64594b8fcd962f1d387b7 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:47.614225 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:47.615084 26048 raft_consensus.cc:606] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:47.613873 27254 ts_tablet_manager.cc:1929] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:47.616575 27254 log.cc:1199] T dc3c86db3d4e4643a77b0fe0b0e687e1 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/dc3c86db3d4e4643a77b0fe0b0e687e1
I20260709 12:23:47.620204 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "407574b4cda94da99be372d249d21fd5"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:47.620910 26203 raft_consensus.cc:606] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:47.623482 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet dc3c86db3d4e4643a77b0fe0b0e687e1 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:47.629923 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6389d613e36a42958d24b00099843b93"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:47.630650 26048 raft_consensus.cc:606] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:47.634351 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:47.634955 26048 raft_consensus.cc:606] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:47.637709 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:47.639438 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:47.640580 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:47.646315 27252 raft_consensus.cc:993] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:47.646859 27252 raft_consensus.cc:1081] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:47.649147 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38294
I20260709 12:23:47.649791 26789 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.650327 26789 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.677484 26789 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.682062 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:47.682590 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:47.683241 26203 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.683604 26203 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:47.684429 26203 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.685622 26203 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.682794 26048 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.696431 26048 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:47.698213 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:23:47.699091 27211 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.705871 26789 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:47.710235 26203 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:47.712697 27211 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:47.717546 27211 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.729400 25951 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.826359 27112 raft_consensus.cc:993] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:47.826931 27112 raft_consensus.cc:1081] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:47.830354 26122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "407574b4cda94da99be372d249d21fd5"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:47.830895 26122 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.831208 26122 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.851086 27227 raft_consensus.cc:993] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:47.851743 27226 raft_consensus.cc:1081] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:47.855974 26124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:23:47.856585 26124 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.856911 26124 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.860217 26122 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:47.862178 26122 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:47.863142 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:47.863726 26203 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.864009 26203 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:47.864714 26203 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:47.865893 26203 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.867805 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:47.868422 26048 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.874607 26124 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:47.876705 26048 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:47.876938 26203 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:47.878901 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:47.880450 27219 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.880985 27219 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:47.881897 27219 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:47.884264 26124 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:47.886000 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:47.886605 26047 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.886874 26047 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:47.887590 26047 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:47.890515 26047 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.891042 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:47.891578 26203 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.898651 25951 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.903024 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:47.905330 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:47.906481 27216 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.907059 27216 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:47.895594 26047 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:47.908087 27216 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:47.927690 25951 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.940258 26124 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:47.945636 26789 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.945806 26047 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:47.947340 27216 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.948324 27219 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.955682 27219 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.960768 26048 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.963013 26789 raft_consensus.cc:2955] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:47.980176 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3fe81c3162f24fc0b8716b2b42f0676f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:47.989269 25951 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:48.000937 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe81c3162f24fc0b8716b2b42f0676f with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:48.001892 27254 tablet_replica.cc:333] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:48.002972 27254 raft_consensus.cc:2243] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:48.003731 27254 raft_consensus.cc:2272] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:48.006909 27254 ts_tablet_manager.cc:1916] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:48.020077 27254 ts_tablet_manager.cc:1929] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:48.020431 27254 log.cc:1199] T 3fe81c3162f24fc0b8716b2b42f0676f P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/3fe81c3162f24fc0b8716b2b42f0676f
I20260709 12:23:48.021867 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 3fe81c3162f24fc0b8716b2b42f0676f (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:48.029135 27226 raft_consensus.cc:993] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:48.029744 27227 raft_consensus.cc:1081] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:48.031477 26124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6389d613e36a42958d24b00099843b93"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:23:48.032058 26124 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.032512 26124 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.040436 26124 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:48.042207 26124 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:48.043339 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:48.043908 26048 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.044186 26048 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:48.044553 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:48.045149 26203 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.045014 26048 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:48.046360 26048 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.051810 26203 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:48.051847 26048 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:48.053177 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:48.053838 27216 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.054320 27216 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:48.055405 27216 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:48.072129 27226 raft_consensus.cc:993] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:48.073282 27226 raft_consensus.cc:1081] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:48.072410 25948 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.074864 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38282
I20260709 12:23:48.075302 26789 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.075573 26789 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.081353 26789 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:48.083226 26789 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:48.083928 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:48.084198 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:48.084648 26048 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.084786 26122 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.085047 26048 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:48.085836 26048 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:48.087062 26048 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.091893 26122 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:48.093273 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:48.094184 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:48.094249 27211 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.094831 27211 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:48.095741 27211 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:48.105634 25948 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.173274 26203 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.174743 27211 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.196005 26048 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.197646 27258 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.358913 26048 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.360257 27216 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.375615 26203 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.378044 27219 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.410897 26203 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.412353 27219 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.425272 26048 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.426877 27216 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.457460 26048 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.458880 27219 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.470492 26203 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.473047 27243 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.477941 26122 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.479614 27258 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.516038 26048 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.517409 27258 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.776731 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:48.821897 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:48.822494 26203 raft_consensus.cc:606] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:48.824887 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:48.825287 26203 raft_consensus.cc:606] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:48.828040 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:48.828544 26203 raft_consensus.cc:606] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:48.830832 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:48.831372 26203 raft_consensus.cc:606] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:48.833781 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:48.834956 26203 raft_consensus.cc:606] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:48.836191 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:48.837644 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:48.838233 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:48.982736 27252 raft_consensus.cc:993] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:48.983254 27193 raft_consensus.cc:1081] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:48.984864 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38294
I20260709 12:23:48.985352 26789 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.985647 26789 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.990298 26789 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:48.991886 26789 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:48.992626 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:48.992808 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:48.993196 26127 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.993359 26201 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.993732 26201 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:48.994320 26201 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:48.995262 26201 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.999469 26127 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:49.000536 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:49.000634 26201 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:49.001509 27211 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.001998 27211 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:49.002911 27211 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:49.013062 25948 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:49.019452 27252 raft_consensus.cc:993] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:49.020011 27252 raft_consensus.cc:1081] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:49.021797 26048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47798
I20260709 12:23:49.022352 26048 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.022735 26048 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.034926 26048 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.039486 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:49.039846 26048 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.040199 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:49.040866 26201 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.041329 26201 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:49.042059 26201 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.043529 26201 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.040199 26127 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.051049 26127 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:49.051085 26201 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:49.052939 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:49.053817 27227 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.056350 27227 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:49.057317 27227 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.075506 25948 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.098773 26201 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:49.105067 26167 consensus_peers.cc:597] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: adcc76aa423a40eb95d31d98a8c8776f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:49.104944 26048 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:49.105139 26127 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:49.106479 27252 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.107349 27112 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.115906 27193 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.117741 26048 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.118187 26127 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.150218 25951 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.153381 27112 raft_consensus.cc:993] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:49.153244 26201 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:49.153887 27112 raft_consensus.cc:1081] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:49.156563 26124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:49.157161 26124 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.157500 26124 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:49.164335 26167 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5a510044c5b459abfd55896cc2d21eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:49.165407 26124 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.167418 26124 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.168720 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:49.169454 26048 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.170246 26051 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:49.171428 26124 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:49.173861 27252 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.177603 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:49.178423 26201 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.178902 26201 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:49.180490 26201 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.180992 27289 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.181897 26201 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.186082 26048 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:49.187386 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:49.188156 27280 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.188742 27280 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:49.189777 27280 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.205855 26051 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.203981 27290 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.226795 25948 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.235250 25951 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.226667 26127 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.257016 26201 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:49.260124 26051 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:49.277374 26201 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.278915 27227 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:49.283030 26018 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 891138e164dc40a7af7618323d52397f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:49.288081 26127 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.290210 27227 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.289633 27226 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.296300 26201 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.300582 26122 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.320746 26201 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:49.317967 25948 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.337589 27252 raft_consensus.cc:993] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:49.339567 26051 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:49.340458 27112 raft_consensus.cc:1081] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:49.342279 27289 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:49.343884 26167 consensus_peers.cc:597] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e9b765338c61436481fe416f17637ab2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:49.347342 26122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:49.348312 26122 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.348845 26122 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.354926 26124 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:49.357169 27112 raft_consensus.cc:993] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:49.363382 27290 raft_consensus.cc:1081] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:49.363443 26122 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.365837 26122 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.366377 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38294
I20260709 12:23:49.367177 26789 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.367643 26789 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.367429 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:49.372725 26047 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.373920 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:49.374670 26201 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.374982 26201 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:49.376057 26201 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.377374 26201 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.385661 26201 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:49.388111 26789 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:49.389115 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:49.393313 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:49.391856 26789 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.394014 26047 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.394953 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:49.393376 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:49.396842 26201 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.397169 26201 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:49.401934 27216 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.402706 27216 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:49.398269 26201 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:49.409001 26201 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:49.415671 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58187.8us
I20260709 12:23:49.416747 27193 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.417172 26201 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
W20260709 12:23:49.417289 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68163.1us
I20260709 12:23:49.418339 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:49.403980 27216 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:49.419571 27211 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.420079 27211 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:49.421062 27211 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:49.426191 27193 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.435192 26048 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.442515 26047 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:49.449115 25951 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.461272 25948 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:49.463565 25949 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.473083 26129 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.492014 27258 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.517860 26127 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.524722 27227 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.538113 26203 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.539633 27211 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:49.544405 26122 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:49.570627 26201 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.575510 27226 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.645893 27316 ts_tablet_manager.cc:933] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.695549 27316 tablet_copy_client.cc:323] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:49.697798 26048 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.708144 27320 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:23:49.729486 27323 ts_tablet_manager.cc:933] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.737205 26212 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet f5a510044c5b459abfd55896cc2d21eb from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:49.737834 26212 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet f5a510044c5b459abfd55896cc2d21eb from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-f5a510044c5b459abfd55896cc2d21eb
I20260709 12:23:49.741818 26195 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.748570 27323 tablet_copy_client.cc:323] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:49.762490 27317 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:23:49.763732 26212 tablet_copy_source_session.cc:215] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:49.765014 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet adcc76aa423a40eb95d31d98a8c8776f from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:49.765618 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet adcc76aa423a40eb95d31d98a8c8776f from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-adcc76aa423a40eb95d31d98a8c8776f
I20260709 12:23:49.772804 27316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a510044c5b459abfd55896cc2d21eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:49.779296 26213 tablet_copy_source_session.cc:215] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:49.786079 27323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcc76aa423a40eb95d31d98a8c8776f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:49.805819 27316 tablet_copy_client.cc:806] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:49.806447 27316 tablet_copy_client.cc:670] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:49.823377 27316 tablet_copy_client.cc:538] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:49.827967 27327 ts_tablet_manager.cc:933] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:49.833249 27316 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:49.840696 27327 tablet_copy_client.cc:323] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:49.842193 26200 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.843114 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet e9b765338c61436481fe416f17637ab2 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:49.843657 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet e9b765338c61436481fe416f17637ab2 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-e9b765338c61436481fe416f17637ab2
I20260709 12:23:49.844748 27323 tablet_copy_client.cc:806] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:49.845532 27323 tablet_copy_client.cc:670] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:49.845737 27211 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.859692 27323 tablet_copy_client.cc:538] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:49.881404 26213 tablet_copy_source_session.cc:215] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:49.885571 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:49.898871 27323 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:49.899462 26048 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.900425 27329 ts_tablet_manager.cc:933] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:49.902608 27320 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.903010 27327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b765338c61436481fe416f17637ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:49.931659 26047 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.939291 27278 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.951967 27329 tablet_copy_client.cc:323] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:49.959233 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 891138e164dc40a7af7618323d52397f from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:23:49.959883 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 891138e164dc40a7af7618323d52397f from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-891138e164dc40a7af7618323d52397f
I20260709 12:23:49.974149 26195 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.978192 27317 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.024587 27316 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:50.031323 27316 tablet_bootstrap.cc:492] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:50.032114 27316 ts_tablet_manager.cc:1403] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.199s	user 0.035s	sys 0.019s
I20260709 12:23:50.039592 27323 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:50.041290 27323 tablet_bootstrap.cc:492] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:50.042028 27323 ts_tablet_manager.cc:1403] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.143s	user 0.021s	sys 0.026s
I20260709 12:23:50.044021 26063 tablet_copy_source_session.cc:215] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:50.044324 27316 raft_consensus.cc:359] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.064393 27316 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:50.065215 27316 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.068985 27329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891138e164dc40a7af7618323d52397f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:50.071600 27316 ts_tablet_manager.cc:1434] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
I20260709 12:23:50.046519 27323 raft_consensus.cc:359] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.072561 27323 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:50.073320 27323 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.076112 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-f5a510044c5b459abfd55896cc2d21eb received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:50.076592 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-f5a510044c5b459abfd55896cc2d21eb on tablet f5a510044c5b459abfd55896cc2d21eb with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.076129 27323 ts_tablet_manager.cc:1434] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 12:23:50.084110 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-adcc76aa423a40eb95d31d98a8c8776f received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:50.084547 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-adcc76aa423a40eb95d31d98a8c8776f on tablet adcc76aa423a40eb95d31d98a8c8776f with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.086030 27327 tablet_copy_client.cc:806] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:50.086709 27327 tablet_copy_client.cc:670] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:50.094915 27329 tablet_copy_client.cc:806] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:50.096997 27327 tablet_copy_client.cc:538] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:50.107753 26790 raft_consensus.cc:1217] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:50.095505 27329 tablet_copy_client.cc:670] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:50.125803 27329 tablet_copy_client.cc:538] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:50.146466 27329 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:50.154340 27327 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:50.164175 26200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:50.164855 26200 raft_consensus.cc:606] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:50.168118 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:50.169147 26048 raft_consensus.cc:606] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.172508 26200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:50.174121 26200 raft_consensus.cc:606] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.192126 26200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:50.192749 26200 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:50.196563 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:50.197152 26203 raft_consensus.cc:606] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.200172 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:50.201915 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.202666 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.208333 27329 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:50.209345 27329 tablet_bootstrap.cc:492] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:50.210045 27329 ts_tablet_manager.cc:1403] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.018s
I20260709 12:23:50.213054 27329 raft_consensus.cc:359] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.213972 27329 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:50.215484 27327 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:50.214636 27329 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.219664 27327 tablet_bootstrap.cc:492] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:50.220438 27327 ts_tablet_manager.cc:1403] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.023s
I20260709 12:23:50.221520 27329 ts_tablet_manager.cc:1434] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.006s
I20260709 12:23:50.222178 27299 raft_consensus.cc:993] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:50.222661 27299 raft_consensus.cc:1081] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:50.223302 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-891138e164dc40a7af7618323d52397f received from {username='slave'} at 127.0.0.1:46972
I20260709 12:23:50.223770 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-891138e164dc40a7af7618323d52397f on tablet 891138e164dc40a7af7618323d52397f with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.223521 27327 raft_consensus.cc:359] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.224460 27327 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:50.224610 26790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38282
I20260709 12:23:50.225129 26790 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.225591 26790 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.225167 27327 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.231695 27327 ts_tablet_manager.cc:1434] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.006s
I20260709 12:23:50.233870 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-e9b765338c61436481fe416f17637ab2 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:50.234301 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-e9b765338c61436481fe416f17637ab2 on tablet e9b765338c61436481fe416f17637ab2 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:50.239393 26790 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.242200 26790 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:50.243232 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:50.243850 26124 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.246539 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:50.247197 26048 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.247643 26048 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:50.248627 26048 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.250281 26048 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.250494 26124 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:50.251929 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:50.252900 27211 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.253362 27211 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:50.254172 27211 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.257576 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:50.281538 25948 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.312873 26789 raft_consensus.cc:1217] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:50.333693 26789 raft_consensus.cc:1217] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:50.358599 27289 raft_consensus.cc:993] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:50.359153 27289 raft_consensus.cc:1081] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:50.362110 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38294
I20260709 12:23:50.362637 26789 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.363034 26789 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.368801 27290 raft_consensus.cc:993] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:50.369328 27290 raft_consensus.cc:1081] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:50.371503 26122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:50.372149 26122 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.372603 26122 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.375589 26789 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.378805 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:50.379091 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:50.379434 26124 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.379675 26200 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.380013 26200 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:50.380824 26200 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.381950 26200 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.397068 26124 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:50.378218 26789 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:50.398563 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:50.399566 27211 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.400096 27211 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:50.401088 27211 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.439615 25949 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.443346 26200 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:50.444322 26122 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:50.451928 26122 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:50.454497 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:50.456264 26195 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.456573 26195 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:50.457262 26195 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:50.458263 26195 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.462496 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:50.463105 26048 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.466385 26195 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:50.467687 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:50.468780 27219 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.469254 27219 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:50.470077 27219 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:50.472220 26048 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:50.484649 26789 raft_consensus.cc:1217] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:23:50.502281 25948 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.533614 27302 raft_consensus.cc:993] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:50.534106 27302 raft_consensus.cc:1081] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:50.535633 26051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47798
I20260709 12:23:50.536803 26051 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.537230 26051 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.537493 27302 raft_consensus.cc:993] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:50.537968 27302 raft_consensus.cc:1081] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:50.539551 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38294
I20260709 12:23:50.540068 26789 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.540439 26789 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.543421 26051 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.545562 26051 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:50.545884 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:50.546484 26122 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.547261 26789 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.549974 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:50.550168 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:50.550551 26129 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.550773 26195 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.551362 26195 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:50.552124 26195 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.553225 26195 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.554674 26122 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:50.556104 26020 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:50.557207 27297 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.557736 26129 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:50.559096 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:50.560065 27258 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.560633 27258 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:50.561537 26195 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:50.561537 27258 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.562741 27297 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:50.549974 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:50.564303 26200 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.564594 26200 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:50.565330 26200 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.571283 27297 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:50.573405 26789 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:50.566884 26200 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.587162 26200 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:50.587405 25948 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.599566 25949 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.613722 26051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:50.614362 26051 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:23:50.617650 26051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:50.618114 26051 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:23:50.619009 25938 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for adcc76aa423a40eb95d31d98a8c8776f already in progress [suppressed 4 similar messages]
I20260709 12:23:50.622110 26051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:50.622661 26051 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:23:50.627149 26000 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:23:50.627620 26000 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for adcc76aa423a40eb95d31d98a8c8776f already in progress
W20260709 12:23:50.627946 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for adcc76aa423a40eb95d31d98a8c8776f already in progress
I20260709 12:23:50.644279 26129 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.645677 27258 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.683363 27304 raft_consensus.cc:1064] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:50.686094 27304 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.689924 26051 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.693611 27345 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.727053 26789 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:50.727676 26051 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:50.730269 26129 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:50.731987 27290 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.733846 27290 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.742892 27302 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.765095 27290 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.770133 27302 raft_consensus.cc:1064] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:50.772866 27302 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.789206 26122 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.796207 26790 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:50.797879 26127 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:50.800544 27289 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:50.802739 27289 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.807204 26048 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.812922 26789 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.817343 25948 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.828311 26051 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:50.830557 27356 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.842944 26122 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.845041 27345 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.852939 26200 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.866823 26790 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.869854 27362 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:23:50.871976 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72601.8us
I20260709 12:23:50.879411 26195 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.881381 27258 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.883651 26050 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.886966 27361 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:50.896407 27362 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.901770 26050 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.905850 27317 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.908723 26047 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.929654 27297 raft_consensus.cc:1064] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:50.928889 26790 raft_consensus.cc:2955] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.933161 27290 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.941964 27297 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:50.948125 26051 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.958596 26124 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.967453 26200 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.969959 27320 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.977780 26122 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.978293 26200 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.978734 26789 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.979231 27299 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:50.980165 27226 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.982935 27299 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:50.982700 26791 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:50.987289 26127 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.990561 27226 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.000684 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f5a510044c5b459abfd55896cc2d21eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:51.005182 27372 raft_consensus.cc:993] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:51.005722 27372 raft_consensus.cc:1081] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:51.006547 26200 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.007684 26124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:23:51.008273 26124 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.008617 26124 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.013664 26788 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Advancing to term 2
I20260709 12:23:51.016073 27299 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:51.030495 26788 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.034133 26124 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:51.046196 27226 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.046924 27372 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.053563 26122 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:51.054241 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:51.055855 26124 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:51.056159 25951 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.057158 26127 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.067488 26789 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.072064 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:51.073385 27226 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.075062 26051 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:51.075563 26051 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:51.076449 26051 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:51.078171 26051 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:51.077433 27374 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 12:23:51.036711 25948 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.083388 26203 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.088912 26051 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:23:51.091926 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:23:51.092358 27226 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:23:51.093091 27219 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:51.093642 27219 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:51.094586 27219 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:51.103832 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet f5a510044c5b459abfd55896cc2d21eb with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:51.105022 26201 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.109107 27258 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.120934 26127 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.124635 27345 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.125057 25949 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:51.152702 27380 tablet_replica.cc:333] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:51.155478 27380 raft_consensus.cc:2243] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:51.155081 25950 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.167433 26203 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:51.169858 26051 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:51.182508 26789 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:51.186127 27302 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.187453 26127 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:51.189111 26789 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:51.189069 27372 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.192261 27372 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.168998 27380 raft_consensus.cc:2272] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:51.202684 26124 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:51.203097 27362 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.204789 27302 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.210624 26124 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.213481 27380 ts_tablet_manager.cc:1916] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:51.217571 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e9b765338c61436481fe416f17637ab2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:51.221968 27299 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.223165 26790 raft_consensus.cc:2955] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.226709 26788 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.242920 26124 raft_consensus.cc:2955] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.261106 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 891138e164dc40a7af7618323d52397f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:51.262054 25949 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.263669 25949 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.272841 27380 ts_tablet_manager.cc:1929] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:51.273509 27380 log.cc:1199] T f5a510044c5b459abfd55896cc2d21eb P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/f5a510044c5b459abfd55896cc2d21eb
I20260709 12:23:51.275569 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet f5a510044c5b459abfd55896cc2d21eb (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:51.289765 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 891138e164dc40a7af7618323d52397f with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:51.295440 27383 tablet_replica.cc:333] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:51.297030 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet e9b765338c61436481fe416f17637ab2 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:51.303555 27384 tablet_replica.cc:333] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:51.304659 27384 raft_consensus.cc:2243] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:51.305397 27384 raft_consensus.cc:2272] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:51.309674 27384 ts_tablet_manager.cc:1916] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:51.310283 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.320780 27383 raft_consensus.cc:2243] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:51.321754 27383 raft_consensus.cc:2272] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:51.324285 27384 ts_tablet_manager.cc:1929] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:51.324842 27384 log.cc:1199] T e9b765338c61436481fe416f17637ab2 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/e9b765338c61436481fe416f17637ab2
I20260709 12:23:51.325004 27383 ts_tablet_manager.cc:1916] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:51.327047 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet e9b765338c61436481fe416f17637ab2 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:51.340353 27383 ts_tablet_manager.cc:1929] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:23:51.340926 27383 log.cc:1199] T 891138e164dc40a7af7618323d52397f P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/891138e164dc40a7af7618323d52397f
I20260709 12:23:51.342738 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 891138e164dc40a7af7618323d52397f (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:51.364670 26124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:51.365150 26124 raft_consensus.cc:606] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:51.367671 26124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "407574b4cda94da99be372d249d21fd5"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:51.368371 26124 raft_consensus.cc:606] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:51.371308 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:51.371850 26203 raft_consensus.cc:606] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:51.375054 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:51.375675 26203 raft_consensus.cc:606] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:51.378249 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:51.378823 26203 raft_consensus.cc:606] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:51.380457 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:51.381978 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:51.382678 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.396909 27290 raft_consensus.cc:993] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:51.397468 27362 raft_consensus.cc:1081] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:51.399173 26788 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38294
I20260709 12:23:51.399660 26788 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.399950 26788 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.406023 26788 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:51.407640 26788 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:51.408593 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:51.408586 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:51.409377 26203 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:51.409502 26051 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.409746 26203 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:51.410600 26203 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:51.411810 26203 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.417970 26051 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:51.421085 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:23:51.421948 27258 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:51.422447 27258 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:51.423281 27258 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:51.426716 26203 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:51.437182 25949 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:51.498975 27290 raft_consensus.cc:993] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:51.499508 27362 raft_consensus.cc:1081] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:51.500937 26127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:51.501489 26127 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.501838 26127 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.507234 26127 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.509645 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:51.510281 26051 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.510493 26127 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:51.511279 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:51.511927 26203 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:51.512296 26203 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:51.513762 26203 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.514834 26203 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.516177 26051 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:51.517431 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:51.518208 27317 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:51.518769 27317 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:51.521380 27317 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.521631 26203 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:51.534072 27370 raft_consensus.cc:993] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:51.534569 27219 raft_consensus.cc:1081] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:51.536329 25949 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.538784 26050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:23:51.539373 26050 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.539734 26050 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.543390 27219 raft_consensus.cc:993] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:51.543908 27219 raft_consensus.cc:1081] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:51.546135 26051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "407574b4cda94da99be372d249d21fd5"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:23:51.546769 26051 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.547212 26051 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.549065 26050 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.550959 26050 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:51.551510 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:51.551918 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:51.552109 26127 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:51.552552 26203 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.552529 26127 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:51.553535 26127 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.553773 26051 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.554870 26127 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.556420 26051 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:51.557140 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:51.557682 26124 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:51.557976 26124 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:51.558605 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:51.558704 26124 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.559312 26201 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.559971 26124 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.565928 26201 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:23:51.566722 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:23:51.567799 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:51.568044 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:51.568612 27299 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:51.568766 27372 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:51.569168 27299 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:51.569310 27372 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:51.570073 27299 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.570560 26203 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:23:51.570104 27372 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.578666 26124 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:23:51.591339 25949 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 3 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.593232 25949 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 3 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.613883 26201 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:51.623838 26201 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:51.625000 27219 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.633402 26788 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Advancing to term 3
I20260709 12:23:51.636688 26051 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:51.640377 27317 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.647969 26788 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:51.649837 27320 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.683303 27320 raft_consensus.cc:1064] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:51.685766 27320 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:51.693184 26050 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:51.694504 26201 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:51.695116 27370 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.696799 27370 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.701597 26789 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:51.702991 27320 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.704105 27389 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.708369 26789 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.710784 26201 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.716995 26050 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.726840 25949 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:51.849390 27362 raft_consensus.cc:993] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:51.849941 27362 raft_consensus.cc:1081] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:51.852119 26127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:51.852757 26127 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.853109 26127 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.858400 26127 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.859849 26127 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:51.860929 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:51.861094 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:51.861646 26050 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.861683 26201 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:51.862149 26201 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:51.862924 26201 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.864746 26201 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.869115 26050 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:51.870154 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:51.870951 27389 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:51.871457 27389 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:51.871472 26201 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:51.872465 27389 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:51.883248 25949 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.958854 26203 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.959506 26127 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:51.960201 27258 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.960770 27372 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.977730 26050 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.979981 27258 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.989145 26203 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:51.991205 27226 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.995570 26201 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:51.997169 27389 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.024171 26050 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.028517 27320 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.080525 26127 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:52.081961 27372 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.091082 26201 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:52.095407 27372 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.311170 26127 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.316941 26203 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.319578 26789 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:52.320140 26203 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:52.322992 27370 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.322333 27320 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:52.325644 27317 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:52.336441 26050 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.338007 27389 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.341820 27370 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:52.351429 26203 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:52.349159 26789 raft_consensus.cc:2955] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:52.376372 25949 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:52.395305 26201 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:52.397173 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet adcc76aa423a40eb95d31d98a8c8776f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:52.401005 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet adcc76aa423a40eb95d31d98a8c8776f with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:52.404799 27406 tablet_replica.cc:333] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:52.407305 27406 raft_consensus.cc:2243] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:52.409286 27406 raft_consensus.cc:2272] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:52.414971 27406 ts_tablet_manager.cc:1916] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:52.425554 26127 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:52.425907 26167 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be17917578e1415f98071364583240c1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:52.427155 27302 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.430385 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.435596 26051 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:52.438153 27302 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.442373 27406 ts_tablet_manager.cc:1929] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:23:52.443063 27406 log.cc:1199] T adcc76aa423a40eb95d31d98a8c8776f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/adcc76aa423a40eb95d31d98a8c8776f
I20260709 12:23:52.446159 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet adcc76aa423a40eb95d31d98a8c8776f (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:52.451777 27362 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.454644 26127 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.455839 26051 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.486838 26129 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:52.494156 25949 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:52.497143 26093 consensus_peers.cc:597] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94c3d95514eb45178f2e5aa92e868e36. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:52.501353 26201 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:52.503607 26051 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:52.503525 27370 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.511304 27370 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.515260 27219 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.518016 26050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:52.518582 26050 raft_consensus.cc:606] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:52.517786 26201 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.530522 26051 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:52.535524 26050 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.538106 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:52.540591 26203 raft_consensus.cc:606] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:52.543411 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:52.544000 26127 raft_consensus.cc:606] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:52.546403 25950 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.552903 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:52.564664 26047 raft_consensus.cc:606] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:23:52.568560 26018 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e3998a28d0e4c4da61419183dc7ffb4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:52.568643 26203 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:52.570690 26129 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:52.572329 27297 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.573280 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:52.573851 26129 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:52.575531 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:52.576723 27372 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.577257 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:52.578053 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.599594 27297 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.604648 26203 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.626595 25951 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.630779 26127 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:52.635851 27372 raft_consensus.cc:993] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:52.636466 27372 raft_consensus.cc:1081] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:52.639322 26122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:23:52.639834 26129 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.654050 26122 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.654560 26122 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.665710 26122 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:52.669445 26093 consensus_peers.cc:597] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ddc5e875aa744c50b603a5072bc7305b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:52.670524 26050 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:52.672169 27317 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.670706 26201 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:52.679405 27370 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.684130 27370 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.688166 26050 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.694998 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:52.695640 26196 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.710706 26196 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:52.719875 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:52.720609 26047 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.720935 26047 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:52.721679 26047 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:52.722638 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:52.722959 26047 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.693287 26122 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:52.737767 26047 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:52.740916 27389 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.741554 27389 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:52.742548 27389 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:52.749440 27320 raft_consensus.cc:993] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:52.749912 27320 raft_consensus.cc:1081] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:52.751727 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38286
I20260709 12:23:52.752440 26789 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.751641 25950 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.752821 26789 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:52.758615 26201 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:52.760493 26789 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.765616 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:52.766183 26122 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:52.766458 26122 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:52.767271 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:52.767150 26122 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.767834 26196 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:52.768292 26122 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:52.775578 26122 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:23:52.776126 26789 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:52.777086 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:52.778115 27410 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:52.778600 27410 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:52.779369 27410 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.780597 26196 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:23:52.790151 25950 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.790343 25951 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 3 to 4, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.859800 27372 raft_consensus.cc:993] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:52.860325 27226 raft_consensus.cc:1081] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:52.862820 26789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38282
I20260709 12:23:52.863400 26789 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.863766 26789 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.869755 26789 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.871425 26789 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:52.872816 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:52.873432 26050 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.873032 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:52.873783 26050 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:52.874054 26196 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.874670 26050 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.875984 26050 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.881078 26196 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:52.882463 27302 raft_consensus.cc:993] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:52.882488 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:52.883168 27302 raft_consensus.cc:1081] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:52.883749 26050 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:52.884331 27410 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.884936 27410 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:52.885890 27410 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.887830 26122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:52.888417 26122 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.888753 26122 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.893875 27219 raft_consensus.cc:993] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:52.894546 27219 raft_consensus.cc:1081] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:52.896553 26052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:23:52.896302 26122 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.897142 26052 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.897517 26052 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.899537 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:23:52.900123 26789 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.899987 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:52.900655 26201 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.900956 26201 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:52.901685 26201 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.902766 26201 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.904963 26122 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:52.905804 26052 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.909602 26789 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:52.910881 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:52.911746 27370 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.912567 27370 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:52.913316 26052 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:52.913571 27370 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.919353 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:52.919932 26122 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.920802 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:23:52.920262 26122 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:52.924132 25950 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.926789 26789 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.927100 26122 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:52.928205 26122 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.925020 26201 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:52.981257 26122 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:52.984090 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:52.984907 27372 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.985400 27372 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:52.986268 27372 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:53.010615 25950 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:53.029150 25949 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:53.031776 27423 ts_tablet_manager.cc:933] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:53.032671 26789 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:53.039435 27424 ts_tablet_manager.cc:933] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:53.042138 27424 tablet_copy_client.cc:323] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:53.045758 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 94c3d95514eb45178f2e5aa92e868e36 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:53.046314 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 94c3d95514eb45178f2e5aa92e868e36 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-94c3d95514eb45178f2e5aa92e868e36
I20260709 12:23:53.047073 27423 tablet_copy_client.cc:323] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:23:53.048810 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet be17917578e1415f98071364583240c1 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:23:53.049314 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet be17917578e1415f98071364583240c1 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-be17917578e1415f98071364583240c1
I20260709 12:23:53.053025 26139 tablet_copy_source_session.cc:215] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:53.055997 27424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c3d95514eb45178f2e5aa92e868e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:53.056973 26213 tablet_copy_source_session.cc:215] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:53.061582 27423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17917578e1415f98071364583240c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:53.072542 27424 tablet_copy_client.cc:806] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:53.073122 27424 tablet_copy_client.cc:670] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:53.078441 27423 tablet_copy_client.cc:806] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:53.079061 27423 tablet_copy_client.cc:670] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:53.079980 27424 tablet_copy_client.cc:538] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:53.083060 27423 tablet_copy_client.cc:538] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:53.092170 27424 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:53.093319 27423 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:53.118588 27423 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:53.119107 27424 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:53.119549 27423 tablet_bootstrap.cc:492] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:53.120033 27424 tablet_bootstrap.cc:492] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:53.120388 27423 ts_tablet_manager.cc:1403] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.011s
I20260709 12:23:53.121559 27424 ts_tablet_manager.cc:1403] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.012s
I20260709 12:23:53.124073 27423 raft_consensus.cc:359] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.124295 27424 raft_consensus.cc:359] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.125082 27423 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:53.125176 27424 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:53.125841 27423 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.125859 27424 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.131261 27423 ts_tablet_manager.cc:1434] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:23:53.131951 27424 ts_tablet_manager.cc:1434] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:23:53.133867 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-94c3d95514eb45178f2e5aa92e868e36 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:53.134281 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-94c3d95514eb45178f2e5aa92e868e36 on tablet 94c3d95514eb45178f2e5aa92e868e36 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:53.135794 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-be17917578e1415f98071364583240c1 received from {username='slave'} at 127.0.0.1:53816
I20260709 12:23:53.136283 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-be17917578e1415f98071364583240c1 on tablet be17917578e1415f98071364583240c1 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:53.191896 27423 ts_tablet_manager.cc:933] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:53.193739 27423 tablet_copy_client.cc:323] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:53.195364 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet ddc5e875aa744c50b603a5072bc7305b from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:53.195930 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet ddc5e875aa744c50b603a5072bc7305b from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-ddc5e875aa744c50b603a5072bc7305b
I20260709 12:23:53.204972 26196 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:23:53.206939 27411 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:23:53.208289 26093 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e3998a28d0e4c4da61419183dc7ffb4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:53.208382 26139 tablet_copy_source_session.cc:215] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:53.212834 27423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddc5e875aa744c50b603a5072bc7305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:53.238427 27423 tablet_copy_client.cc:806] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:53.239064 27423 tablet_copy_client.cc:670] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:53.243968 27423 tablet_copy_client.cc:538] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:53.252532 26196 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:53.254316 27423 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:53.263826 27389 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:53.291976 26052 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:53.293498 27370 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:53.381997 26201 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.384598 27370 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:53.433461 26122 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:23:53.439636 27423 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:53.440187 27411 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:53.451962 26788 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.453490 27320 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:53.452399 27423 tablet_bootstrap.cc:492] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:53.462441 27423 ts_tablet_manager.cc:1403] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.208s	user 0.025s	sys 0.012s
I20260709 12:23:53.464547 26127 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.465675 27423 raft_consensus.cc:359] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.466488 27423 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:53.470464 27423 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.473272 27423 ts_tablet_manager.cc:1434] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:23:53.475219 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-ddc5e875aa744c50b603a5072bc7305b received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:53.475764 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-ddc5e875aa744c50b603a5072bc7305b on tablet ddc5e875aa744c50b603a5072bc7305b with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:53.480195 26201 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.485438 27421 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:53.493213 27258 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:53.636260 26788 raft_consensus.cc:1217] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:53.665495 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:53.668704 26788 raft_consensus.cc:1217] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:23:53.680351 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80819.3us
W20260709 12:23:53.756628 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47661.7us
I20260709 12:23:53.767935 26045 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.779166 26788 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.794421 27442 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:53.813704 27258 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:23:53.822094 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 125613us
I20260709 12:23:53.824832 27423 ts_tablet_manager.cc:933] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:53.827484 26790 raft_consensus.cc:1217] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:53.847810 27423 tablet_copy_client.cc:323] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:53.852272 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 9e3998a28d0e4c4da61419183dc7ffb4 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:23:53.852936 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 9e3998a28d0e4c4da61419183dc7ffb4 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-9e3998a28d0e4c4da61419183dc7ffb4
I20260709 12:23:53.864064 26139 tablet_copy_source_session.cc:215] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:53.884166 27423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3998a28d0e4c4da61419183dc7ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:53.909838 27423 tablet_copy_client.cc:806] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:53.910527 27423 tablet_copy_client.cc:670] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:53.915326 27423 tablet_copy_client.cc:538] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:53.926674 27423 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:23:53.961854 27423 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:53.962869 27423 tablet_bootstrap.cc:492] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:23:53.963553 27423 ts_tablet_manager.cc:1403] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.007s
I20260709 12:23:53.966894 27423 raft_consensus.cc:359] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.967721 27423 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:23:53.968603 27423 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:53.976572 27423 ts_tablet_manager.cc:1434] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.004s
I20260709 12:23:53.978399 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-9e3998a28d0e4c4da61419183dc7ffb4 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:23:53.978900 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-9e3998a28d0e4c4da61419183dc7ffb4 on tablet 9e3998a28d0e4c4da61419183dc7ffb4 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:54.009635 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:54.010308 26047 raft_consensus.cc:606] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:54.014261 26198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:54.014878 26198 raft_consensus.cc:606] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:54.018085 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:54.018501 26127 raft_consensus.cc:606] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:54.019644 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:54.025134 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:54.027029 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.068953 27362 raft_consensus.cc:993] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:54.069527 27362 raft_consensus.cc:1081] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:54.071954 26127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:54.073163 26127 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.073513 26127 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.079299 26127 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:54.083086 26127 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:54.083242 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:54.083889 26047 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.084631 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:54.085268 26198 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:54.085644 26198 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:54.086627 26198 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:54.088999 26198 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.099943 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:54.101272 26198 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:54.102808 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:54.103770 27448 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:54.104261 27448 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:54.105216 27448 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:54.118116 26790 raft_consensus.cc:1217] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:54.130753 25949 catalog_manager.cc:5697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.173106 27219 raft_consensus.cc:993] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:54.173604 27219 raft_consensus.cc:1081] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:54.176591 27448 raft_consensus.cc:1064] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:23:54.177031 27448 raft_consensus.cc:1068] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Service unavailable: leader transfer in progress
I20260709 12:23:54.177664 26048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:23:54.178133 26048 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.178515 26048 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.183712 26048 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:54.185973 26048 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:54.186388 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:54.186683 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:54.187029 26129 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:54.187280 26198 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.187342 26129 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:54.188354 26129 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:54.189920 26129 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.195082 26198 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:54.196594 26019 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:54.197376 27442 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:54.197906 27442 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:54.199280 26129 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:54.198818 27442 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:54.207000 27219 raft_consensus.cc:1064] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:54.209462 27219 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:54.217607 27302 raft_consensus.cc:1064] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:54.227545 25949 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.220319 27302 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:54.238211 26790 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.238863 26198 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.239759 27219 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.242375 26048 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.246865 27448 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.265525 27448 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.274956 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:54.275761 26048 raft_consensus.cc:606] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:23:54.297509 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:54.298238 26047 raft_consensus.cc:606] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:23:54.330821 26788 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:54.349884 26127 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:54.348398 27445 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:23:54.355087 27362 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.372396 27389 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.374269 26048 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.381462 26047 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:54.384106 27467 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.384475 26790 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.390532 27297 raft_consensus.cc:993] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:23:54.391077 27297 raft_consensus.cc:1081] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Signalling peer a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:23:54.400408 26198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:48534
I20260709 12:23:54.401116 26198 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.396056 26196 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.401523 26198 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:54.405005 25949 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 3 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:54.413699 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:54.414340 26047 raft_consensus.cc:606] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:23:54.420140 26000 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:23:54.420539 26000 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for ddc5e875aa744c50b603a5072bc7305b already in progress
W20260709 12:23:54.420871 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for ddc5e875aa744c50b603a5072bc7305b already in progress
I20260709 12:23:54.430675 26198 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:54.432940 26198 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:54.434089 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:54.434691 26047 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:54.434983 26047 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:54.435534 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:54.436177 26127 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:54.435721 26047 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:54.437286 26047 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:54.453568 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:23:54.454870 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:54.455713 27445 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:54.456447 27445 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:54.457520 27445 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:54.455621 26047 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:23:54.459748 27464 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.474872 26127 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:54.480367 26129 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.463521 26789 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.495527 26048 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.500061 27219 raft_consensus.cc:1064] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:54.501647 26196 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.502331 26790 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.502609 27219 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:54.506623 27389 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:54.518329 27448 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.532226 26201 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:54.534510 27320 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:54.537046 27370 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:54.539383 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 94c3d95514eb45178f2e5aa92e868e36 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:54.544842 26196 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.539542 26790 raft_consensus.cc:2955] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.577647 25949 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 2 to 3, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.584483 25951 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 3 to 4, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 4 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:54.629768 26789 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.631474 27447 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.634732 26048 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.637322 26203 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.640445 27219 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.645654 26047 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:54.645517 27219 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.652443 25950 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 4 to 5, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.653367 27389 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:54.664474 26203 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:54.672511 26790 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:54.677541 27467 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.682124 27219 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.697021 26203 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.704340 26048 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:54.708074 27445 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:54.716415 26051 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.716223 27467 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.727077 26790 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.734290 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet 94c3d95514eb45178f2e5aa92e868e36 with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:54.738369 25952 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index 2 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.743824 26129 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:54.746290 26788 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Advancing to term 2
I20260709 12:23:54.747339 27482 tablet_replica.cc:333] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:54.744725 26048 raft_consensus.cc:2955] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.750301 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be17917578e1415f98071364583240c1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:54.754349 27482 raft_consensus.cc:2243] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:54.755187 27482 raft_consensus.cc:2272] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:54.755065 26790 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.760023 27482 ts_tablet_manager.cc:1916] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:54.762079 26788 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:54.763289 27372 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.765846 27372 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:54.791657 26198 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:54.791425 25949 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 4, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.800379 27372 raft_consensus.cc:993] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:54.800846 27372 raft_consensus.cc:1081] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:54.807363 26127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:23:54.808193 27442 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.809480 26127 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.809928 26127 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.817328 26127 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:54.870122 26047 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:54.877525 26127 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:54.878564 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:54.888317 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:54.888923 26047 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:54.889181 26047 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:54.889904 26047 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:54.897747 26047 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.917943 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet be17917578e1415f98071364583240c1 with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56074
I20260709 12:23:54.919872 27467 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:54.929910 27482 ts_tablet_manager.cc:1929] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:54.930488 27482 log.cc:1199] T 94c3d95514eb45178f2e5aa92e868e36 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/94c3d95514eb45178f2e5aa92e868e36
I20260709 12:23:54.932435 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet 94c3d95514eb45178f2e5aa92e868e36 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:54.937953 27489 tablet_replica.cc:333] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:23:54.938453 26121 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:54.939160 27489 raft_consensus.cc:2243] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:54.940028 27489 raft_consensus.cc:2272] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:54.941838 26047 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:23:54.944959 27489 ts_tablet_manager.cc:1916] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:54.946818 26790 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.948812 27219 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:54.955296 26047 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.955996 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:54.959504 26789 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.962250 27219 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:54.959558 27320 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:54.964953 27320 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:54.965859 27320 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:54.967832 27489 ts_tablet_manager.cc:1929] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:54.968595 27489 log.cc:1199] T be17917578e1415f98071364583240c1 P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/be17917578e1415f98071364583240c1
I20260709 12:23:54.972815 26047 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.957506 27389 raft_consensus.cc:2955] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:54.989809 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet be17917578e1415f98071364583240c1 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:55.005370 26127 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:55.013284 27465 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:55.003548 25950 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 4 to 5, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.030530 25949 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.034178 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 9e3998a28d0e4c4da61419183dc7ffb4 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48518
I20260709 12:23:55.066720 27494 tablet_replica.cc:333] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:23:55.067337 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9e3998a28d0e4c4da61419183dc7ffb4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:55.072769 27494 raft_consensus.cc:2243] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:55.073484 27494 raft_consensus.cc:2272] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:55.075699 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.077255 27494 ts_tablet_manager.cc:1916] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:55.096364 27494 ts_tablet_manager.cc:1929] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:55.096917 27494 log.cc:1199] T 9e3998a28d0e4c4da61419183dc7ffb4 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/9e3998a28d0e4c4da61419183dc7ffb4
I20260709 12:23:55.099110 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 9e3998a28d0e4c4da61419183dc7ffb4 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:55.123600 26198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:55.124228 26198 raft_consensus.cc:606] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:55.126744 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:55.127355 26127 raft_consensus.cc:606] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:55.129940 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:55.130396 26127 raft_consensus.cc:606] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:55.132684 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:55.133210 26127 raft_consensus.cc:606] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:55.134317 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:55.136173 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:55.136914 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.358465 26203 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.360711 26789 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Advancing to term 3
I20260709 12:23:55.367846 26789 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:55.368650 26203 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:55.369400 27219 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:55.371387 27219 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:55.377458 27219 raft_consensus.cc:1064] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:23:55.377871 27219 raft_consensus.cc:1068] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:23:55.385948 26048 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:55.387835 27492 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:55.389302 27492 raft_consensus.cc:1064] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:23:55.396772 27488 raft_consensus.cc:993] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:55.397338 27488 raft_consensus.cc:1081] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:55.402103 26129 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:23:55.407052 26129 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.407487 26129 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.391583 27492 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.419428 26788 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:55.421262 26129 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:55.423038 26203 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:55.425208 26129 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:55.426067 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:55.426736 26048 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.428561 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:55.429157 26203 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:55.429445 26203 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:55.430160 26203 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:55.431109 27447 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:55.431727 26203 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.435359 26048 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:55.436167 27219 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:55.441366 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:55.437201 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:55.442585 27320 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:55.443109 27320 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:55.444868 26048 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:55.447266 27448 raft_consensus.cc:993] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:23:55.447361 27447 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:55.445036 27320 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:55.447890 27448 raft_consensus.cc:1081] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Signalling peer a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:23:55.457118 26196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:48540
I20260709 12:23:55.457682 26196 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.458029 26196 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.458552 27448 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.466956 26198 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.473187 26048 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.469485 26196 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:55.481050 25950 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.487383 26196 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:55.489494 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:55.490171 26051 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.492166 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:55.492789 26129 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:55.493062 26129 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:55.493759 26129 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:55.500542 26790 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.500602 26129 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.510143 27447 raft_consensus.cc:993] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:55.510825 27370 raft_consensus.cc:1081] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:23:55.513451 26788 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38286
I20260709 12:23:55.514029 26788 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.514371 26788 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.516806 25950 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } attrs { replace: true } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.522982 26051 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:23:55.524348 26788 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.528004 26788 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:55.530306 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:23:55.530938 26198 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.531435 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:55.533044 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:55.533336 27465 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:55.533696 26122 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:55.534006 26122 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:55.534147 27465 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:23:55.534837 26122 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.535073 27465 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:55.536196 26122 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.538735 26198 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:23:55.540103 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:55.541352 27491 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:55.541880 27491 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:23:55.542761 27491 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.540192 26129 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:23:55.550676 26122 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:23:55.558353 25952 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.561100 25951 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 3 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.564363 26122 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.570674 27219 raft_consensus.cc:993] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:55.571158 27219 raft_consensus.cc:1081] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:23:55.571715 26788 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:55.573354 27447 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:55.575096 26048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:23:55.575742 26048 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.576364 26048 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.578219 26198 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:55.579607 27447 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:55.583312 27219 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.589582 26788 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.594836 26048 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.598078 26048 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:55.598773 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:23:55.599043 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:23:55.599469 26122 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:55.599669 26790 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.599819 26122 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:55.600109 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ddc5e875aa744c50b603a5072bc7305b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:55.600893 26122 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.601954 26122 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.598016 26198 raft_consensus.cc:2955] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:23:55.609135 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:55.610373 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:23:55.611135 27372 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:55.611583 27372 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:55.612480 27372 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:55.621381 26122 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:55.624053 25951 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index 5 to 6, VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) evicted. New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.627969 25949 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.636529 26042 tablet_service.cc:1558] Processing DeleteTablet for tablet ddc5e875aa744c50b603a5072bc7305b with delete_type TABLET_DATA_TOMBSTONED (TS 8fd5643e7b494fa79c3d0aaf1e6e8152 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47790
I20260709 12:23:55.639472 27500 tablet_replica.cc:333] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:23:55.640780 27500 raft_consensus.cc:2243] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:55.641472 27500 raft_consensus.cc:2272] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:55.644479 27500 ts_tablet_manager.cc:1916] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:55.656610 27500 ts_tablet_manager.cc:1929] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:23:55.656967 27500 log.cc:1199] T ddc5e875aa744c50b603a5072bc7305b P 8fd5643e7b494fa79c3d0aaf1e6e8152: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/wals/ddc5e875aa744c50b603a5072bc7305b
I20260709 12:23:55.658473 25938 catalog_manager.cc:5028] TS 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): tablet ddc5e875aa744c50b603a5072bc7305b (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:23:55.873344 26198 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:55.874891 27447 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:55.884724 26048 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:55.894304 27447 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:55.921968 26048 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:55.923311 27445 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:55.938658 26122 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:55.941478 27505 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:56.028395 26122 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:56.029776 27491 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:56.042290 26198 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:56.043880 27410 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:56.056963 26122 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.063346 27442 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.081878 26790 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.083457 27372 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.145972 26122 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:56.154348 26051 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:56.154382 26093 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ddbca55e34c47f2b2d10e46e4779bd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:56.156018 26198 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:56.155931 27219 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.157467 27448 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:56.163852 27219 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.165556 26048 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.167423 26198 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.174759 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.183115 26051 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:56.184873 25950 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:56.191737 26198 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:56.201812 27372 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.203747 26122 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:56.204366 26018 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 93ec87f552a04ce49312a2ef65f862ce. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:56.205425 27372 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.212668 27372 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.217834 26198 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.227962 26122 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.248793 26048 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:23:56.264158 26018 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69c5f2ff7a584f80b9a347f6adb3c2f8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:56.265285 25950 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 3, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.266251 26051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49f4a1a801de461182f8315a884b9889"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:23:56.266861 26051 raft_consensus.cc:606] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:23:56.268548 26129 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:56.271240 27372 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:56.271842 26198 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:56.274286 27442 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:56.278128 26122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:23:56.280592 26122 raft_consensus.cc:606] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:23:56.281196 27372 raft_consensus.cc:993] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:56.282665 27372 raft_consensus.cc:1081] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:23:56.284042 26196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:23:56.284456 26122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49f4a1a801de461182f8315a884b9889"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:23:56.284632 26196 raft_consensus.cc:606] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:23:56.285145 26122 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:56.285560 26122 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.286514 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:56.288169 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:56.288911 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.291661 26203 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.293767 26122 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:56.300411 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:23:56.301018 26788 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.301169 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:23:56.301811 26048 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:56.302078 26048 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:23:56.302788 26048 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:56.304252 26048 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.297991 27372 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.312827 26122 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:56.316597 26788 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:56.317775 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:56.318630 27448 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:56.319149 27448 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:23:56.320043 27448 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:56.319294 26129 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.344400 25952 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:56.336643 26048 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:56.352017 26122 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:56.357867 25949 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:56.375620 26093 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f720b2e4986f496694c7578beef67cd4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:56.377234 26048 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:56.379212 26196 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:56.379050 27447 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:56.381822 27447 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:56.405014 27447 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:56.424136 26203 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:23:57.358310 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:57.683435 27531 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:57.684152 27531 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:57.684983 27530 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:57.685729 27530 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:57.686368 27531 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:57.688580 27530 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:56.421299 26048 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:23:58.057904 26203 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.630s	user 0.019s	sys 0.000s
I20260709 12:23:58.067147 27536 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.069810 27533 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.071772 27533 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.074249 27533 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.067931 27535 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.077142 27535 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.077728 27536 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.087808 27535 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.089411 27535 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.098464 27535 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.104394 27539 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.105345 27539 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.108669 27539 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.100330 27534 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.100224 27541 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.094180 27536 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.099067 27543 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.120604 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 848) took 1708 ms. Trace:
I20260709 12:23:58.125173 27534 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.120893 27544 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.134502 27535 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.132867 27533 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.127190 26203 rpcz_store.cc:276] 0709 12:23:56.406306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.423404 (+ 17098us) service_pool.cc:224] Handling call
0709 12:23:56.423779 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.423988 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:58.058641 (+1634653us) log.cc:844] Serialized 14 byte log entry
0709 12:23:58.113788 (+ 55147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:58.113857 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:58.113916 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.114005 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:58.114167 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18953,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":54645,"threads_started":2,"wal-append.queue_time_us":33088}
I20260709 12:23:58.135725 27533 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.137992 27533 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.138484 27533 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.125730 27539 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.129267 27541 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.143196 27541 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.128822 27543 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.146468 27543 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.177619 27550 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.139621 27531 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.180490 27552 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.193169 27553 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.193843 27553 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.196576 27536 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.197341 27536 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.205066 27554 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.205904 27554 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.211769 27536 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.212806 27536 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.213549 27536 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.217864 27554 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.220170 27554 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.220954 27554 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.156723 27543 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.152853 27539 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.153904 27533 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.223869 27554 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.139451 27544 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.228880 27562 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.226516 27549 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.251837 27548 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.228762 27552 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:58.234660 27540 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.252210 27546 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.257289 27563 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.258832 27539 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.236485 27534 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.298799 27553 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.303244 27534 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.230746 27531 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.242370 27558 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.234402 27543 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.226322 27550 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.301543 27546 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.313946 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 874) took 1900 ms. Trace:
I20260709 12:23:58.326128 26048 rpcz_store.cc:276] 0709 12:23:56.413824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.420533 (+  6709us) service_pool.cc:224] Handling call
0709 12:23:56.420918 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.421140 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:23:58.092707 (+1671567us) log.cc:844] Serialized 14 byte log entry
0709 12:23:58.313490 (+220783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:58.313553 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:58.313604 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.313686 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:58.313839 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46607,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":13261,"thread_start_us":220036,"threads_started":2,"wal-append.queue_time_us":37076}
I20260709 12:23:58.348377 27568 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.305132 27563 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.331666 27556 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.331866 27541 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.268486 27562 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.311817 27534 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.280184 27549 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.366796 27551 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.309376 27548 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.352252 27532 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.381913 27532 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.371992 27536 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.387321 27559 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.387868 27549 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.300419 27540 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.389348 27539 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.409092 27553 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:23:58.409695 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.000s
W20260709 12:23:58.410035 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.005s	sys 0.000s
I20260709 12:23:58.376890 27551 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.374627 27541 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.446125 27566 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.447736 27546 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.457552 27575 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.459323 27567 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.407684 27554 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.317117 27558 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.468789 27549 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.469028 27565 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.393458 27574 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.473263 27538 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.473755 27561 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.473774 27571 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.371088 27568 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.409695 27553 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.465454 27535 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.429909 27539 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.488760 27560 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.470024 27533 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.478464 27546 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.489454 27560 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.458372 27575 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.497892 27560 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.498512 27575 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.379781 27556 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.481238 27536 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.483908 27543 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.460047 27567 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.472453 27554 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.490577 27545 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.477622 27549 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.494886 27546 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.508538 27544 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.480110 27566 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.505070 27552 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.512724 27531 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.442613 27559 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.520752 27530 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.506671 27535 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.516573 27536 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.521724 27584 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.525349 27542 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.493435 27574 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.530138 27532 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.492934 27538 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.542419 27564 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.546064 27559 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.546875 27559 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.535152 27534 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.534582 27572 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.534893 27551 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.488348 27565 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.547592 27559 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.527091 27545 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.560400 27538 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.560868 27545 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.539647 27557 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.560938 27559 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.541436 27531 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.542928 27553 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.543946 27533 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.543339 27548 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.507865 27571 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.499461 27561 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.553362 27563 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.538509 27581 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.565833 27585 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.540390 27530 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.550372 27542 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.561795 27569 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.557548 27586 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.567929 27550 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.573571 27588 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.563956 27532 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.568749 27541 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.568629 27551 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.579917 27545 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.569268 27533 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.567013 27531 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.604607 27531 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.551844 27584 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.591888 27570 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.588848 27546 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.555212 27564 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.589412 27562 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.604384 27565 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.585585 27585 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.599277 27576 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.596179 27540 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.601871 27539 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.561861 27557 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.585875 27591 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.605813 27590 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.613206 27587 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.596015 27532 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.564719 27572 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.605904 27570 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.599710 27586 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.611940 27558 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.591326 27581 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.646703 27567 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.679337 27542 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.607293 27580 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:23:58.597222 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002a1d80 after lost signal to thread 24865
I20260709 12:23:58.611025 27546 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.628340 27582 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.728173 27574 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.637423 27578 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.725303 27554 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.652876 27535 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.607229 27545 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.727209 27549 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.657150 27584 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.652591 27566 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.602309 27551 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.608561 27592 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.622046 27556 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.685374 27531 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:58.727854 27571 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.842528 27531 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.761080 27533 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.688288 27585 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:58.810225 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002cba80 after lost signal to thread 25986
I20260709 12:23:58.609377 27569 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.634276 27589 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.780412 27580 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.810997 27578 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:58.767401 27536 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.818848 27530 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:23:58.851516 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002cbb40 after lost signal to thread 26000
I20260709 12:23:58.817939 27546 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.606889 27588 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:58.760758 27587 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.856714 27539 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:58.837230 27592 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:58.806253 27591 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:58.877079 26812 debug-util.cc:398] Leaking SignalData structure 0x7b0800332c40 after lost signal to thread 26078
I20260709 12:23:58.760926 27590 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.883601 25949 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New cstate: current_term: 1 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.875675 27539 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.876178 27568 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.889824 27569 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.853298 27589 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:58.898553 27557 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.899874 27572 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.728543 27576 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:23:58.895434 26812 debug-util.cc:398] Leaking SignalData structure 0x7b0800259a20 after lost signal to thread 26153
I20260709 12:23:58.906579 27580 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.890697 27532 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.910534 27564 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.908699 27539 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:23:58.928469 26812 debug-util.cc:398] Leaking SignalData structure 0x7b0800342a80 after lost signal to thread 26227
I20260709 12:23:58.930397 27545 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:58.940310 26812 debug-util.cc:398] Leaking SignalData structure 0x7b0800331ca0 after lost signal to thread 26228
I20260709 12:23:58.925882 27587 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.854950 27582 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:58.931089 27531 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:58.965554 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:23:58.966785 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.021s	sys 0.022s
I20260709 12:23:58.930903 27581 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.930138 27590 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.949985 27591 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.935729 27561 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.926474 27570 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:23:58.967413 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.021s	sys 0.022s
I20260709 12:23:58.937691 27551 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.972434 26788 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:58.936261 27592 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.930848 27586 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:23:58.970696 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf380 after lost signal to thread 26815
I20260709 12:23:58.967624 27546 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.970288 27545 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.970924 27564 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.986084 27588 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:58.983669 27545 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.995632 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:58.996383 26790 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:58.981472 27581 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:58.997881 27545 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:58.998644 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:58.999276 26789 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:59.004386 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:58.998075 27581 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:58.984508 27570 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:59.005218 26788 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
W20260709 12:23:59.012373 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.017s	sys 0.026s
W20260709 12:23:59.015300 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.017s	sys 0.026s
I20260709 12:23:59.015656 27601 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:59.016494 27601 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.019450 27601 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:23:59.020412 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.001s	sys 0.026s
W20260709 12:23:59.020864 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.001s	sys 0.026s
I20260709 12:23:59.005220 27592 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.025790 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.026404 26129 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:23:59.025269 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.005358 27546 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:59.029807 27591 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:58.998771 27578 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.008634 27590 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.037894 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:58.996397 27564 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:59.010320 27539 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.041214 27591 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.039625 27546 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.050853 27591 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.051424 27546 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.052446 27539 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.056320 27604 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.029668 27551 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:59.055397 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.023626 27588 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.069473 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:59.012909 27570 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.064522 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.033509 27601 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.022836 27592 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.075220 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.080730 26127 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
W20260709 12:23:59.081700 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.007s	sys 0.024s
I20260709 12:23:59.041430 27599 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:23:59.082661 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.008s	sys 0.024s
I20260709 12:23:59.082167 26047 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:23:59.076342 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.083535 27589 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.044108 27590 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.021858 27579 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.084594 27588 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.085507 27597 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.032941 27596 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:59.086231 27570 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.056923 27604 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.064491 27531 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:23:59.088658 27532 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:59.132483 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.141126 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.084123 27582 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.081966 27576 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.097365 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.377082 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.350147 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.392127 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.167296 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.393003 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.393184 26048 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:23:59.317117 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.393421 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.351548 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.352046 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.163580 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.426131 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.425120 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.225229 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.158875 27539 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.448982 26122 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:23:59.205087 27588 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.450511 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.452699 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.255751 27589 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:59.099730 27605 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.212714 27581 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.082948 27551 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.265805 27567 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.161458 27579 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.385830 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.397646 27564 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.201828 27590 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.458695 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.133266 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.201961 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.397838 27592 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.454380 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.481175 26788 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:59.452217 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.363373 27602 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.100184 27578 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.133589 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.490329 26786 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:59.490437 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.491660 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.273644 27562 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.099994 27545 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:23:59.201620 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.460645 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.163228 27591 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:59.221468 27570 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.403134 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.462044 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.462015 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.460698 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.133795 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.351698 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.089960 27601 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.517901 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.470690 26199 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:59.470808 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.520932 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.470899 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.523556 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.521397 26050 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:23:59.487910 26045 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:23:59.163383 27597 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.481626 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.482695 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.225256 27606 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:59.541785 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.543742 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.403679 27532 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:23:59.163084 27603 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.224982 27600 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.213321 27596 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.489022 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.459839 27589 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.512421 26203 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:59.465546 27605 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.510474 27602 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.103616 27599 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.518599 26198 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:59.522310 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.576257 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.672763 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.520488 26194 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:59.492347 26783 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:59.096696 27585 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:23:59.468577 27567 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:23:59.528148 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.532716 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.537062 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.513872 27591 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.517964 27570 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.626451 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.702484 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.495667 27578 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.570860 27608 ts_tablet_manager.cc:933] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:23:59.506794 27562 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.508680 27545 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.525846 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.673084 26129 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:59.673115 26200 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:23:59.517022 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.608297 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.673451 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.646509 27579 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.709551 26198 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:59.710479 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.711426 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.711867 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:59.675078 27604 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.642275 27601 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.560496 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.682943 27611 ts_tablet_manager.cc:933] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.683497 27551 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.683828 26047 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:23:59.684696 26201 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:23:59.566068 27532 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.607070 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.719792 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.720887 26049 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:23:59.690200 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:59.700842 27585 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.722158 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:59.691330 27602 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.693779 27605 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.694303 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.695331 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.618353 27603 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.698859 26122 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:59.698922 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.699470 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.698858 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.642246 27597 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.705209 27567 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:23:59.706130 27578 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.733109 26121 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:23:59.706228 27562 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.706830 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:23:59.707439 27545 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.710670 27570 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.718238 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.720033 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.688783 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.732671 27582 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:59.724861 26050 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:59.686892 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.733130 26202 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:23:59.735427 26093 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:59.732427 27551 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:59.739928 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.739017 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.741943 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.700723 27600 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.745618 27596 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.747221 26048 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:23:59.619243 27606 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:23:59.750798 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.754753 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.758576 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.759642 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.760980 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.761137 26203 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:23:59.761586 26200 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:23:59.752399 27582 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:59.765920 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.756485 27551 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:59.767416 26050 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:23:59.769523 27582 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.769840 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.759505 27599 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.761154 27608 tablet_copy_client.cc:323] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:23:59.771377 27585 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.791780 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.794267 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.793164 27606 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:23:59.834192 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82945.6us
I20260709 12:23:59.794219 27603 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.775369 27551 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.850421 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.851154 26125 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:59.856278 27611 tablet_copy_client.cc:323] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:23:59.776782 27531 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:59.867642 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.868630 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.792146 27600 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.870247 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.870096 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.870182 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.874261 27531 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:23:59.878938 27531 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.878995 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.881829 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.915289 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:23:59.934286 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 96393.1us
I20260709 12:23:59.942481 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:23:59.944347 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.944504 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:23:59.947132 26757 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.588s (SENT) [suppressed 1 similar messages]
I20260709 12:23:59.948504 27532 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.952519 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.954216 27582 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:23:59.964366 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:23:59.964260 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:23:59.980055 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.985728 26124 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:59.961272 27589 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.963660 26120 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:59.965814 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.989149 27603 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.009862 27603 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.010344 27603 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.010792 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:23:59.980341 27599 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:59.991140 26121 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.012452 27599 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:00.012195 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.012892 27599 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.013443 26194 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:00.017313 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.019104 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.992267 27582 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.992949 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.962201 27551 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:00.022877 27551 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:23:59.994648 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:23:59.996114 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.023855 26125 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:23:59.996484 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.024385 26128 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.024847 26126 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:24:00.026337 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.028663 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.997000 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:23:59.998756 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.030740 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.031505 27602 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.031605 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.031910 27602 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:00.032393 27602 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.033982 27609 ts_tablet_manager.cc:933] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:23:59.966259 27616 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.001784 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.003707 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.004807 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:23:59.966073 26197 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:00.033291 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.034869 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.035054 26789 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:24:00.044335 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.043247 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.038264 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.047724 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.048429 27597 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.059085 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.075299 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.050380 27570 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.081712 27570 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.082230 27570 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.079847 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.068631 26129 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:00.041761 27616 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:00.074185 27578 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.085570 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:24:00.090600 27578 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:00.090828 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 69c5f2ff7a584f80b9a347f6adb3c2f8 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-69c5f2ff7a584f80b9a347f6adb3c2f8
I20260709 12:24:00.091121 27578 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.099375 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.099895 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:00.045318 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 98325.4us
I20260709 12:24:00.100601 26050 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:00.101605 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.043035 27567 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.104596 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.104697 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.101137 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.088019 27604 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.078575 27605 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.108807 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.076426 27597 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.111362 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.111384 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.113160 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.113509 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:00.050809 27545 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.105970 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.120666 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.128412 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:00.130187 27618 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.092721 27616 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.108845 27596 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.132258 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.126865 26126 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:24:00.133739 26063 tablet_copy_source_session.cc:215] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:00.134202 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.134279 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:00.108070 27602 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:00.134948 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2051) took 1003 ms. Trace:
I20260709 12:24:00.118036 27604 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:00.137283 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.118356 27605 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.137900 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.138872 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.148562 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:00.167457 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.121368 27597 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.115211 27567 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.173959 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.170238 27551 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.162391 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.176677 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.176726 27567 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.177896 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:00.178239 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.158281 26047 rpcz_store.cc:276] 0709 12:23:59.131986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.132158 (+   172us) service_pool.cc:224] Handling call
0709 12:24:00.134770 (+1002612us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:00.134888 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672671104}
I20260709 12:24:00.164326 27602 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:00.165177 27605 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.166381 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.163892 27591 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:00.154440 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.195483 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.196812 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.126744 27545 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.168610 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.171687 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 164) took 1064 ms. Trace:
I20260709 12:24:00.171765 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.196841 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.172184 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.198076 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.174207 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.173218 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2039) took 1169 ms. Trace:
I20260709 12:24:00.176676 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.143263 27596 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.179505 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.164772 27604 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.186476 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.185086 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.140105 27603 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:00.192806 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.172196 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:00.157725 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116018us
I20260709 12:24:00.219023 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.202416 26051 rpcz_store.cc:276] 0709 12:23:59.003925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.004085 (+   160us) service_pool.cc:224] Handling call
0709 12:23:59.004367 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.700513 (+696146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.700545 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:59.700585 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:59.700613 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.172975 (+472362us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400229554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:00.173030 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.173148 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600533248}
W20260709 12:24:00.245025 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 96564.8us
I20260709 12:24:00.248829 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:00.198206 26195 rpcz_store.cc:276] 0709 12:23:59.106244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.131404 (+ 25160us) service_pool.cc:224] Handling call
0709 12:23:59.161536 (+ 30132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.722379 (+560843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.722410 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:59.722454 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:59.722484 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.170266 (+447782us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:00.170307 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.170419 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289367424}
I20260709 12:24:00.248639 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.268757 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.270290 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.271347 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.271922 26047 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:00.272648 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.272840 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.249614 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.276515 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.327883 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:00.324481 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 224642us
I20260709 12:24:00.336982 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.338757 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.341020 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.275563 27582 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.224220 27596 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.218111 27545 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.221916 27601 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.354354 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.347815 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.350128 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.273327 27609 tablet_copy_client.cc:323] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.130:40611
I20260709 12:24:00.358412 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.359802 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.321305 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.362565 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:00.370234 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.371644 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.351737 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.355150 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.375523 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.376190 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.376539 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.376631 26050 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:00.376756 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.377036 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.377858 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.347453 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.379726 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.380429 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.220508 27578 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:00.372723 27610 ts_tablet_manager.cc:933] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
I20260709 12:24:00.381253 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:00.382066 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 136645us
I20260709 12:24:00.390606 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.390621 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.383622 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:00.393014 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:00.384449 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47598.1us
I20260709 12:24:00.354396 27601 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:00.403107 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.397292 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.401662 26047 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:00.376339 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.412496 26048 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:00.406137 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 206431us
I20260709 12:24:00.407225 26202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:00.408736 27627 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.377882 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.418561 26202 raft_consensus.cc:606] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:00.418658 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:00.396906 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.426819 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:00.430490 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 185512us
I20260709 12:24:00.432829 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.423296 25949 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 290) took 1526 ms. Trace:
I20260709 12:24:00.442168 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:00.447461 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44941.8us
I20260709 12:24:00.452512 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.443416 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.410744 27601 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.469645 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.468745 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.397320 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:24:00.505577 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.445492 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:00.507179 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.504215 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:00.509909 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 89128.6us
W20260709 12:24:00.469713 27531 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.004s
I20260709 12:24:00.437407 27627 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.524789 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.511770 27629 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.532922 27614 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.421950 27532 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:00.537956 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.536420 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.535332 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.470747 27598 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.510833 27628 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.540045 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:00.584172 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44227.2us
I20260709 12:24:00.586179 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:00.542258 27618 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.436360 25949 rpcz_store.cc:276] 0709 12:23:58.876964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.877140 (+   176us) service_pool.cc:224] Handling call
0709 12:24:00.403415 (+1526275us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:59.342185 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:59.389811 (+ 47626us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:59.390315 (+   504us) write_op.cc:435] Acquired schema lock
0709 12:23:59.392292 (+  1977us) tablet.cc:662] Decoding operations
0709 12:23:59.464754 (+ 72462us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:59.467799 (+  3045us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:59.468438 (+   639us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:59.488072 (+ 19634us) tablet.cc:701] Row locks acquired
0709 12:23:59.488097 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:23:59.509128 (+ 21031us) write_op.cc:270] Start()
0709 12:23:59.515523 (+  6395us) write_op.cc:276] Timestamp: P: 1783599839507520 usec, L: 0
0709 12:23:59.515593 (+    70us) op_driver.cc:348] REPLICATION: starting
0709 12:23:59.675648 (+160055us) log.cc:844] Serialized 599 byte log entry
0709 12:24:00.084821 (+409173us) op_driver.cc:464] REPLICATION: finished
0709 12:24:00.146932 (+ 62111us) write_op.cc:301] APPLY: starting
0709 12:24:00.147056 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:00.147219 (+   163us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:00.147386 (+   167us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:00.147754 (+   368us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:00.147972 (+   218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:00.148027 (+    55us) write_op.cc:312] APPLY: finished
0709 12:24:00.148369 (+   342us) log.cc:844] Serialized 23 byte log entry
0709 12:24:00.148595 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:00.148749 (+   154us) write_op.cc:454] Released schema lock
0709 12:24:00.148812 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:00.149313 (+   501us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:00.149355 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":61779,"apply.run_cpu_time_us":2382,"apply.run_wall_time_us":2382,"num_ops":1,"prepare.queue_time_us":121491,"prepare.run_cpu_time_us":15669,"prepare.run_wall_time_us":449175,"raft.queue_time_us":204882,"raft.run_cpu_time_us":8190,"raft.run_wall_time_us":54871,"replication_time_us":564597,"spinlock_wait_cycles":6016,"thread_start_us":227258,"threads_started":4,"wal-append.queue_time_us":168831}]]}
I20260709 12:24:00.523002 27602 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:00.530313 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 63620.5us
W20260709 12:24:00.536500 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 94398.7us
I20260709 12:24:00.551479 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet 8ddbca55e34c47f2b2d10e46e4779bd0 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-8ddbca55e34c47f2b2d10e46e4779bd0
I20260709 12:24:00.555728 26126 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:00.554101 27629 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.580341 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.581327 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.522866 27634 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.592481 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.609429 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:00.595793 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 194971us
I20260709 12:24:00.596181 27644 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.597980 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.551544 27627 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.555074 27603 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:00.559201 27614 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.551609 27531 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:00.558357 27570 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:00.590476 27588 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:00.647521 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 195317us
I20260709 12:24:00.593091 27589 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.590577 27636 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.593191 27602 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:00.616532 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 197940us
I20260709 12:24:00.639094 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:00.612624 27598 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.638936 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:00.721403 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 82539.7us
I20260709 12:24:00.583254 27628 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.682767 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:00.655603 27614 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.607203 27629 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.611541 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.614277 27634 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.714314 26048 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.714732 27600 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.714919 26127 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.741348 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:00.742964 26791 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.715749 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:00.718884 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:00.706229 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 67254.4us
I20260709 12:24:00.726245 26124 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.770201 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:00.771435 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:00.726384 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43738.8us
I20260709 12:24:00.693857 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.729437 27582 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.677431 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:00.641530 27585 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.730346 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.735069 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:00.692534 27588 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:00.683652 27610 tablet_copy_client.cc:323] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.129:44989
I20260709 12:24:00.737116 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:00.695894 27589 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:00.686861 27622 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:00.712944 27602 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.729693 27628 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.688512 27578 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:00.773948 27637 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.774863 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:00.779464 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49200.3us
I20260709 12:24:00.647379 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:00.791342 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 246582us
I20260709 12:24:00.805188 27618 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:00.811964 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 118182us
I20260709 12:24:00.881230 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.882464 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.739509 27638 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.812633 27589 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:00.847723 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 125314us
I20260709 12:24:00.774246 27640 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:00.778488 27562 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:24:00.812928 27622 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:00.805918 27588 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.782269 27585 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:00.745693 27634 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:00.876582 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70266.3us
I20260709 12:24:00.850952 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.845582 27600 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:00.847450 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 886) took 1174 ms. Trace:
I20260709 12:24:00.849975 27637 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:00.898104 27596 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.001s
I20260709 12:24:00.898756 27585 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.896669 27633 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:00.911736 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 192895us
I20260709 12:24:00.891070 27622 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.889878 27579 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:00.923538 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50444us
W20260709 12:24:00.924904 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74027.2us
I20260709 12:24:00.931639 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.884402 27638 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:00.927299 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.886335 27640 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:00.913578 27600 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.925252 26139 tablet_copy_source_session.cc:215] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:24:00.926713 27599 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.913s	user 0.008s	sys 0.005s
I20260709 12:24:00.916488 27637 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.938657 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.924817 27644 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.941522 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:00.950687 27616 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.796s	user 0.009s	sys 0.000s
I20260709 12:24:00.929112 27633 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:00.933108 27648 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.950809 26791 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:00.940138 27618 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:00.954032 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.940284 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.944099 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:00.929068 27643 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.984311 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:00.940376 27638 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.902079 27647 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.980154 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:00.953405 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.952055 27599 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:00.929821 27596 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:00.991171 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50985.5us
I20260709 12:24:00.981446 27633 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.939359 27640 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.952629 27531 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:00.913792 26045 rpcz_store.cc:276] 0709 12:23:59.672913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.328334 (+655421us) service_pool.cc:224] Handling call
0709 12:24:00.715529 (+387195us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:00.845073 (+129544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3598080}
I20260709 12:24:01.000140 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:01.000882 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56237.5us
W20260709 12:24:00.989392 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49336.6us
I20260709 12:24:01.009349 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:01.001721 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 111929us
I20260709 12:24:00.968267 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:01.003223 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.010553 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.011024 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.011364 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.012156 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.012423 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.012850 27659 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:01.013095 27636 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.933207 27645 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.013643 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.004415 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:01.014053 27645 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:01.014518 27645 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.993229 27532 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.989480 27639 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.017012 26122 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:01.016654 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.985518 27570 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:00.988531 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:01.026104 27659 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.026600 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.014776 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:00.967926 27652 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.018890 26047 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:01.005044 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51715.3us
I20260709 12:24:01.019376 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:00.981071 27616 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.041589 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:01.035321 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.005828 27617 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:01.040563 25952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 282) took 1950 ms. Trace:
I20260709 12:24:01.042686 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.045394 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.008500 27644 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.052850 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.056216 26052 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:01.049657 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.049850 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.043816 25952 rpcz_store.cc:276] 0709 12:23:59.090506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.090662 (+   156us) service_pool.cc:224] Handling call
0709 12:24:01.040487 (+1949825us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152603,"reported_tablets":2}
I20260709 12:24:01.041229 27630 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.058228 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.059602 27617 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:01.060540 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.062947 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.064477 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:01.070938 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 296102us
I20260709 12:24:01.030108 27618 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.084049 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:01.084496 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65169.7us
I20260709 12:24:01.073709 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:01.029660 27578 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.030047 27641 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.027223 27591 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:01.009249 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 401464us
I20260709 12:24:01.015323 27656 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.027263 27648 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.055994 27532 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.050284 27659 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:01.095274 27656 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:01.027052 27654 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.083272 27630 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:01.087714 27545 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.713s	user 0.000s	sys 0.011s
W20260709 12:24:01.092686 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 210278us
I20260709 12:24:01.224786 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.226311 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.198050 27562 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:01.200488 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.231911 27639 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.168789 27656 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.232307 27545 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:01.086400 27582 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.235903 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:01.236649 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.124684 27651 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.132748 27618 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:01.138829 27567 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.961s	user 0.004s	sys 0.008s
I20260709 12:24:01.228307 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2034) took 1437 ms. Trace:
W20260709 12:24:01.143102 27604 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.823s	user 0.012s	sys 0.000s
I20260709 12:24:01.087971 27578 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:01.231782 27579 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:01.160781 27597 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.987s	user 0.013s	sys 0.003s
I20260709 12:24:01.159765 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.124384 27636 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.099506 27614 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.239952 26127 rpcz_store.cc:276] 0709 12:23:59.775932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.776468 (+   536us) service_pool.cc:224] Handling call
0709 12:23:59.777362 (+   894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.741969 (+964607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.212926 (+470957us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b5400185bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:01.213114 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155747712}
I20260709 12:24:01.261842 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 913) took 1142 ms. Trace:
I20260709 12:24:01.285804 26047 rpcz_store.cc:276] 0709 12:24:00.119740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.405621 (+285881us) service_pool.cc:224] Handling call
0709 12:24:00.405867 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.042955 (+637088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.261616 (+218661us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400172354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:01.261781 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409236864}
I20260709 12:24:01.241972 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.272362 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:01.239133 27639 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.260638 27578 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.235142 27602 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:01.287452 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45546.4us
I20260709 12:24:01.294987 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.260463 27545 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.227694 27659 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.296707 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:24:01.297927 26127 raft_consensus.cc:606] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:01.282539 27627 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.006s
I20260709 12:24:01.298835 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.299911 27650 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.300376 27651 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.301055 27616 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.261694 27604 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.164016 27664 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 12:24:01.256705 27567 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.270537 27597 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.312346 26053 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:01.305176 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 178333us
I20260709 12:24:01.305614 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.305701 27627 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.155045 27617 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.286545 27636 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:01.301594 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148047us
I20260709 12:24:01.262933 27629 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.302798 27608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c5f2ff7a584f80b9a347f6adb3c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:01.289917 27579 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.314937 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:01.319310 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 235331us
I20260709 12:24:01.321846 26785 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.317358 27639 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.324558 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.306878 27651 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:01.327792 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.329190 26050 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.329490 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.323603 27636 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.337605 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.312855 27663 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.343789 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:01.341066 27605 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.147s	user 0.014s	sys 0.000s
I20260709 12:24:01.357761 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.355671 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:24:01.355541 27668 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.358736 26127 raft_consensus.cc:606] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.322815 27628 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.340914 27651 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.361588 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.395555 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.382117 27635 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.397222 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.397840 27626 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.382712 27668 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.399613 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:01.345041 26046 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.402262 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.403446 27635 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:01.412577 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 891) took 1726 ms. Trace:
I20260709 12:24:01.340721 27585 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.404634 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.356122 26786 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.409574 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 918) took 1064 ms. Trace:
I20260709 12:24:01.401391 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:01.422214 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:01.413295 26050 rpcz_store.cc:276] 0709 12:23:59.686160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.377570 (+691410us) service_pool.cc:224] Handling call
0709 12:24:00.377813 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.341261 (+963448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.412323 (+ 71062us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b5400228b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:01.412513 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187529472}
I20260709 12:24:01.384780 27657 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.424350 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.400121 27597 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.406924 27616 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.412084 27665 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:01.420493 26053 rpcz_store.cc:276] 0709 12:24:00.345072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.417628 (+ 72556us) service_pool.cc:224] Handling call
0709 12:24:00.537563 (+119935us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:00.537760 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.312961 (+775201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.409343 (+ 96382us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:01.409512 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802787584}
I20260709 12:24:01.408425 27632 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.406726 27606 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.411343 27648 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.350342 27637 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:01.386139 27605 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.435097 26787 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.400696 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.466516 27628 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.444725 27614 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.469901 27641 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.471904 27659 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.449121 27632 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:01.483562 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 885) took 1875 ms. Trace:
I20260709 12:24:01.450156 27665 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:01.407722 27650 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:24:01.494957 27650 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.495193 27605 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.495499 27650 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:01.472316 27649 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.476568 27627 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.493698 27637 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.496838 27585 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.492847 27665 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.460537 27602 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:01.513868 27622 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.002s
I20260709 12:24:01.514386 27622 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.522312 27622 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:01.529625 27634 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.603s	user 0.000s	sys 0.011s
I20260709 12:24:01.498318 27665 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.484058 26046 rpcz_store.cc:276] 0709 12:23:59.608086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.175113 (+567027us) service_pool.cc:224] Handling call
0709 12:24:00.175453 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.402227 (+1226774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.483267 (+ 81040us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:01.483493 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643990272}
I20260709 12:24:01.536298 27599 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:01.537794 27632 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.538862 27604 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:01.546734 27596 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.487157 27567 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.539237 27608 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38282 (request call id 320) took 2536 ms. Trace:
I20260709 12:24:01.552265 27576 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:01.506474 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 142639us
W20260709 12:24:01.542778 27601 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.035s	user 0.005s	sys 0.008s
I20260709 12:24:01.549816 26783 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:01.602934 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.606660 27641 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.608682 27641 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:01.609061 27641 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.610605 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:01.612273 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:01.542544 27634 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.461295 27635 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.553439 27576 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.615090 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:01.627339 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.555244 27578 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.601888 27629 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.553609 27599 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:24:01.618762 27657 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.618592 27628 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.635226 27622 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.606232 27601 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.646225 27587 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:01.631079 27576 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:01.640789 27638 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.000s
I20260709 12:24:01.602115 27656 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.659467 27578 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:01.551538 27608 rpcz_store.cc:276] 0709 12:23:59.002508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.003039 (+   531us) service_pool.cc:224] Handling call
0709 12:23:59.707870 (+704831us) ts_tablet_manager.cc:934] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
0709 12:24:01.024691 (+1316821us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:01.511188 (+486497us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:01.511302 (+   114us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:01.538359 (+ 27057us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:01.538647 (+   288us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:01.538956 (+   309us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:01.539028 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":17428,"tablet-copy.queue_time_us":533059,"thread_start_us":418293,"threads_started":2}
I20260709 12:24:01.654604 27657 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:01.664027 27657 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:01.655710 27640 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.638s	user 0.005s	sys 0.005s
W20260709 12:24:01.650767 27600 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.005s
I20260709 12:24:01.672060 26052 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.677610 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:01.680224 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:01.680687 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 267913us
I20260709 12:24:01.680935 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:01.681803 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:01.682590 27648 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.682972 27648 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:01.683329 27648 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.683434 27668 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:01.683885 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:01.686223 27551 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:01.687160 27626 raft_consensus.cc:993] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:24:01.687639 27532 raft_consensus.cc:1081] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:24:01.689009 27616 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:01.689352 27618 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:01.690107 27616 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.690475 27618 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.690828 27618 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:01.692262 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 380724us
W20260709 12:24:01.693681 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.654381 27628 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:01.702572 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.697698 26198 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.697753 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:01.695067 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:01.655268 27622 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:01.610260 27650 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:01.710561 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83342.6us
I20260709 12:24:01.712323 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:01.713469 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:01.713990 26201 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.714148 26195 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:01.714998 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:01.717119 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.729738 27576 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.731117 27576 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.713114 27629 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:01.733934 27588 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.835s	user 0.011s	sys 0.002s
I20260709 12:24:01.734520 26784 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.734426 27588 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.708969 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:01.702211 27600 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.701126 27608 tablet_copy_client.cc:806] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
W20260709 12:24:01.739341 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.690488 27616 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:01.710206 27628 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.747175 27611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ddbca55e34c47f2b2d10e46e4779bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:01.709715 27635 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.713678 27622 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.726085 27578 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.663568 27638 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.725769 27604 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.739298 27585 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.770960 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.772964 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.776073 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:24:01.691605 27618 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:01.701843 27640 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.763051 26786 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.784945 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.779114 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.762636 26123 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.749374 27608 tablet_copy_client.cc:670] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:01.765345 27578 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:01.787501 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.787562 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.784209 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.767014 26139 tablet_copy_service.cc:140] P 9e23389325a6411c9b0634f4f6af54c3: Received BeginTabletCopySession request for tablet f720b2e4986f496694c7578beef67cd4 from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:35926)
I20260709 12:24:01.714524 27636 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:01.793681 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:01.789077 26785 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.794595 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:01.794687 27651 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.795446 27578 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.795984 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1386) took 1421 ms. Trace:
I20260709 12:24:01.796903 26045 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.800870 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.784032 26063 tablet_copy_service.cc:140] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Received BeginTabletCopySession request for tablet 93ec87f552a04ce49312a2ef65f862ce from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:46972)
I20260709 12:24:01.784350 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.807025 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.783031 27588 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.814900 26051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:24:01.825949 26051 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:01.826364 26051 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:01.826448 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:01.817653 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 92584.2us
W20260709 12:24:01.793151 27633 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.007s
W20260709 12:24:01.825152 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.004s	sys 0.038s
W20260709 12:24:01.828682 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.004s	sys 0.038s
W20260709 12:24:01.829195 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 715375us
I20260709 12:24:01.830871 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.833154 27672 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.813827 27651 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:01.825773 27645 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.811s	user 0.010s	sys 0.002s
W20260709 12:24:01.835219 27589 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.943s	user 0.011s	sys 0.003s
I20260709 12:24:01.835079 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.834007 26063 tablet_copy_service.cc:161] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Beginning new tablet copy session on tablet 93ec87f552a04ce49312a2ef65f862ce from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:46972: session id = 24e61c63097e4d3dbb149050e6fcc997-93ec87f552a04ce49312a2ef65f862ce
I20260709 12:24:01.817371 26139 tablet_copy_service.cc:161] P 9e23389325a6411c9b0634f4f6af54c3: Beginning new tablet copy session on tablet f720b2e4986f496694c7578beef67cd4 from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:35926: session id = 24e61c63097e4d3dbb149050e6fcc997-f720b2e4986f496694c7578beef67cd4
W20260709 12:24:01.884801 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49811.8us
I20260709 12:24:01.810042 27622 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.799026 27666 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.215s
I20260709 12:24:01.887373 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.890610 27659 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.837133 27674 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.891636 26787 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.893514 27622 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.894798 27638 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.895124 27648 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.895721 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.896811 27601 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:01.880630 26046 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:01.901824 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:01.903126 27659 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:01.882777 27645 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.888139 27672 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.883070 27634 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.906378 27640 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:01.931169 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.949515 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.933987 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:01.933449 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.900027 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.817147 26123 rpcz_store.cc:276] 0709 12:24:00.374939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.728595 (+353656us) service_pool.cc:224] Handling call
0709 12:24:00.765507 (+ 36912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.789370 (+1023863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.795691 (+  6321us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54001f54d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:01.795892 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236370176}
W20260709 12:24:01.951583 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51642.9us
I20260709 12:24:01.952395 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.939397 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:01.942375 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:01.944327 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.946657 26201 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.953264 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:01.815917 27616 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.838796 27633 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.959830 26048 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:01.960857 26202 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.961948 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.963017 27663 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.963654 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.963760 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.962908 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.889760 27589 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.970322 27646 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.975759 27640 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.976532 27646 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:01.977020 27646 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.949914 27674 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:01.961737 27659 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.965600 26051 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:01.972429 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.980780 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.962705 27666 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.965956 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.964651 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.958945 27660 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.971400 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.984618 26203 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:01.972671 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.981319 26048 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:01.981866 26049 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:01.972447 27605 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.896083 27657 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:01.985961 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.988687 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.990810 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:01.993163 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:01.995162 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:01.995316 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:01.993198 27634 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.992446 27647 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.994814 26054 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:01.996418 27638 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.998075 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.998457 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.998268 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.011847 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:02.013204 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:02.018446 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 188421us
I20260709 12:24:01.999755 27606 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.002153 27664 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.029099 26791 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.983916 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.032577 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:02.034487 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:01.984108 27666 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:01.994323 27616 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.038769 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.040719 26051 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:01.999946 27622 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.002149 27605 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.000110 27638 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:02.060426 27545 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.995653 27601 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:24:02.038398 27637 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.072755 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:02.075006 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.076422 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.082975 26201 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:02.079041 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.079254 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.084220 27673 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:02.084782 27673 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:02.085224 27673 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.085421 27641 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.062461 27608 tablet_copy_client.cc:538] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:02.066900 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:02.087163 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67421.4us
I20260709 12:24:02.011303 27585 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:24:02.008359 27674 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:02.064791 27606 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.090935 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.102380 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.084820 27662 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.106850 27648 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:02.107968 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.108664 26195 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.096284 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:02.076869 27638 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.077037 27545 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:02.099689 27567 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.087118 27634 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:02.106628 27636 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.112362 27551 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:02.119395 27545 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:02.137514 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.115620 27659 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.120628 26782 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.119321 27633 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.149196 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.149881 26198 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:24:02.152745 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.156129 26195 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:02.149413 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.159932 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.106133 27666 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.147706 27551 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.177314 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.186147 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.187240 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.203881 27630 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.204403 27630 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.205185 27630 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.206379 27630 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.209477 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.210361 26045 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:02.164461 27680 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 12:24:02.215682 27684 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.149s	user 0.003s	sys 0.000s
I20260709 12:24:02.222048 27680 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.224630 27680 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.224884 27618 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.184445 27657 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.237898 26198 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:02.249215 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.221385 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:02.245312 27680 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.246973 27664 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:02.247452 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82400.6us
W20260709 12:24:02.194839 27665 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.001s
I20260709 12:24:02.255968 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.174755 27682 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:24:02.257654 26786 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:02.263242 26791 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.263562 27682 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.264019 27682 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.264971 27682 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.266335 27682 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.260286 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:02.261464 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.238389 26063 tablet_copy_source_session.cc:215] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:02.270062 27681 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:02.271157 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.184813 27685 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
W20260709 12:24:02.272164 27635 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.002s
I20260709 12:24:02.272365 27685 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.272770 26196 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:02.273351 27608 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:24:02.272842 26199 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:02.273874 27576 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.542s	user 0.015s	sys 0.000s
I20260709 12:24:02.275612 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:02.276556 27600 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.276896 27650 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:02.278806 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.282341 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.283722 27648 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.287052 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.225857 27578 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.289920 27683 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.291793 26198 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.294621 27551 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:02.284214 27681 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:02.297501 26199 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.297883 27582 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.298321 27582 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.298468 26196 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.299019 27663 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.299289 27582 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.300423 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:02.301378 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.303126 27599 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:02.303562 27599 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:02.303890 27628 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.557s	user 0.003s	sys 0.006s
I20260709 12:24:02.304453 27599 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.299908 26199 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.307005 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 321) took 1583 ms. Trace:
I20260709 12:24:02.309450 26051 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.309917 27656 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.306012 27599 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.311468 27645 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.268950 27641 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.272858 27685 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.313354 27685 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.314633 27626 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:02.314839 27685 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.315795 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.307487 26784 rpcz_store.cc:276] 0709 12:24:00.723611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.727331 (+  3720us) service_pool.cc:224] Handling call
0709 12:24:01.381913 (+654582us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:01.382215 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.735159 (+352944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.306694 (+571535us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54004ffe54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:02.306897 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768778368}
I20260709 12:24:02.318495 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.323267 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.323361 26129 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:02.325686 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:02.321805 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.327682 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:02.320307 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.329511 27633 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.330065 27633 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.330958 27633 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:02.331959 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.334743 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.292218 27680 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.359361 27657 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:02.360426 27657 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.351917 27656 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.360797 27657 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.337667 27664 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.361747 27657 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.333360 27648 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:02.364683 27648 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.365069 27648 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.366189 27648 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.367633 27648 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.369156 27674 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.373386 27606 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.374298 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.375191 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.375319 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2083) took 1325 ms. Trace:
I20260709 12:24:02.340492 27582 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.338840 27637 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.409852 26196 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:02.376608 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.379158 26784 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.315251 27626 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.386708 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:02.319345 27628 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:02.394440 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.400723 26785 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:02.402406 27632 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.403528 26052 rpcz_store.cc:276] 0709 12:24:01.050446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.220069 (+169623us) service_pool.cc:224] Handling call
0709 12:24:01.220372 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.701421 (+481049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.375074 (+673653us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540006fc54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:02.375254 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038183296}
I20260709 12:24:02.412429 26201 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.417851 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.424803 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:02.425436 27647 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.428427 26789 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.432364 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.419268 27641 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:02.440714 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.442646 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:02.441560 27647 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.418095 27657 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.441134 27692 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:24:02.416168 27626 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:02.443562 27673 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.447789 27681 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:02.443781 27532 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.448462 27683 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:02.448086 27647 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.455668 27673 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.455510 27694 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:02.456076 27647 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.451357 27660 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:24:02.452066 27600 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:24:02.448325 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.455693 27683 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:02.458794 27692 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.442500 27616 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:24:02.459877 27660 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.416478 27545 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.458182 27638 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.485071 27683 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:02.486131 27646 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.509s	user 0.003s	sys 0.007s
I20260709 12:24:02.487732 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:02.490509 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.459494 27578 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.493815 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.495404 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.461107 27582 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.498317 27660 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.493530 27692 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.524600 27617 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:02.525228 27692 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.500403 27654 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:02.504810 26788 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.482339 27601 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.491132 27588 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.517306 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:02.527700 27692 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.518908 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.506549 26197 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.533301 27611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38286 (request call id 290) took 3529 ms. Trace:
I20260709 12:24:02.534610 27589 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.538084 26139 tablet_copy_source_session.cc:215] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:02.506848 27545 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:02.544664 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.545428 26202 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:02.515803 27637 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.548780 26200 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.549005 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.549190 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.549818 26050 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:02.549940 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.515578 27582 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.552999 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.554148 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.554489 27628 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:02.507473 27650 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:24:02.557267 27597 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.557543 27654 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.557719 27597 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.558051 27654 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.559139 27605 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.559569 27605 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:02.560468 27605 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.562340 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.563954 26128 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:02.558717 27597 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.561445 27660 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.530133 27616 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.513547 27646 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.587250 26200 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:02.587589 27646 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:02.568878 27618 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:02.529369 27601 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.589541 27601 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.590601 27673 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.590715 27601 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.591114 27673 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:02.582783 27694 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.561774 27605 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.569708 26127 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.571772 27638 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.548839 27610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ec87f552a04ce49312a2ef65f862ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:02.559149 27654 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.602593 27694 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:02.603224 27694 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.575928 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.579574 26202 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.580796 26194 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.605110 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:02.562230 27617 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:02.592875 26127 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.593627 26047 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.594183 26195 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:02.593595 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:02.609920 26782 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.613929 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:02.614109 27652 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.615262 27652 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.615590 27652 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.617843 27650 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:02.618551 27650 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.620321 27617 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.621201 27588 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:02.623279 27617 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.624454 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:02.627198 27638 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.627928 27638 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.604426 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:02.639017 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.641711 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.641927 26051 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:02.643218 27618 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.643415 27551 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.643854 27551 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:02.613973 26127 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.645579 26203 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.646629 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:02.649116 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.653252 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.644408 27597 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.656306 27628 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:02.657632 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.658372 26128 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:02.658509 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:02.659195 26201 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:02.659245 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:02.661038 26128 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 1.
I20260709 12:24:02.661154 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:02.661984 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.662129 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.652271 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.662890 26128 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:02.663491 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:02.663645 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.663205 27582 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:02.664880 27582 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.666711 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.668496 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.666711 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.623837 27617 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:02.671777 26128 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.657455 27628 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:24:02.666752 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54095.7us
I20260709 12:24:02.671098 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.671182 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.612726 27646 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:02.651845 27675 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:24:02.644906 27660 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.664137 27666 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.679394 27587 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.679690 27654 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.679800 27663 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:02.670589 27651 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.680647 27674 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.684170 27646 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.677167 26123 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:02.673523 26126 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:02.685454 27694 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.670861 27616 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.533773 27611 rpcz_store.cc:276] 0709 12:23:59.003925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.004565 (+   640us) service_pool.cc:224] Handling call
0709 12:23:59.716615 (+712050us) ts_tablet_manager.cc:934] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
0709 12:24:01.742830 (+2026215us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:02.327328 (+584498us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:02.327480 (+   152us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:02.532525 (+205045us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:02.532843 (+   318us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:02.533168 (+   325us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:02.533238 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":91622,"mutex_wait_us":5194,"spinlock_wait_cycles":7424,"tablet-copy.queue_time_us":621739,"thread_start_us":260910,"threads_started":2}
I20260709 12:24:02.686367 27675 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.685951 27551 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.676512 27632 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.681578 27635 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:24:02.687423 27638 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.674506 27628 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.619714 27650 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.592015 27673 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.691313 26128 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:02.733280 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.719024 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.734855 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:02.695322 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.778616 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:02.779261 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44502.6us
W20260709 12:24:02.755496 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 154081us
I20260709 12:24:02.756403 26127 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.693061 27588 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
W20260709 12:24:02.780891 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.780059 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:02.738564 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.759589 26124 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:02.760500 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:02.760730 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.761498 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.763597 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.788403 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.789196 26128 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:02.677340 27617 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.675825 27652 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.794297 26051 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.799954 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:02.766144 27675 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.806202 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.775137 27611 tablet_copy_client.cc:806] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:02.810927 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:02.815723 26122 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.785135 27651 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.791777 27659 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:02.758448 27683 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.796635 27641 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.826323 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:02.808419 27589 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.815536 27673 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.814291 27637 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.843070 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:02.845263 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.848654 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.849315 26124 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:02.838881 26127 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:02.795426 27647 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:24:02.858884 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.823452 27611 tablet_copy_client.cc:670] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:02.822351 27659 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:02.843127 27609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f720b2e4986f496694c7578beef67cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:02.842083 27641 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.836455 27578 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:02.865801 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:02.893460 26124 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:02.843681 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.858664 27545 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.839020 27640 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:24:02.836108 27651 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:02.889740 27659 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.912367 27617 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:02.882424 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:02.914021 26788 raft_consensus.cc:2376] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3. Re-sending same reply.
I20260709 12:24:02.882800 26199 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:02.916970 26120 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.917594 27657 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:02.919323 26120 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.921561 27692 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.921974 27692 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.922770 27692 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:02.924386 26125 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.925547 26125 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:02.927237 27551 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.928613 27641 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.929934 26128 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.932642 27545 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:02.933648 27545 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.934484 25952 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:02.924120 27692 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.884130 26121 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:02.892100 27675 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:02.940855 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:02.955144 26783 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:02.942405 27567 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.956333 27567 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.956831 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:02.957201 27567 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.958483 27567 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.939909 27634 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:02.960798 27634 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:02.961881 27634 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:02.942679 27645 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.963387 27675 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.963512 27645 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:02.964468 27645 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.965816 27645 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:02.968432 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.975097 26052 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:02.890581 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.892163 26122 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
W20260709 12:24:02.870900 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.098922 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:02.900030 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:02.881752 27682 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:03.101048 27682 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.103518 27682 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:02.884155 27628 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.905974 27651 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:02.909178 27646 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:02.910856 27652 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.104604 27634 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.950074 27683 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:02.951437 27696 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:24:02.951632 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:02.951936 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:03.108825 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:02.981575 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.110092 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:02.936206 25952 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:03.115016 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:03.117030 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 252911us
I20260709 12:24:03.117103 27683 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.119459 27683 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.120345 27633 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:02.846844 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.121126 27633 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:03.121848 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 275022us
I20260709 12:24:03.122345 27633 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:03.050414 27666 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.124579 27666 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:03.126688 27659 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.127197 27666 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:03.124593 27641 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:03.128921 27657 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:03.129285 27657 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:03.129532 27659 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:03.130687 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.130296 27657 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.132119 27638 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:03.133895 27633 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:03.134295 27633 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:03.135491 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.135430 27633 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.136729 27685 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:03.138882 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:03.139256 27657 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.145135 27611 tablet_copy_client.cc:538] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:03.071275 27597 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.161207 27597 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:03.073185 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 310930us
W20260709 12:24:03.073261 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 116491us
W20260709 12:24:03.163137 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.164253 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.166760 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:03.167058 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:03.172983 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.119854 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.119973 26124 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:02.983697 26128 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:03.160138 27683 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:03.161099 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:03.160843 27633 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:03.194082 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:03.173018 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.082288 27610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38282 (request call id 321) took 4069 ms. Trace:
I20260709 12:24:03.168000 26047 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:03.163437 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:03.182613 26195 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:03.184432 26129 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.178216 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:03.186383 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:03.188871 27646 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:03.211639 26054 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:03.196339 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.210193 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.216087 26048 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 4. Ops: []
I20260709 12:24:03.215682 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:03.241582 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45340.2us
W20260709 12:24:03.224439 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 104720us
I20260709 12:24:03.130286 27659 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.244553 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.245070 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:03.251394 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:03.200356 27673 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:03.215346 27605 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.240415 27683 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.238962 27606 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:03.215626 27656 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.238875 27700 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.238828 27614 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
W20260709 12:24:03.247978 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.275938 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.212394 27646 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:03.212942 27641 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.246579 27675 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:03.246459 27636 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.258378 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.280521 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.252168 27652 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:03.268581 27605 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:03.257045 27682 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.270686 27645 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:03.277303 27597 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.277540 27656 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:03.298393 27656 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.291044 27682 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.299717 27682 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.288975 27702 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:03.301162 27702 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:03.302093 27702 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.291704 27647 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.304914 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:03.298903 27663 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:03.309726 27663 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:03.311033 27663 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.312932 27663 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.282790 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.316398 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:03.323742 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 67382.4us
I20260709 12:24:03.325804 26201 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.270170 27683 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.274595 27606 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.332798 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:03.339334 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94857.5us
W20260709 12:24:03.368031 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.394665 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53555.2us
I20260709 12:24:03.395041 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.396553 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:03.323987 27652 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.397229 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.329217 27650 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:03.330541 27641 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:03.393293 27705 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:24:03.394665 27582 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:03.441227 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44220.7us
I20260709 12:24:03.442235 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:03.404345 27551 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.407923 27588 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:03.332072 27646 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.413571 27650 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:03.197356 27610 rpcz_store.cc:276] 0709 12:23:59.013293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.038633 (+ 25340us) service_pool.cc:224] Handling call
0709 12:24:00.317444 (+1278811us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b712811a2 00007f5b7a1c2889 00007f5b792f3d88 00007f5b793b7a07 00007f5b793e9f4e 00007f5b793cefdd 00007f5b793cef89 00007f5b793cef19 00007f5b793ceee1
0709 12:24:00.398808 (+ 81364us) ts_tablet_manager.cc:934] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
0709 12:24:02.496276 (+2097468us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:02.945766 (+449490us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:02.945889 (+   123us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:03.081367 (+135478us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:03.081699 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:03.082035 (+   336us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:03.082224 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":261,"spinlock_wait_cycles":28445568,"tablet-copy.queue_time_us":585240,"thread_start_us":658057,"threads_started":2}
I20260709 12:24:03.401151 27632 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:24:03.453895 27698 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:24:03.469398 27628 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.452318 27551 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:03.480544 27709 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:24:03.481077 26199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:03.506399 26199 raft_consensus.cc:606] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.506897 26129 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:03.496407 27610 tablet_copy_client.cc:806] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
W20260709 12:24:03.507391 27704 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:24:03.507891 27610 tablet_copy_client.cc:670] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:03.482090 27611 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:24:03.486681 26127 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:24:03.487424 27713 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:24:03.462651 27604 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:03.453511 27640 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.491216 27589 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:03.495115 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98639.3us
I20260709 12:24:03.499056 26195 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:03.490676 27682 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.460711 27700 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:24:03.438427 27605 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.488641 27650 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.518823 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:03.505615 27551 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.519721 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:03.513211 27609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38286 (request call id 298) took 4425 ms. Trace:
I20260709 12:24:03.540416 27609 rpcz_store.cc:276] 0709 12:23:59.087249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.087960 (+   711us) service_pool.cc:224] Handling call
0709 12:24:00.033673 (+945713us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b712811a2 00007f5b7a1c2889 00007f5b792f3d88 00007f5b793b7a07 00007f5b793e9f4e 00007f5b793cefdd 00007f5b793cef89 00007f5b793cef19 00007f5b793ceee1
0709 12:24:00.034299 (+   626us) ts_tablet_manager.cc:934] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
0709 12:24:02.796438 (+2762139us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:03.077949 (+281511us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:03.078111 (+   162us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:03.490330 (+412219us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:03.490662 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:03.491022 (+   360us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:03.513096 (+ 22074us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":75618,"spinlock_wait_cycles":28484352,"tablet-copy.queue_time_us":538697,"thread_start_us":496616,"threads_started":2}
I20260709 12:24:03.541992 27609 tablet_copy_client.cc:806] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:03.524797 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:24:03.527947 26121 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:03.506959 27683 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:03.537715 27652 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.538447 26199 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.543164 26129 raft_consensus.cc:606] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:03.503257 27647 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:03.512594 27606 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:03.551731 26129 raft_consensus.cc:614] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:03.511838 27666 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.557179 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:03.561290 26195 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:03.566560 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:03.512400 27701 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:24:03.512494 27638 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:24:03.620062 27601 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.523705 27604 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:03.651671 27719 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:24:03.653652 27636 raft_consensus.cc:993] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:24:03.654148 27636 raft_consensus.cc:1081] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:24:03.551765 27609 tablet_copy_client.cc:670] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:03.559917 27632 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:03.656194 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 131708us
I20260709 12:24:03.665463 26127 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:03.665268 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:03.666541 26129 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:24:03.666601 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:03.667421 26129 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:03.668498 26129 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:03.669238 27715 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.669656 27715 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:03.669848 27606 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.670053 27715 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.621650 27683 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.616060 27588 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.520406 27641 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.569870 27718 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:03.678393 26129 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.568471 27589 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:03.679328 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 181218us
I20260709 12:24:03.569116 27666 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:03.683914 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56586.4us
I20260709 12:24:03.679665 27683 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.660290 27651 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:24:03.678835 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:03.661921 27601 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.683980 27718 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:03.682960 27588 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.716944 27666 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:03.698678 27683 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.683661 27637 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:24:03.679831 27632 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:03.725374 27682 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:03.733855 27633 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.003s
I20260709 12:24:03.737926 27646 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:03.743446 27721 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 12:24:03.745604 27718 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.747732 27718 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.779193 27715 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.782243 27715 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:03.779197 27683 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.685738 26129 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:03.834023 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:03.836776 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:03.856158 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:03.856879 26197 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:03.858695 26201 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:03.859112 26121 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:03.862668 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:03.848747 26051 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.871671 26790 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:03.874496 27606 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.878845 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:03.880020 26201 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:03.880661 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.882210 26127 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:03.882313 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:03.848711 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:24:03.883914 27683 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:03.889564 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:03.890802 26788 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:03.892515 27652 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.730495 27604 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:03.893652 27736 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:24:03.894048 27606 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.894806 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:03.896469 27617 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.897533 27617 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.893646 27601 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:03.901165 27589 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:03.901859 27664 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:03.903036 27606 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:03.892426 27654 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:24:03.892498 27648 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:03.905293 27648 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.886726 26129 raft_consensus.cc:606] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:03.906400 26054 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:03.906684 26129 raft_consensus.cc:614] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:03.912684 27715 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.913704 27715 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.914669 27632 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:03.914924 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:03.915431 27632 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.916661 26782 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:03.918057 27567 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:03.918797 27567 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:03.919003 27654 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:03.920068 27654 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.921469 27610 tablet_copy_client.cc:538] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:03.922676 27663 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.923344 27663 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.924486 27675 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:03.926008 27663 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:03.925686 27675 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.928632 27666 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:03.929471 27666 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:03.880676 26201 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:03.946834 26052 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:03.949635 27650 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:03.955250 26045 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.956480 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:03.957255 26197 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.957605 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:03.958711 26047 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:03.958878 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:03.926051 27654 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:03.959793 26053 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:03.971349 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:03.972806 27635 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.973666 27635 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:03.974242 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72237.4us
I20260709 12:24:03.974781 27632 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:03.977159 27608 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:03.978195 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:03.978793 27608 tablet_bootstrap.cc:492] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:24:03.979048 26202 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:03.987208 27611 tablet_bootstrap.cc:700] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent opening tablet: real 0.109s	user 0.001s	sys 0.001s
I20260709 12:24:03.893648 27652 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.993382 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:03.994274 26203 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:03.994375 26200 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:03.994633 27675 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:03.995908 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:03.996670 26196 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:03.996717 27715 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.002121 27666 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.009397 26124 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:04.011652 27720 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.012408 27720 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.012944 27720 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.013988 26196 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.051502 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.052502 26128 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:04.053824 27648 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.055028 26201 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:03.980614 27567 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.056926 26201 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.066411 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.067160 26202 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.068792 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.069499 26196 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.074476 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.077025 27701 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.077885 27701 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:04.051798 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74387.6us
I20260709 12:24:04.081652 27650 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:04.082682 27650 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.084205 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:04.084867 27646 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.085575 27646 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.085944 27653 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.086108 27646 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.086589 27653 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.087380 27664 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.087791 27664 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:04.089042 27653 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:04.090747 27675 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.089457 27641 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.091651 27675 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.094620 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.088164 27664 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.103375 27666 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.104404 27666 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.106603 26122 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:04.095752 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.116153 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:04.111778 27641 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.121505 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.122773 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.123349 27641 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.123526 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.124684 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.114310 27608 ts_tablet_manager.cc:1403] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 1.841s	user 0.033s	sys 0.026s
I20260709 12:24:04.126430 27617 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.153025 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:04.154472 27638 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:04.154974 27685 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.155514 27698 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:24:04.124293 27635 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.137955 27663 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.168093 27663 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.168360 27588 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.168624 27663 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.140388 27654 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.177757 27702 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:24:04.178438 27654 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.179219 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.180406 26046 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:04.180783 26787 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.168105 27609 tablet_copy_client.cc:538] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:04.184813 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.186184 27567 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.165856 26122 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.166071 27685 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.166742 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.190703 26054 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:04.176379 27637 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:04.176666 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.177809 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.194216 26202 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:04.200376 26052 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:04.203195 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:04.207063 26127 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.208060 27739 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.208607 27739 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.209021 27739 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.208940 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.210197 26047 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:04.208344 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.212680 26201 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:04.212834 26128 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:04.210582 27702 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.227399 26196 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.204881 27588 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.222680 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.231694 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.232286 27616 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.233289 27616 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.234709 26200 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:04.232525 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.242550 26197 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.243759 26194 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:04.245572 27650 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.246176 27632 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.249114 27685 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.249858 27722 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:04.250284 27722 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:04.255514 27720 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.256824 27675 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.257722 27701 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.258473 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.214840 27666 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.259347 27702 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:04.261929 27551 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.007s
I20260709 12:24:04.264938 27700 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:04.205225 27652 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.232481 26125 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:04.217226 27648 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.281881 26127 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:04.283020 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:04.273028 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56558.9us
I20260709 12:24:04.268409 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.285300 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.286193 26124 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:04.286674 26051 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:04.270852 26049 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.302688 27646 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:04.285887 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 273957us
W20260709 12:24:04.309919 27718 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.008s
I20260709 12:24:04.312994 27652 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.313620 27652 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.320744 27656 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.321349 27656 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.323558 27656 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:04.323760 27588 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.326680 27588 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:04.328907 26199 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.006s
I20260709 12:24:04.329511 27720 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.330073 26199 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:04.330953 26049 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:04.332665 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:04.333422 27651 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.334088 27651 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.336658 27651 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.337193 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.337548 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.338130 26048 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:04.338347 26196 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:04.345152 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:04.346122 27730 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.346524 27730 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.346674 27628 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.346933 27730 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.347307 27628 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.346385 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.347819 27628 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:04.348872 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 132686us
W20260709 12:24:04.349560 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64336.1us
I20260709 12:24:04.355618 27632 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.356168 27632 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:04.356786 27632 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.357110 26782 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:04.357473 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.356184 26054 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.331055 27673 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.360988 27673 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.362579 27617 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:04.363173 27617 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.363643 27617 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:04.364394 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.364213 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.364969 26054 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.365962 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.366597 26054 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.367489 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:04.270726 27654 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.368901 27700 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.376102 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:04.276192 27616 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.281220 27648 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.251207 27722 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.343734 27664 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.385051 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.396029 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.396026 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.382244 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.396275 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.394529 27654 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.397356 27648 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.399591 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.400439 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.393626 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.402094 26053 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:04.404760 26127 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:04.407030 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:04.406152 26790 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:04.410180 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.370081 27551 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:04.414214 26052 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
W20260709 12:24:04.414898 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45082.7us
I20260709 12:24:04.417378 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:04.399292 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.425503 26787 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:04.419872 27682 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.426743 27682 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.422101 27666 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:04.409963 27656 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.428890 27656 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.402096 26782 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:04.429266 27656 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.413592 27638 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:24:04.431525 25952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 187) took 1584 ms. Trace:
I20260709 12:24:04.425699 27659 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.429096 27682 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.406678 27654 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.432293 27659 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.437367 27659 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.413622 26784 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:04.438915 27610 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:24:04.440136 27722 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.404098 27720 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.420037 26194 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:04.477002 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:04.478412 27637 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:04.478978 27637 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:04.479199 27663 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.479954 27637 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.483844 26050 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:04.444713 27653 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:04.488529 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.489331 26200 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:04.489939 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.490545 26201 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
W20260709 12:24:04.490630 27742 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:24:04.491518 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.492115 26196 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:04.447441 26124 raft_consensus.cc:2368] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 2. Current term is 3.
I20260709 12:24:04.431958 27638 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:04.454008 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.455724 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:04.458732 27673 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.461019 27700 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.469780 27701 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.471439 27702 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.473860 27650 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.477180 27654 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:04.452040 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 217285us
I20260709 12:24:04.515738 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:04.514575 27582 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.529790 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.530110 27582 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.530620 27617 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:04.517715 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 100883us
I20260709 12:24:04.532624 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:04.392436 27608 raft_consensus.cc:359] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:04.533337 27608 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:24:04.533737 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.534101 27608 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:04.534752 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.534535 27628 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.524688 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.536417 27702 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.536652 26788 raft_consensus.cc:2368] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:04.536784 26196 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:04.536834 27628 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.537315 27675 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.537778 27675 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.538260 27675 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.525696 27588 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.523335 27683 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.536054 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.529258 27654 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:04.529554 27741 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.396s	user 0.005s	sys 0.000s
I20260709 12:24:04.530527 26201 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:04.525864 27650 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.501050 27638 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:04.431962 25952 rpcz_store.cc:276] 0709 12:24:02.818193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.818343 (+   150us) service_pool.cc:224] Handling call
0709 12:24:04.399841 (+1581498us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:03.289850 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:03.491622 (+201772us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:03.496666 (+  5044us) write_op.cc:435] Acquired schema lock
0709 12:24:03.502556 (+  5890us) tablet.cc:662] Decoding operations
0709 12:24:03.514083 (+ 11527us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:03.514147 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:03.514184 (+    37us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:03.519486 (+  5302us) tablet.cc:701] Row locks acquired
0709 12:24:03.519504 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:24:03.519654 (+   150us) write_op.cc:270] Start()
0709 12:24:03.519777 (+   123us) write_op.cc:276] Timestamp: P: 1783599843519609 usec, L: 0
0709 12:24:03.519796 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:24:03.522705 (+  2909us) log.cc:844] Serialized 919 byte log entry
0709 12:24:03.783725 (+261020us) op_driver.cc:464] REPLICATION: finished
0709 12:24:04.016619 (+232894us) write_op.cc:301] APPLY: starting
0709 12:24:04.016839 (+   220us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:04.017150 (+   311us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:04.036543 (+ 19393us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:04.051166 (+ 14623us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:04.051606 (+   440us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:24:04.051697 (+    91us) write_op.cc:312] APPLY: finished
0709 12:24:04.052210 (+   513us) log.cc:844] Serialized 29 byte log entry
0709 12:24:04.057953 (+  5743us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:04.058155 (+   202us) write_op.cc:454] Released schema lock
0709 12:24:04.058255 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:04.209948 (+151693us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:04.213500 (+  3552us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":232401,"apply.run_cpu_time_us":4857,"apply.run_wall_time_us":172126,"num_ops":2,"prepare.queue_time_us":156067,"prepare.run_cpu_time_us":15214,"prepare.run_wall_time_us":320851,"raft.queue_time_us":65581,"raft.run_cpu_time_us":32919,"raft.run_wall_time_us":289329,"replication_time_us":263728,"thread_start_us":404068,"threads_started":4,"wal-append.queue_time_us":158559}]]}
I20260709 12:24:04.521425 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:24:04.571308 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50249.3us
I20260709 12:24:04.430104 26045 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.512737 27666 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:04.501287 26791 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.536881 27702 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.560081 27617 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.559794 27588 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.676785 25950 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 2, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 2 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:04.678476 27588 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.679014 27609 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:24:04.547427 27683 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.560539 27698 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.560954 27654 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.561332 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.563333 27650 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.563793 27608 ts_tablet_manager.cc:1434] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.419s	user 0.007s	sys 0.002s
I20260709 12:24:04.544768 27616 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:04.564666 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:04.565996 27682 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:04.566910 27582 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.548573 27551 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.568666 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.569265 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.531936 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:04.627055 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78850.1us
I20260709 12:24:04.631858 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.577361 27651 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.637388 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.644363 26121 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:04.646253 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 101624us
I20260709 12:24:04.547538 27720 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.639734 26785 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:04.741969 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.645267 27648 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:04.650254 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.650770 25949 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 291) took 1545 ms. Trace:
I20260709 12:24:04.599982 27701 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:04.623903 27702 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:04.755450 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 207261us
I20260709 12:24:04.679232 27588 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.633312 27730 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.636126 27739 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.739144 27651 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.739876 26200 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:04.740440 27673 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.740525 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.741290 27616 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.686044 27698 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.686689 27683 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:04.701184 27608 ts_tablet_manager.cc:1467] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Tablet startup took 563ms
I20260709 12:24:04.750855 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.751876 25949 rpcz_store.cc:276] 0709 12:24:03.092115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.092305 (+   190us) service_pool.cc:224] Handling call
0709 12:24:04.632680 (+1540375us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
I20260709 12:24:04.763504 26201 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:04.785480 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.786079 26201 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.787381 27675 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.767071 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.788740 26203 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.790217 27616 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.790493 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.751451 26198 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.769822 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.770485 27651 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.721491 27663 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.776326 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.695250 27646 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.780885 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.811689 26200 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.755363 27582 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.790591 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.779577 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.726898 26046 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:04.821045 26052 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.825598 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:04.826159 26054 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.696484 27652 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:04.816017 27637 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:04.822201 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.822357 27551 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.822762 27722 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.822460 27638 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.822189 26196 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:04.768903 27588 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.771929 27673 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:04.852741 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73235.5us
I20260709 12:24:04.832697 26202 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:04.823457 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:04.830950 26047 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.782051 27698 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.831833 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:04.843343 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 242291us
I20260709 12:24:04.863511 26053 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:04.845468 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 103047us
I20260709 12:24:04.754568 27664 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.854708 26051 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:04.837697 27551 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.792310 27682 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:04.864205 26201 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:04.730789 27666 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.779337 27650 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:04.875200 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.875880 26198 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.868314 27663 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.871232 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.870409 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:04.845686 27652 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.865935 26782 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:04.885967 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:04.850483 27588 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.889364 27652 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.862452 27637 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.867906 27659 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.936424 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.844744 27582 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.874780 27750 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:24:04.935014 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49187.4us
I20260709 12:24:04.933192 27652 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:05.021844 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49757.4us
I20260709 12:24:04.954049 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.954536 26201 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:04.951862 26202 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.938737 26124 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.974982 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.970228 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.097966 26122 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:05.098922 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:04.973682 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.099817 26128 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:05.105302 26125 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
W20260709 12:24:05.029537 27656 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.004s
W20260709 12:24:04.962733 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.889930 27638 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:04.970890 27632 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.008s
I20260709 12:24:05.086449 27701 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.018603 27730 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.085986 27673 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.076615 27551 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.084559 27654 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.943811 27653 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.088269 27648 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:04.985706 27739 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.027357 27718 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.862632 27722 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.939824 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:04.967988 27637 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.084897 27683 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:04.931840 27664 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.180s
I20260709 12:24:04.944252 27646 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:04.973275 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.113057 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:05.216495 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 103518us
I20260709 12:24:04.967164 27545 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.976066 26126 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.939234 27617 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.106493 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:05.157125 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.219406 26122 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:04.942472 27578 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.159235 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:05.166265 26203 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:04.962826 27751 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.022110 27700 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
W20260709 12:24:04.781601 27608 ts_tablet_manager.cc:1469] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Trace:
0709 12:23:59.002508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.003039 (+   531us) service_pool.cc:224] Handling call
0709 12:23:59.707870 (+704831us) ts_tablet_manager.cc:934] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
0709 12:24:01.024691 (+1316821us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:01.511188 (+486497us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:01.511302 (+   114us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:01.538359 (+ 27057us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:01.538647 (+   288us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:01.538956 (+   309us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:01.539028 (+    72us) inbound_call.cc:177] Queueing success response
0709 12:24:02.256311 (+717283us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:04.164480 (+1908169us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:04.373210 (+208730us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:04.373833 (+   623us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":17428,"raft.queue_time_us":1054,"raft.run_cpu_time_us":493,"raft.run_wall_time_us":30293,"tablet-copy.queue_time_us":533059,"tablet-download-pool.queue_time_us":17887,"tablet-download-pool.run_cpu_time_us":1944,"tablet-download-pool.run_wall_time_us":228544,"thread_start_us":418293,"threads_started":2}
I20260709 12:24:05.110286 27765 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.084213 27651 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:05.208652 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 341049us
I20260709 12:24:04.939167 27663 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.289213 26093 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:05.116624 27750 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.074978 27762 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.113157 26046 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:04.955240 27759 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:05.022033 27758 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.980481 27748 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:05.207778 26054 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:05.212587 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.155194 27650 raft_consensus.cc:476] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Not starting pre-election -- already a leader
W20260709 12:24:05.222287 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 125289us
I20260709 12:24:05.226011 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:05.226447 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 293246us
I20260709 12:24:05.296442 27682 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.154379 27632 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.116843 27656 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:04.939908 27749 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.283s
I20260709 12:24:05.210445 27551 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:05.216430 27673 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.215742 27654 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.959223 27659 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.222105 27578 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.206936 27653 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.300345 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.308492 27759 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:05.313030 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 206607us
I20260709 12:24:05.337579 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:05.215119 27588 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.290889 27666 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:05.320107 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:05.226009 27761 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:05.323455 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97539.5us
I20260709 12:24:05.214452 27648 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.224381 27718 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.323622 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:05.328130 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:05.322897 27756 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.294629 27545 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.312623 26198 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:05.315169 27701 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.292112 27617 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.317265 27748 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:05.298488 27702 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.347734 26196 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:05.325322 27616 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.297999 27646 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.410795 27683 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.412441 27545 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.413128 27653 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.413290 27588 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.413868 27653 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:05.413908 27701 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:05.415179 27774 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.415987 27774 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.417227 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:05.417958 26053 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:05.418542 27774 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.418908 27663 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.419145 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.419890 26194 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:05.348991 26123 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:05.340346 27682 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.333128 27578 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.375245 26129 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:05.372598 27648 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.350461 27654 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.410607 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:05.414548 27653 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:05.324601 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:05.374816 27748 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.344698 27722 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.316398 27762 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.300426 27751 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:05.377616 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:05.346519 27673 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.340788 27758 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.325623 27765 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:05.342202 27759 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.484041 26121 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:24:05.447753 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 98245us
I20260709 12:24:05.615056 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:05.618822 27775 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.370820 27700 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:05.630513 27683 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:05.631037 27683 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.631424 27683 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.496037 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.634011 26120 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:05.379026 27739 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:05.648274 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:05.371784 27761 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:05.368152 27651 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.502285 27588 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:05.695400 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 371850us
I20260709 12:24:05.484225 27666 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:05.420599 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:05.548563 27774 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:05.697196 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67219.7us
I20260709 12:24:05.697820 26048 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.598215 27648 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.582247 27718 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.592592 27617 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.585486 27702 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.580565 27659 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:05.699986 26202 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 12:24:05.487589 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:05.602166 27698 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.000s
I20260709 12:24:05.425622 27663 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.738956 27698 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.739123 27663 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.740263 26050 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:05.741345 27698 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.741535 27663 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.352906 27551 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.742024 27778 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.742491 27778 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.742847 27778 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.743764 27551 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.586759 27675 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.612948 27632 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.700508 26196 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:05.443305 27616 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.747427 27616 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.701663 27653 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.702252 26169 leader_election.cc:400] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 2. Current term is 3.
I20260709 12:24:05.748169 26169 leader_election.cc:403] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:05.487890 27582 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.702587 27588 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.702816 27637 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:05.704388 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.027s	sys 0.023s
W20260709 12:24:05.754813 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.027s	sys 0.024s
W20260709 12:24:05.755120 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 651253us
I20260709 12:24:05.755715 27588 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.704941 26046 raft_consensus.cc:2368] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3.
I20260709 12:24:05.705726 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:05.698643 27750 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.707157 27578 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.710074 27751 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.712430 27654 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.732143 26049 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:05.712961 27739 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:05.377832 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:05.715232 27673 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.379310 27756 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.715338 27683 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.716327 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:05.483965 27773 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:05.567484 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-69c5f2ff7a584f80b9a347f6adb3c2f8 received from {username='slave'} at 127.0.0.1:46972
I20260709 12:24:05.797734 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.797904 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-69c5f2ff7a584f80b9a347f6adb3c2f8 on tablet 69c5f2ff7a584f80b9a347f6adb3c2f8 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:05.792572 27750 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:05.798467 26126 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:05.630256 27700 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.798794 27750 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.567564 27768 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:05.614835 27656 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.567291 27776 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.799880 27768 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.494483 27767 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.800810 27767 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.801194 27767 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.802531 27768 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.725718 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:05.803455 27720 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.803648 27582 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:05.803781 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78155.2us
I20260709 12:24:05.619593 26785 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:05.804296 27582 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:05.805279 27582 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.806110 27611 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:05.807381 27611 tablet_bootstrap.cc:492] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:24:05.732758 26122 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:05.809595 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:05.810130 26122 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.755929 27748 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.813791 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:05.792337 27759 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.814759 26199 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:05.566075 27682 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.815900 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.699445 27775 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.817155 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:05.747674 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:05.704484 27701 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.824553 27720 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.824380 27578 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:05.825078 27720 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:05.825192 27775 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.729457 27758 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.750175 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:05.774513 27637 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.817188 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.775301 27722 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:05.833884 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.824398 27750 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.834313 27722 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.834659 27773 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.835065 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:05.835311 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.835813 26198 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:05.712827 27638 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.592s	user 0.004s	sys 0.006s
I20260709 12:24:05.785110 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:05.787842 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82191.7us
I20260709 12:24:05.729863 27774 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.841027 26787 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:05.727684 27761 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.797194 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.797966 26120 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.799345 27700 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.798743 27739 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:05.619592 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.800427 27776 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.736558 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:05.818352 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:05.752058 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50027.8us
I20260709 12:24:05.836511 27765 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.835872 26195 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:05.835101 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.799506 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.834424 26199 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:05.808137 27611 ts_tablet_manager.cc:1403] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 2.326s	user 0.042s	sys 0.050s
I20260709 12:24:05.825536 27720 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.849658 27779 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:24:05.843505 26018 leader_election.cc:400] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:05.821197 27759 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.844173 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:05.849476 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:05.826033 27702 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:05.848222 26121 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:05.851679 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:05.842269 27683 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.836665 27675 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.842345 27774 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:05.834333 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.839818 27775 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:24:05.794428 27588 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:05.842658 27750 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.859619 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:05.846691 27758 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:24:05.861939 26194 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:05.972649 26785 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:05.981379 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.857686 27722 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.989661 26018 leader_election.cc:403] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:05.989919 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:05.980758 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:05.990597 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 150450us
I20260709 12:24:05.976092 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:05.991719 27698 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:05.992187 27701 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:05.991926 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:05.993314 26124 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:05.922701 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:05.993810 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 142172us
I20260709 12:24:05.982963 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 934) took 1122 ms. Trace:
I20260709 12:24:05.994273 26051 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:05.861888 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.995376 26202 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:05.995698 26053 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
W20260709 12:24:05.987684 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 169375us
I20260709 12:24:05.958431 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.897482 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:05.997761 26198 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:05.990227 27588 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.997706 26203 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:05.864872 26122 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:05.997244 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:05.955199 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:05.994771 26196 rpcz_store.cc:276] 0709 12:24:04.860630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.939113 (+ 78483us) service_pool.cc:224] Handling call
0709 12:24:05.982666 (+1043553us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:05.982850 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582955264}
W20260709 12:24:06.002401 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84842.2us
I20260709 12:24:06.003212 26047 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:06.003161 27551 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:06.003997 27551 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:05.911566 26757 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.536s (SENT) [suppressed 15 similar messages]
I20260709 12:24:05.913067 27785 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.860586 27776 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.011025 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:05.989784 26194 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.010618 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.006176 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:06.040922 26052 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:05.991223 27750 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:05.859826 27632 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 2. Current term is 3.
I20260709 12:24:05.873775 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:06.051982 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46633.7us
I20260709 12:24:05.972976 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:06.005079 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:06.011502 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:06.049410 27761 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.050035 27588 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:05.919050 27718 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
I20260709 12:24:05.873790 27788 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:05.992321 27663 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:05.920719 27751 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.071167 27788 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:06.071712 27788 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.071362 27751 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:05.970919 27652 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.859s	user 0.009s	sys 0.003s
I20260709 12:24:05.914978 27773 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.060128 26782 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:06.005473 26126 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:06.050931 27616 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.050160 27750 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.103765 26054 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:06.069118 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.005065 27589 raft_consensus.cc:2764] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:06.074638 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.008248 26123 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:06.056036 26053 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:06.118676 26052 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:06.004628 27701 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.118675 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.129690 26054 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.123413 26121 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.124053 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.060031 27768 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:05.856772 27666 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.004712 27675 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.005980 27702 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.074251 27637 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:06.066252 27683 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.080406 27578 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:06.131315 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.141284 26054 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
W20260709 12:24:06.139307 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 227770us
I20260709 12:24:06.051263 27785 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.146760 27785 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.070398 27762 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.066715 27673 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:24:06.155875 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.189429 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:06.155884 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:06.085050 27652 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.121286 27663 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.166244 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.328797 26051 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.184674 27551 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:06.330578 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.331604 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.342806 26052 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.160379 27578 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.175082 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:06.160404 27767 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.252617 27656 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.160422 27683 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.336916 27663 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.249478 27616 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:06.285935 27778 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.002s
I20260709 12:24:06.186417 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.204792 26121 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:06.339057 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.252159 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.340965 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:06.361387 26788 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.362658 26054 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.150108 27637 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.298904 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:06.343987 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.344803 26047 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:06.345731 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:06.271485 27610 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.091s	user 0.000s	sys 0.004s
I20260709 12:24:06.356087 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.352336 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1436) took 1256 ms. Trace:
I20260709 12:24:06.357048 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.359097 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.305680 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:06.160954 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.361267 26791 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:06.360339 26124 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.369994 26052 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:06.141430 27768 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.163023 27675 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.373706 26053 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:06.372766 26128 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:06.165272 26120 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:06.543848 26788 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:06.355297 27551 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.249787 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:06.359297 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.545204 26782 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:06.352394 27567 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.546216 27567 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.547181 27786 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.547384 27718 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.547783 27786 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:06.548557 27567 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.548175 27718 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.549494 26049 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:06.550308 27786 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.550848 27786 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.550943 27718 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.551324 27786 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.372685 26127 rpcz_store.cc:276] 0709 12:24:05.096427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.488140 (+391713us) service_pool.cc:224] Handling call
0709 12:24:06.174858 (+686718us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280e9c 00007f5b71280a5b 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:06.352265 (+177407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19269888}
I20260709 12:24:06.428762 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:06.429303 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.453198 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:06.559029 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105916us
I20260709 12:24:06.559820 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:06.560972 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:06.561609 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.365056 27637 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.563271 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.424204 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.360610 27751 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.564613 26125 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:06.570489 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.370736 26787 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:06.367995 27589 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.570298 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.586264 27589 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.586948 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.587604 26123 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:06.587649 27611 raft_consensus.cc:359] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:06.588673 27589 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.527756 26126 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:06.588835 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.589986 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:06.590597 26126 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.375525 26784 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:06.583778 27610 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.089s	user 0.001s	sys 0.001s
I20260709 12:24:06.595521 27610 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:06.529024 27652 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.596285 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2115) took 1292 ms. Trace:
I20260709 12:24:06.596725 26123 rpcz_store.cc:276] 0709 12:24:05.295399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.586641 (+1291242us) service_pool.cc:224] Handling call
0709 12:24:06.588182 (+  1541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.597280 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.597950 26120 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:06.598210 26123 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:06.598819 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2116) took 1303 ms. Trace:
I20260709 12:24:06.599709 26053 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:24:06.585117 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1435) took 1489 ms. Trace:
I20260709 12:24:06.416702 27701 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.571277 27758 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.591974 27720 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.004s
I20260709 12:24:06.583303 26128 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.600592 26125 rpcz_store.cc:276] 0709 12:24:05.096108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.428371 (+1332263us) service_pool.cc:224] Handling call
0709 12:24:06.585058 (+156687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.601673 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:06.603206 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.601130 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.583352 26121 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.355837 27656 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.493487 27685 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.526959 27702 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:06.602530 27801 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 12:24:06.368736 27778 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.698608 27610 tablet_bootstrap.cc:492] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
W20260709 12:24:06.701699 27776 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.651s	user 0.008s	sys 0.003s
I20260709 12:24:06.461457 27797 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:06.570279 27675 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.527416 27783 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.487442 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:06.697571 27785 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.650034 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
W20260709 12:24:06.597301 27788 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
W20260709 12:24:06.708439 26050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2155) took 2133 ms (client timeout 1588 ms). Trace:
I20260709 12:24:06.651033 26788 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:06.706668 27701 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.602445 27767 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.588836 27578 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.392037 27616 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.601696 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.716224 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.696501 27758 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:06.718396 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 157335us
I20260709 12:24:06.740113 26789 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:06.741357 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:06.600023 27663 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.701418 27652 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.752972 27616 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.752799 27652 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.753473 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.754585 26784 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:06.754840 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:06.726089 27786 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.727161 27610 ts_tablet_manager.cc:1403] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 2.289s	user 0.030s	sys 0.043s
I20260709 12:24:06.710558 27768 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.650858 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:06.599148 26123 rpcz_store.cc:276] 0709 12:24:05.295787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.597442 (+1301655us) service_pool.cc:224] Handling call
0709 12:24:06.597821 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.598579 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.598769 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.707046 27750 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.588689 27611 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:24:06.766983 27611 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:06.723701 26050 rpcz_store.cc:269] 0709 12:24:04.568099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.572324 (+  4225us) service_pool.cc:224] Handling call
0709 12:24:06.700928 (+2128604us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400229a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:06.701088 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134478592}
I20260709 12:24:06.654277 27551 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.722615 27659 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:06.729538 27646 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.732401 27567 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:06.717469 27685 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.733322 27775 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:06.733790 27756 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.725898 27788 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.735260 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.719738 27797 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.718333 27675 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.821702 27675 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.823205 27797 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:06.738021 27785 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.825412 27656 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.825882 27785 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.738885 27802 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.715895 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:06.743117 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:06.702216 27720 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:06.836483 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 106153us
I20260709 12:24:06.743712 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:06.745626 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:06.837579 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94527.3us
I20260709 12:24:06.838140 26788 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:06.749097 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:06.839808 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 90808.4us
I20260709 12:24:06.840708 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:06.750350 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:06.751967 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:06.720453 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1442) took 1098 ms. Trace:
I20260709 12:24:06.721455 27783 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.845804 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:06.754882 27652 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.718312 27776 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.779235 26791 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.758438 27663 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.764212 26785 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:06.760576 27798 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.739375 26126 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:06.742092 27701 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.794456 27611 ts_tablet_manager.cc:1434] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.802s	user 0.024s	sys 0.015s
I20260709 12:24:06.756570 27730 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:06.768647 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:06.794303 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:06.841827 26789 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.784863 27761 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.845335 26124 rpcz_store.cc:276] 0709 12:24:05.621689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.563736 (+942047us) service_pool.cc:224] Handling call
0709 12:24:06.720372 (+156636us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.755923 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:06.875809 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 192) took 1022 ms. Trace:
I20260709 12:24:06.882074 26787 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:06.882037 26126 rpcz_store.cc:276] 0709 12:24:05.853324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.589694 (+736370us) service_pool.cc:224] Handling call
0709 12:24:06.873419 (+283725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.866659 27663 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.849233 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:06.840104 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:06.858556 26127 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:06.785079 27750 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.863880 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:06.849709 27700 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.954s	user 0.013s	sys 0.000s
I20260709 12:24:06.843535 27773 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.847872 27785 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.888181 27785 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.887832 27700 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.888597 27785 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.848649 27756 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.890070 27700 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.872890 26128 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.850406 27797 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.881480 26197 raft_consensus.cc:2368] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3.
I20260709 12:24:06.785037 27659 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.790195 27567 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.859970 27589 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.904767 27701 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:06.906592 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66596.4us
I20260709 12:24:06.907215 27750 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.907413 26756 leader_election.cc:400] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3.
I20260709 12:24:06.907675 26756 leader_election.cc:403] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:06.908517 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:06.909854 27773 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.910279 27773 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.910640 27773 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.909616 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:06.836289 27775 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:06.879544 26199 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
W20260709 12:24:06.875543 27611 ts_tablet_manager.cc:1467] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Tablet startup took 881ms
I20260709 12:24:06.880914 27776 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.881357 27788 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.895154 27797 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.872505 27798 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.904919 27589 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.878063 26194 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:06.901726 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:06.912388 27758 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:06.947947 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46332.1us
I20260709 12:24:06.915356 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:06.938040 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:06.891541 27768 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.944037 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:06.953465 26167 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:06.953708 26196 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.896582 27786 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.844955 27802 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.945317 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:06.896140 27700 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.904343 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 965) took 1072 ms. Trace:
I20260709 12:24:06.899353 27810 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:06.981051 27701 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:06.952981 26201 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:06.980458 26048 rpcz_store.cc:276] 0709 12:24:05.831543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.831726 (+   183us) service_pool.cc:224] Handling call
0709 12:24:05.831966 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.860532 (+ 28566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.860580 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.860639 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:05.860683 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.904057 (+1043374us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:06.904102 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.904263 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65099648}
I20260709 12:24:06.874367 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1440) took 1267 ms. Trace:
I20260709 12:24:06.982069 27720 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.942153 27765 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:06.982592 27700 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.983369 27765 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.984570 27802 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.985538 27765 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.985816 27775 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:06.952255 27589 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.986297 27700 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.987103 27610 raft_consensus.cc:359] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:06.988008 27610 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:24:06.968999 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:06.973315 27768 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:06.988852 27610 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:06.991130 27783 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.991528 27768 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:06.992461 27609 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:06.993418 27609 tablet_bootstrap.cc:492] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:24:06.994117 27609 ts_tablet_manager.cc:1403] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 2.315s	user 0.031s	sys 0.045s
I20260709 12:24:06.994251 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:06.947367 27798 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.890498 27756 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:06.955915 27786 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.939119 27628 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:06.955205 27776 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:06.935600 27664 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3.
I20260709 12:24:06.978271 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:06.946851 27797 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.947829 27778 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.001976 26789 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:06.998383 26167 leader_election.cc:400] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:07.007762 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.019747 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.024209 26201 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:07.000850 27758 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.982249 26125 rpcz_store.cc:276] 0709 12:24:05.607246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.696612 (+1089366us) service_pool.cc:224] Handling call
0709 12:24:06.711318 (+ 14706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.711544 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:06.781495 (+ 69951us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.874044 (+ 92549us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.874099 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:06.874138 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.874202 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.874315 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136837,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":114,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6858,"thread_start_us":147862,"threads_started":2,"wal-append.queue_time_us":123743}
I20260709 12:24:07.023732 26167 leader_election.cc:403] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:06.995256 27651 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.002509 27810 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:07.005916 27786 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.006170 27776 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.019886 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:06.999933 26196 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:07.008219 27788 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:07.023208 27768 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.006469 27802 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:07.033309 27758 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.051402 27545 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:07.039654 27551 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.041527 27776 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:07.054687 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56427.4us
I20260709 12:24:07.059646 26018 leader_election.cc:400] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:07.067904 26018 leader_election.cc:403] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:07.049562 27685 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:07.052284 27788 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:07.049039 27768 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.072922 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.072870 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.091794 26047 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:07.048455 27632 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.080168 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.080577 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.066156 27545 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.056667 27766 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:07.095502 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.060559 27758 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.088645 27788 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.086819 27768 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.041203 27651 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.093057 26048 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:07.063664 26194 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:07.038491 27809 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:24:07.085798 27810 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.067827 27700 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.101812 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:07.097831 27702 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.125072 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.105369 27567 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:07.158897 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.178853 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.101117 26049 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:07.098455 27632 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.126664 27589 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.190577 27756 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.192179 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:07.192449 27589 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:07.192776 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 970) took 1330 ms. Trace:
I20260709 12:24:07.169634 27659 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:07.193318 26045 rpcz_store.cc:276] 0709 12:24:05.862084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.873435 (+ 11351us) service_pool.cc:224] Handling call
0709 12:24:05.897890 (+ 24455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.279391 (+381501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.299695 (+ 20304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.305831 (+  6136us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:06.308093 (+  2262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.192504 (+884411us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400021a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:07.192566 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.192720 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818096384}
I20260709 12:24:07.196544 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1438) took 1836 ms. Trace:
I20260709 12:24:07.197304 26120 rpcz_store.cc:276] 0709 12:24:05.360027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.598631 (+1238604us) service_pool.cc:224] Handling call
0709 12:24:06.598955 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.602030 (+  3075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.602073 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.602617 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.990232 (+387615us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b18001f0fc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:07.082158 (+ 91926us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:07.097753 (+ 15595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.101946 (+  4193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.196180 (+ 94234us) raft_consensus.cc:1685] finished
0709 12:24:07.196208 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.196471 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1026,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":100133,"spinlock_wait_cycles":46542208,"thread_start_us":72988,"threads_started":1,"wal-append.queue_time_us":497046}
I20260709 12:24:07.201009 27651 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.159157 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.206905 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:07.210000 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 154592us
I20260709 12:24:07.218538 26201 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.188777 27656 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:07.205008 27628 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:07.230288 27748 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.231383 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.215480 27785 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.235318 27683 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.235729 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2118) took 1225 ms. Trace:
I20260709 12:24:07.272934 27823 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.272835 26128 rpcz_store.cc:276] 0709 12:24:06.009896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.696649 (+686753us) service_pool.cc:224] Handling call
0709 12:24:06.872717 (+176068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.890785 (+ 18068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.235549 (+344764us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54001d1fd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:07.235675 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376205568}
I20260709 12:24:07.235576 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.273588 27823 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.271518 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:07.276139 26788 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:07.236802 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.237160 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:07.276746 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.239713 27739 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:07.277318 26051 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:07.277681 27739 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.278230 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.278663 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:07.240720 26129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 180) took 2271 ms (client timeout 1536 ms). Trace:
I20260709 12:24:07.279831 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2170) took 1149 ms. Trace:
I20260709 12:24:07.241855 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.242736 26201 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
W20260709 12:24:07.280174 26129 rpcz_store.cc:269] 0709 12:24:04.968883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.973432 (+  4549us) service_pool.cc:224] Handling call
0709 12:24:07.240475 (+2267043us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54001f90d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:07.240626 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618612480}
I20260709 12:24:07.280227 26053 rpcz_store.cc:276] 0709 12:24:06.139767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.706825 (+567058us) service_pool.cc:224] Handling call
0709 12:24:06.715784 (+  8959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.885651 (+169867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.885681 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.885731 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:06.885766 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.279624 (+393858us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54000f9954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:07.279668 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.279779 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390158592}
I20260709 12:24:07.277311 26045 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.282271 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.282898 26201 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:07.283710 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.244284 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.285282 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:07.285807 26198 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.287238 27758 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.252753 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.281931 27628 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.289314 27632 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:07.289561 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:07.290174 26201 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.203797 27700 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:07.264941 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2126) took 1107 ms. Trace:
I20260709 12:24:07.266894 27673 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.268577 27785 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:07.270728 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.270937 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.271322 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.271637 26202 raft_consensus.cc:2368] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:07.271576 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.271359 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:07.327739 26791 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:07.264582 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:07.328457 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63945.5us
W20260709 12:24:07.328755 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57499.5us
I20260709 12:24:07.219797 27675 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.241662 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.228087 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.336563 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:07.352384 26791 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.336696 27632 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:07.305508 27610 ts_tablet_manager.cc:1434] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.544s	user 0.008s	sys 0.003s
I20260709 12:24:07.353317 26782 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:07.353570 27610 ts_tablet_manager.cc:1467] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Tablet startup took 594ms
I20260709 12:24:07.277427 26785 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:07.354382 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.355079 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.355676 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.356284 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.357445 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.358302 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.358942 26787 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:07.349694 26045 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:07.359872 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.302815 27673 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.361089 27773 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.361841 27673 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.346215 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.363196 26788 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
W20260709 12:24:06.940341 27611 ts_tablet_manager.cc:1469] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Trace:
0709 12:23:59.003925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.004565 (+   640us) service_pool.cc:224] Handling call
0709 12:23:59.716615 (+712050us) ts_tablet_manager.cc:934] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
0709 12:24:01.742830 (+2026215us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:02.327328 (+584498us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:02.327480 (+   152us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:02.532525 (+205045us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:02.532843 (+   318us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:02.533168 (+   325us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:02.533238 (+    70us) inbound_call.cc:177] Queueing success response
0709 12:24:03.481721 (+948483us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:05.992507 (+2510786us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:06.072667 (+ 80160us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:06.425271 (+352604us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":91622,"mutex_wait_us":46176,"raft.queue_time_us":14549,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":34291,"spinlock_wait_cycles":7424,"tablet-copy.queue_time_us":621739,"tablet-download-pool.queue_time_us":13241,"tablet-download-pool.run_cpu_time_us":1005,"tablet-download-pool.run_wall_time_us":238250,"thread_start_us":260910,"threads_started":2}
I20260709 12:24:07.374949 26791 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:07.353168 27632 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.295274 26121 rpcz_store.cc:276] 0709 12:24:06.157425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.700634 (+543209us) service_pool.cc:224] Handling call
0709 12:24:06.715350 (+ 14716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.796708 (+ 81358us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:06.860593 (+ 63885us) log.cc:844] Serialized 14 byte log entry
0709 12:24:07.020827 (+160234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.020886 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:07.020928 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.264692 (+243764us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b54001c4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:07.264738 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.264879 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14949,"prepare.queue_time_us":79486,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":4311,"raft.queue_time_us":15086,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":8560,"spinlock_wait_cycles":111168384,"thread_start_us":184555,"threads_started":2,"wal-append.queue_time_us":205729}
I20260709 12:24:07.345091 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.370100 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:07.309407 26198 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:07.375416 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.376889 26200 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:07.378350 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.385133 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:07.346909 27821 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:24:07.326087 27739 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.388177 26791 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:07.385404 26788 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:07.385810 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.394840 26201 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:07.395961 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.263592 27656 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.341851 27675 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:07.337065 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.374694 27701 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:07.418222 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.414342 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:07.351395 27702 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:07.425946 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 190) took 1664 ms. Trace:
I20260709 12:24:07.426378 26127 rpcz_store.cc:276] 0709 12:24:05.761562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.583424 (+821862us) service_pool.cc:224] Handling call
0709 12:24:07.425774 (+842350us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:07.425887 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220783360}
I20260709 12:24:07.428140 27656 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:07.429793 27682 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:07.430797 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.433172 27773 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.433893 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.434688 26201 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:07.435637 27786 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.397219 27685 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:07.405197 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.406024 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.406708 27545 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:07.408627 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 238) took 2038 ms (client timeout 1593 ms). Trace:
I20260709 12:24:07.435755 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.409870 27798 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.384119 27632 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:07.374591 27820 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:07.309376 27663 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:07.409976 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.436203 26169 leader_election.cc:400] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:07.438256 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.400285 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:07.449105 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:07.450966 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:07.403270 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.420987 26783 raft_consensus.cc:2368] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:07.421450 26785 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:07.353891 27610 ts_tablet_manager.cc:1469] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Trace:
0709 12:23:59.013293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.038633 (+ 25340us) service_pool.cc:224] Handling call
0709 12:24:00.317444 (+1278811us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b712811a2 00007f5b7a1c2889 00007f5b792f3d88 00007f5b793b7a07 00007f5b793e9f4e 00007f5b793cefdd 00007f5b793cef89 00007f5b793cef19 00007f5b793ceee1
0709 12:24:00.398808 (+ 81364us) ts_tablet_manager.cc:934] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989)
0709 12:24:02.496276 (+2097468us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:02.945766 (+449490us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:02.945889 (+   123us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:03.081367 (+135478us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:03.081699 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:03.082035 (+   336us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:03.082224 (+   189us) inbound_call.cc:177] Queueing success response
0709 12:24:04.438452 (+1356228us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:06.761957 (+2323505us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:06.767040 (+  5083us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:06.767633 (+   593us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":261,"mutex_wait_us":22532,"raft.queue_time_us":87551,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":3399,"spinlock_wait_cycles":28445568,"tablet-copy.queue_time_us":585240,"tablet-download-pool.queue_time_us":157,"tablet-download-pool.run_cpu_time_us":2046,"tablet-download-pool.run_wall_time_us":412520,"thread_start_us":658057,"threads_started":2}
I20260709 12:24:07.422631 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.420507 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.423991 27701 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.337705 27700 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.425028 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.399325 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.384325 27578 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.456604 26782 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:07.463984 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.447093 27756 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:07.451002 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.454038 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:07.397962 27748 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:07.468919 27748 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.403177 27683 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:07.462673 27768 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.437238 27685 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.474138 26785 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:07.475175 27768 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:07.475790 27685 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.463250 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.463764 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.477310 26198 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:07.464516 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.478538 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.479274 26195 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:07.461874 27578 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.445771 27820 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.456130 27545 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.443068 27798 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:07.441033 26197 rpcz_store.cc:269] 0709 12:24:05.370633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.420976 (+ 50343us) service_pool.cc:224] Handling call
0709 12:24:05.972841 (+551865us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:07.408390 (+1435549us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b5400300554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:07.408532 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894652032}
I20260709 12:24:07.477700 27785 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.442898 27797 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:07.470818 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:07.501525 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.492900 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.437803 27786 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:07.473122 27748 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.472326 27683 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.477097 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:07.526154 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49150.6us
I20260709 12:24:07.419646 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:24:07.527418 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 107833us
I20260709 12:24:07.466084 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.461773 27589 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:07.451766 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 230506us
I20260709 12:24:07.482364 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.467697 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:07.541782 26195 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:07.490137 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.466704 27756 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:07.446334 27663 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.496801 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.502244 26128 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:07.510893 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:07.590006 26127 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:07.512768 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:07.590888 26196 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:07.485460 27578 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.487118 27820 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.528093 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:07.512456 25949 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 4, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:07.596680 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68662.5us
I20260709 12:24:07.597302 25949 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 3, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 3 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:07.544346 26790 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:07.492429 27545 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:07.551507 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.493131 27758 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:07.589017 26122 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
W20260709 12:24:07.587479 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 125200us
I20260709 12:24:07.612993 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.616762 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.619172 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.500806 27785 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.527657 27683 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.622047 25950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 273) took 4499 ms. Trace:
I20260709 12:24:07.600180 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:07.646135 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46038.2us
I20260709 12:24:07.554170 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:07.551020 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.543766 27834 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:07.647480 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.551031 27798 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.603848 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.623802 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.541280 27589 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:07.624585 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.746003 26127 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:07.746107 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.606612 25949 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:07.648447 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94382.8us
I20260709 12:24:07.748140 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:07.592860 27756 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:07.551501 27673 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.742661 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.742832 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:07.597882 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.642078 27652 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.551132 27786 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 12:24:07.586890 27659 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 12:24:07.624429 26125 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:07.649979 27702 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.793700 27750 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.808379 27750 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:07.753571 25949 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:07.809617 27750 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.810765 27750 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:07.793202 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 362) took 1516 ms (client timeout 1687 ms). Trace:
W20260709 12:24:07.812466 26783 rpcz_store.cc:269] 0709 12:24:06.276625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.301166 (+ 24541us) service_pool.cc:224] Handling call
0709 12:24:06.719864 (+418698us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:07.252608 (+532744us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280e9c 00007f5b71280a5b 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:07.792934 (+540326us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b54004ba854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:07.793109 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186959872}
W20260709 12:24:07.784368 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 246863us
W20260709 12:24:07.754154 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 104541us
I20260709 12:24:07.814385 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.815150 27663 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.815573 27663 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:07.816035 27663 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.816923 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.817133 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-8ddbca55e34c47f2b2d10e46e4779bd0 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:24:07.817873 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-8ddbca55e34c47f2b2d10e46e4779bd0 on tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:07.818171 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.819267 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.805691 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:07.821472 26124 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.815789 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:07.751585 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:07.744069 26199 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:07.844323 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:07.858453 26048 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:24:07.860203 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 212785us
I20260709 12:24:07.863931 26198 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:07.589735 27582 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:07.808293 27702 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.876960 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:07.745113 27748 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.643678 27785 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:07.643038 27835 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:24:07.859236 27798 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:07.877741 26054 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.869720 27628 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:07.869264 27778 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.877164 27765 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.882364 26197 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:07.743176 27664 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:07.749886 27589 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.604529 27685 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:07.620642 27675 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:07.786873 27810 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.786262 27758 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.881457 26200 raft_consensus.cc:2368] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:07.884617 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:07.818179 27823 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:07.860988 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109490us
I20260709 12:24:07.807017 26123 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:07.860868 27766 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.798640 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:07.786540 27761 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:07.890892 26093 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:07.963856 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73114.9us
I20260709 12:24:07.871990 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.878183 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:07.881399 27702 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.859431 27834 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:07.890872 27748 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.970259 27748 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.863574 27632 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:07.970196 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.972009 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:07.755472 27609 raft_consensus.cc:359] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:07.973161 27609 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:24:07.973562 26194 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:07.935432 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:07.902325 27798 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.644452 25950 rpcz_store.cc:276] 0709 12:24:03.102418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.102630 (+   212us) service_pool.cc:224] Handling call
0709 12:24:07.601900 (+4499270us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:04.960626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:04.977137 (+ 16511us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:04.977334 (+   197us) write_op.cc:435] Acquired schema lock
0709 12:24:04.977355 (+    21us) tablet.cc:662] Decoding operations
0709 12:24:04.977749 (+   394us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:04.977833 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:04.977863 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:04.978127 (+   264us) tablet.cc:701] Row locks acquired
0709 12:24:04.978157 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:24:04.978331 (+   174us) write_op.cc:270] Start()
0709 12:24:04.978479 (+   148us) write_op.cc:276] Timestamp: P: 1783599844978268 usec, L: 0
0709 12:24:04.978498 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:24:04.979422 (+   924us) log.cc:844] Serialized 541 byte log entry
0709 12:24:05.364476 (+385054us) op_driver.cc:464] REPLICATION: finished
0709 12:24:05.774732 (+410256us) write_op.cc:301] APPLY: starting
0709 12:24:05.774823 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:05.774923 (+   100us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:05.775045 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:05.775414 (+   369us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:05.775609 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:05.775656 (+    47us) write_op.cc:312] APPLY: finished
0709 12:24:05.775859 (+   203us) log.cc:844] Serialized 23 byte log entry
0709 12:24:05.776072 (+   213us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:05.776161 (+    89us) write_op.cc:454] Released schema lock
0709 12:24:05.776204 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:05.776552 (+   348us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:05.776583 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":229490,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":406061,"apply.run_cpu_time_us":1799,"apply.run_wall_time_us":1800,"num_ops":1,"prepare.queue_time_us":183814,"prepare.run_cpu_time_us":3765,"prepare.run_wall_time_us":87262,"replication_time_us":353223,"thread_start_us":445776,"threads_started":2}]]}
I20260709 12:24:07.904681 27778 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.978434 27778 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.943903 26786 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:07.911194 27765 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:07.979863 26786 pending_rounds.cc:85] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 4
I20260709 12:24:07.980337 26786 pending_rounds.cc:107] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:07.980829 26786 raft_consensus.cc:2889] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:07.980425 27765 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.882017 27582 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.919101 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:07.922747 27628 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:07.965409 26050 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.931507 27758 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.929783 27664 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:07.971185 27588 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.923312 27739 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:24:07.963130 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:07.995364 27758 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.968595 27761 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.985846 27766 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:07.902760 27545 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.933770 27685 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.964536 27797 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:08.007833 26020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.020s	sys 0.072s
W20260709 12:24:08.017498 26020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.020s	sys 0.072s
W20260709 12:24:08.017876 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 658557us
I20260709 12:24:08.018774 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:07.999852 27628 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:07.983999 27834 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.008307 26045 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.013552 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.052896 27765 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.054418 27588 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:08.074882 27739 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:08.012084 27766 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:08.074919 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.075492 27685 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:08.093068 26757 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.536s (SENT)
I20260709 12:24:08.070363 27632 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:08.075788 27588 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:08.183275 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.183668 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1341) took 1080 ms. Trace:
I20260709 12:24:08.184875 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.076079 26120 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:08.194696 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.211086 26121 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
W20260709 12:24:08.223290 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.211922 27582 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:08.233958 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140936us
I20260709 12:24:08.205212 27673 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:08.173305 27685 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:08.056592 27609 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:08.182140 27797 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.246682 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.020s	sys 0.066s
W20260709 12:24:08.247990 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.020s	sys 0.066s
I20260709 12:24:08.217775 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1349) took 1020 ms. Trace:
I20260709 12:24:08.080829 27664 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.184213 26199 rpcz_store.cc:276] 0709 12:24:07.104338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.106858 (+  2520us) service_pool.cc:224] Handling call
0709 12:24:08.183424 (+1076566us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:08.183606 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777853696}
I20260709 12:24:08.321841 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:08.237697 27685 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:08.255303 27701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.028s	sys 0.020s
I20260709 12:24:08.328353 26200 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:08.343786 26048 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:08.287147 27802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.015s	sys 0.039s
I20260709 12:24:08.319558 27582 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.356689 26756 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.027s	sys 0.060s
I20260709 12:24:08.363371 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:08.350060 27701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.028s	sys 0.021s
W20260709 12:24:08.367431 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.003s	sys 0.017s
I20260709 12:24:08.255295 27797 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:08.340056 27685 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:08.388379 26756 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.027s	sys 0.060s
I20260709 12:24:08.388702 26121 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
W20260709 12:24:08.404889 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 988273us
W20260709 12:24:08.405325 26054 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.508s	user 0.016s	sys 0.010s
I20260709 12:24:08.405728 26054 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:08.286509 26194 rpcz_store.cc:276] 0709 12:24:07.192419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.206628 (+ 14209us) service_pool.cc:224] Handling call
0709 12:24:08.185292 (+978664us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:08.212902 (+ 27610us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409602816}
W20260709 12:24:08.390488 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.003s	sys 0.018s
W20260709 12:24:08.410307 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 594555us
I20260709 12:24:08.413707 26019 leader_election.cc:400] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:08.415408 26019 leader_election.cc:403] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:08.421366 26124 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.595s	user 0.012s	sys 0.005s
I20260709 12:24:08.421944 26124 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:08.428682 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:08.430809 26200 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:08.431093 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:08.431483 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:08.431847 26196 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:08.429782 26790 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Advancing to term 3
I20260709 12:24:08.425841 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.442906 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:08.430636 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:08.445103 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2185) took 1318 ms (client timeout 1588 ms). Trace:
I20260709 12:24:08.444990 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:08.445542 26047 rpcz_store.cc:269] 0709 12:24:07.126740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.158548 (+ 31808us) service_pool.cc:224] Handling call
0709 12:24:08.444881 (+1286333us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400229a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:08.445010 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832621312}
I20260709 12:24:08.446089 26787 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:08.446717 27786 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.447839 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:08.448431 26786 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:08.448727 27786 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:08.473212 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.007s	sys 0.022s
W20260709 12:24:08.473598 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.007s	sys 0.022s
I20260709 12:24:08.414048 27772 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:08.436829 27776 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.347s	user 0.010s	sys 0.014s
W20260709 12:24:08.436978 27663 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.620s	user 0.014s	sys 0.024s
W20260709 12:24:08.436338 27820 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.814s	user 0.024s	sys 0.001s
I20260709 12:24:08.423291 27701 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.414685 27675 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:08.443662 27756 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.657s	user 0.005s	sys 0.022s
I20260709 12:24:08.400614 27582 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.479327 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:08.510157 26050 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.524s	user 0.017s	sys 0.011s
I20260709 12:24:08.485083 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:08.472226 27578 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.850s	user 0.020s	sys 0.005s
I20260709 12:24:08.436666 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.452399 27773 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.439695 27823 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:24:08.512323 25949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.004s	sys 0.026s
W20260709 12:24:08.543028 25949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.005s	sys 0.026s
W20260709 12:24:08.515841 26125 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.012s	sys 0.014s
W20260709 12:24:08.525866 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.004s	sys 0.031s
I20260709 12:24:08.527287 26050 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
W20260709 12:24:08.479032 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.030s
W20260709 12:24:08.544107 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.000s	sys 0.030s
I20260709 12:24:08.507489 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:08.543527 26125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.012s	sys 0.014s
W20260709 12:24:08.543869 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 624792us
W20260709 12:24:08.548357 27810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.031s
W20260709 12:24:08.549490 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.014s	sys 0.030s
W20260709 12:24:08.556898 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.014s	sys 0.030s
W20260709 12:24:08.557382 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 574474us
W20260709 12:24:08.541739 26202 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.026s	sys 0.030s
W20260709 12:24:08.559864 26202 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.026s	sys 0.030s
I20260709 12:24:08.563166 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.491668 27772 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.494997 27776 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:08.567654 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:08.496093 27663 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.580031 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:08.501889 27820 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.506173 27756 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:08.568312 27810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.005s	sys 0.031s
W20260709 12:24:08.560416 27700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.020s	sys 0.033s
I20260709 12:24:08.542569 27773 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.527081 26197 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:08.598318 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.599233 26124 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:08.572135 27652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.012s	sys 0.016s
W20260709 12:24:08.593039 26129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.027s	sys 0.012s
W20260709 12:24:08.645092 26129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.027s	sys 0.012s
W20260709 12:24:08.645248 26168 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.588s (SENT)
I20260709 12:24:08.646373 26203 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:08.647558 26791 raft_consensus.cc:2368] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6.
I20260709 12:24:08.649469 26051 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:08.650852 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:08.596735 27700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.020s	sys 0.033s
I20260709 12:24:08.645710 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:08.603960 27652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.012s	sys 0.016s
I20260709 12:24:08.587692 27810 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:08.650964 27545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.009s	sys 0.012s
I20260709 12:24:08.602443 27773 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.544806 27578 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:08.661664 27609 ts_tablet_manager.cc:1434] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 1.667s	user 0.013s	sys 0.012s
W20260709 12:24:08.683156 27545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.009s	sys 0.012s
I20260709 12:24:08.671510 27845 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.691073 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 279784us
I20260709 12:24:08.674003 27859 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:08.704258 26169 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.021s	sys 0.006s
I20260709 12:24:08.694317 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:08.682551 27632 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.683099 27857 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
W20260709 12:24:08.686652 27609 ts_tablet_manager.cc:1467] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Tablet startup took 1692ms
I20260709 12:24:08.698346 27756 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:08.730113 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1348) took 1538 ms (client timeout 1547 ms). Trace:
I20260709 12:24:08.687911 27797 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:08.699723 27850 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 12:24:08.703222 27848 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:08.679736 27652 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:08.650607 27675 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.716275 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:08.714236 26169 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.022s	sys 0.006s
W20260709 12:24:08.802131 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 643164us
I20260709 12:24:08.712711 27862 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:08.714579 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 277242us
W20260709 12:24:08.720572 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.716897 27628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.017s	sys 0.012s
W20260709 12:24:08.721237 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.024s	sys 0.033s
W20260709 12:24:08.730569 26203 rpcz_store.cc:269] 0709 12:24:07.192102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.241620 (+ 49518us) service_pool.cc:224] Handling call
0709 12:24:08.729911 (+1488291us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:08.730054 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3051570048}
W20260709 12:24:08.731838 26791 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 579) took 1348 ms (client timeout 1695 ms). Trace:
I20260709 12:24:08.728160 27823 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.814728 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.024s	sys 0.033s
W20260709 12:24:08.809373 26167 leader_election.cc:336] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.695s (SENT)
W20260709 12:24:08.815407 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 796682us
I20260709 12:24:08.815605 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:08.816196 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:08.735313 26052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.008s	sys 0.028s
I20260709 12:24:08.713616 27859 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:08.818015 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:08.818610 26052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.009s	sys 0.028s
I20260709 12:24:08.730914 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:08.816898 26791 rpcz_store.cc:269] 0709 12:24:07.383641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.399703 (+ 16062us) service_pool.cc:224] Handling call
0709 12:24:08.731641 (+1331938us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540056aed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:08.731777 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2779880704}
W20260709 12:24:08.819341 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103218us
I20260709 12:24:08.826409 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:08.831722 26121 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:08.824595 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.836385 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:08.838474 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:08.733776 27701 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:08.817548 27628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.017s	sys 0.013s
W20260709 12:24:08.840516 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 109696us
I20260709 12:24:08.706269 27845 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:08.844774 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:08.847771 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.848841 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:08.889796 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:08.849094 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.892459 26052 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 1.
I20260709 12:24:08.892763 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:08.893625 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.894613 26052 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:08.895576 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.849887 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:08.845922 27786 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.841552 27582 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:08.850597 27756 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.814254 27862 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:08.866696 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:08.845584 27854 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.812911 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.902735 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.836040 27859 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.840907 27673 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.904001 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:08.851850 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 269915us
W20260709 12:24:08.914533 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46615.2us
I20260709 12:24:08.912312 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:08.912535 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1362) took 1673 ms. Trace:
I20260709 12:24:08.912469 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 196) took 1400 ms. Trace:
I20260709 12:24:08.848742 27802 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.889716 27588 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.894088 27545 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.955868 26048 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:08.937572 26051 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:08.934759 26050 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:08.939424 26052 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:24:08.940215 27756 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:08.819029 27865 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:24:08.945175 26094 leader_election.cc:400] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:08.961241 26094 leader_election.cc:403] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.962060 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:08.963505 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:08.938555 26125 rpcz_store.cc:276] 0709 12:24:07.498193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.820057 (+321864us) service_pool.cc:224] Handling call
0709 12:24:08.000026 (+179969us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:08.000226 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.000459 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:08.566780 (+566321us) log.cc:844] Serialized 14 byte log entry
0709 12:24:08.862299 (+295519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:08.862359 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:08.862413 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.862499 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.898416 (+ 35917us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14371,"prepare.queue_time_us":692708,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":1073,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":13258,"spinlock_wait_cycles":43178368,"thread_start_us":848532,"threads_started":2}
I20260709 12:24:08.840687 27823 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:08.949203 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104563us
I20260709 12:24:08.903837 27820 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:08.905517 27856 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:08.953534 26020 connection.cc:256] RPC call timeout handler was delayed by -1.0173848790000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 12:24:08.958314 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91720.1us
I20260709 12:24:08.938370 27854 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.980455 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:08.980855 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:08.958877 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.982448 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.818445 27810 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:08.912285 27776 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:08.974243 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:08.974874 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.911659 27663 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:08.983385 26047 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:08.984397 26050 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:08.942698 27772 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:08.932413 27862 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.009307 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.010026 26051 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:09.011318 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:08.997208 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
W20260709 12:24:09.001305 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 182143us
I20260709 12:24:09.015913 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:08.964601 27756 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:09.081272 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64371.8us
I20260709 12:24:09.019675 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.082711 26050 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:08.957813 27802 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:09.082935 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:09.083948 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72828.8us
I20260709 12:24:08.982291 27868 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.982113 27864 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.960858 27673 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:09.095288 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:08.986905 27854 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:09.081328 27589 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.148s	user 0.014s	sys 0.006s
I20260709 12:24:08.980692 27582 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.987589 27856 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:09.096419 27778 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.117s	user 0.013s	sys 0.011s
I20260709 12:24:08.979549 26063 tablet_copy_service.cc:342] P 8fd5643e7b494fa79c3d0aaf1e6e8152: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-93ec87f552a04ce49312a2ef65f862ce received from {username='slave'} at 127.0.0.1:46972
I20260709 12:24:09.010998 26120 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:09.117669 27856 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:09.093132 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.107342 27589 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:09.100595 27851 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260709 12:24:08.987221 27545 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:09.087200 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.159789 26048 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:08.986996 27878 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:09.004467 27578 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:09.100926 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.100924 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:09.013289 27701 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.091390 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:08.725754 27609 ts_tablet_manager.cc:1469] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Trace:
0709 12:23:59.087249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.087960 (+   711us) service_pool.cc:224] Handling call
0709 12:24:00.033673 (+945713us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b712811a2 00007f5b7a1c2889 00007f5b792f3d88 00007f5b793b7a07 00007f5b793e9f4e 00007f5b793cefdd 00007f5b793cef89 00007f5b793cef19 00007f5b793ceee1
0709 12:24:00.034299 (+   626us) ts_tablet_manager.cc:934] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
0709 12:24:02.796438 (+2762139us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:03.077949 (+281511us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:03.078111 (+   162us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:03.490330 (+412219us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:03.490662 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:03.491022 (+   360us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:03.513096 (+ 22074us) inbound_call.cc:177] Queueing success response
0709 12:24:04.052700 (+539604us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b180015c160. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b7115ced8 00007f5b712b7de2 00007f5b7931cd23 00007f5b7931c55d 00007f5b793b83d8 00007f5b793e9f4e 00007f5b793cefdd
0709 12:24:04.678524 (+625824us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:06.994410 (+2315886us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:07.252339 (+257929us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:07.253178 (+   839us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":75618,"mutex_wait_us":3022,"raft.queue_time_us":4965,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":31277,"spinlock_wait_cycles":38692992,"tablet-copy.queue_time_us":538697,"tablet-download-pool.queue_time_us":12698,"tablet-download-pool.run_cpu_time_us":2545,"tablet-download-pool.run_wall_time_us":471951,"thread_start_us":496616,"threads_started":2}
I20260709 12:24:09.087453 27772 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.086305 27820 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:09.090299 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 954) took 1815 ms (client timeout 1958 ms). Trace:
I20260709 12:24:09.124557 26050 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.124737 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:09.123045 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115936us
W20260709 12:24:09.103312 27685 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.682s	user 0.017s	sys 0.005s
I20260709 12:24:09.151904 26195 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:09.103953 26045 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.030s	user 0.016s	sys 0.011s
I20260709 12:24:09.094424 27866 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:09.103884 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:08.960366 27652 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:09.247931 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 155886us
I20260709 12:24:09.162477 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:09.212400 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:09.352658 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57657.4us
W20260709 12:24:09.350957 26201 rpcz_store.cc:269] 0709 12:24:07.274678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.448814 (+174136us) service_pool.cc:224] Handling call
0709 12:24:09.090066 (+1641252us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400162e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:09.090222 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3317689216}
W20260709 12:24:09.304965 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 330768us
I20260709 12:24:09.355120 26047 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:09.355549 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.357004 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:09.126252 26063 tablet_copy_service.cc:434] P 8fd5643e7b494fa79c3d0aaf1e6e8152: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-93ec87f552a04ce49312a2ef65f862ce on tablet 93ec87f552a04ce49312a2ef65f862ce with peer 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:09.152655 27683 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.399s	user 0.020s	sys 0.010s
W20260709 12:24:09.124358 27858 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 12:24:09.123636 27582 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:09.359953 27852 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.511s	user 0.000s	sys 0.001s
W20260709 12:24:09.225107 27876 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:24:09.127168 27845 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:09.247539 27871 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.311s	user 0.001s	sys 0.001s
I20260709 12:24:09.123904 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:09.432507 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75576.8us
I20260709 12:24:09.212944 27874 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.449645 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:09.291182 27750 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.480s	user 0.019s	sys 0.009s
I20260709 12:24:09.292289 27589 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:09.202606 27847 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.285755 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:09.443645 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 335331us
I20260709 12:24:09.203161 27881 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:09.444336 26050 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:09.353138 26045 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:09.451783 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.400851 27802 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.151343 27778 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:09.460156 26121 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.000s
I20260709 12:24:09.429404 27582 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:09.477356 26054 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:09.472980 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 133580us
I20260709 12:24:09.479171 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:09.480376 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:09.481468 26047 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:09.456179 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 195) took 1958 ms. Trace:
I20260709 12:24:09.358893 27820 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:09.485064 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84652.2us
I20260709 12:24:09.355934 27685 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.451599 27663 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.489595 26121 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:09.439632 27578 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:24:09.312805 27673 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.538160 27700 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.492161 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2200) took 1612 ms. Trace:
I20260709 12:24:09.429039 27683 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.377329 27772 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.546284 26122 rpcz_store.cc:276] 0709 12:24:07.497888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.750647 (+252759us) service_pool.cc:224] Handling call
0709 12:24:07.811295 (+ 60648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:07.811776 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.811812 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.812660 (+   848us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.253322 (+440662us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:08.253380 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.253475 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.455931 (+1202456us) raft_consensus.cc:1685] finished
0709 12:24:09.455971 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.456117 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5878,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":440251,"threads_started":1,"wal-append.queue_time_us":882579}
I20260709 12:24:09.555466 26124 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:09.540711 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:09.427268 27701 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:09.463207 27866 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:09.546449 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 194) took 2048 ms. Trace:
I20260709 12:24:09.563117 26127 rpcz_store.cc:276] 0709 12:24:07.497590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.751258 (+253668us) service_pool.cc:224] Handling call
0709 12:24:07.841905 (+ 90647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:07.864219 (+ 22314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.864274 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.864689 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.473997 (+609308us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:08.474051 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.474121 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.546210 (+1072089us) raft_consensus.cc:1685] finished
0709 12:24:09.546235 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.546393 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":23514,"thread_start_us":585452,"threads_started":1,"wal-append.queue_time_us":811811}
I20260709 12:24:09.548000 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:09.548087 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98553us
I20260709 12:24:09.495370 26197 raft_consensus.cc:2368] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3.
I20260709 12:24:09.481120 27857 raft_consensus.cc:2764] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:09.481846 27881 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:09.557662 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78560.5us
I20260709 12:24:09.482221 27823 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.567080 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:09.539572 27582 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.539306 27750 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:09.580979 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.580003 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:09.581774 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.583935 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:09.587934 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47298.8us
I20260709 12:24:09.486898 26120 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:09.596433 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:09.561091 27656 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:09.576262 27877 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 12:24:09.611969 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:09.449040 27545 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:09.634511 27879 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 12:24:09.635571 27856 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.001s
I20260709 12:24:09.603211 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:09.629412 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62626.8us
I20260709 12:24:09.563349 27700 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.611583 27578 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:09.612686 27766 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:09.651712 27766 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:09.652169 27766 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:09.654330 27786 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.740s	user 0.007s	sys 0.002s
I20260709 12:24:09.655930 27685 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.635638 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:09.636948 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:09.657696 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:09.473973 27847 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.642880 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:09.613060 27873 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
I20260709 12:24:09.642974 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:09.693356 26198 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:09.614281 27872 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.374s	user 0.005s	sys 0.000s
W20260709 12:24:09.647709 27702 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.645s	user 0.016s	sys 0.007s
I20260709 12:24:09.649204 27683 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:09.553814 26045 rpcz_store.cc:276] 0709 12:24:07.876867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.929227 (+ 52360us) service_pool.cc:224] Handling call
0709 12:24:09.489374 (+1560147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.649629 27700 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:09.650180 27578 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:09.649776 27856 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:09.651247 26194 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:09.456161 27874 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.548836 27663 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.679935 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.685595 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:09.582665 27881 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:09.611966 27701 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.612186 27885 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:09.621165 27823 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:09.639933 27785 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.729s	user 0.015s	sys 0.014s
I20260709 12:24:09.700217 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:09.613540 27900 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:09.654757 27786 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.700459 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:09.709544 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72711.7us
W20260709 12:24:09.688014 27798 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.684s	user 0.010s	sys 0.012s
I20260709 12:24:09.743788 27798 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.745869 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2146) took 1096 ms. Trace:
W20260709 12:24:09.684633 27880 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:24:09.746258 26124 rpcz_store.cc:276] 0709 12:24:08.649290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.650569 (+  1279us) service_pool.cc:224] Handling call
0709 12:24:09.745651 (+1095082us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54001f95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:09.745787 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068994944}
I20260709 12:24:09.747565 26200 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:09.710841 27894 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:09.748502 27885 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.749562 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2128) took 1933 ms. Trace:
I20260709 12:24:09.644477 27845 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.750794 27885 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:09.751438 27889 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:09.751168 27845 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.751948 27889 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:09.752928 27889 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.753937 27845 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:09.718137 26051 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.688s	user 0.000s	sys 0.011s
W20260709 12:24:09.755368 27886 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
I20260709 12:24:09.755671 26051 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:09.757303 26051 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:09.708617 27702 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.763082 26791 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:09.763732 26196 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.758976 27785 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.740023 27823 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:09.771623 27785 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:09.772571 27823 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.774580 27874 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.778661 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 261) took 1336 ms. Trace:
I20260709 12:24:09.719306 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:09.720293 27578 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.693106 27675 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:09.729676 26122 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:09.655570 26049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2186) took 2520 ms (client timeout 2211 ms). Trace:
I20260709 12:24:09.654839 27896 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:09.801533 26051 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.632378 27750 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:09.716629 27758 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.707s	user 0.014s	sys 0.010s
W20260709 12:24:09.739890 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 136927us
I20260709 12:24:09.740420 26788 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:09.741679 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:09.727552 27748 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.757s	user 0.015s	sys 0.014s
I20260709 12:24:09.742950 26201 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:24:09.762676 27888 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 12:24:09.624902 27866 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.721499 27685 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:09.779593 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:09.750100 26129 rpcz_store.cc:276] 0709 12:24:07.816342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.972690 (+156348us) service_pool.cc:224] Handling call
0709 12:24:07.973048 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.645523 (+672475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.645552 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.672357 (+ 26805us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.942354 (+269997us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:08.944210 (+  1856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.944432 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.749267 (+804835us) raft_consensus.cc:1685] finished
0709 12:24:09.749296 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.749475 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148201,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":21059,"thread_start_us":837989,"threads_started":2,"wal-append.queue_time_us":742165}
I20260709 12:24:09.793865 26045 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:24:09.855372 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:09.856654 27866 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.856889 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:09.857586 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:09.858592 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:09.783243 26200 rpcz_store.cc:276] 0709 12:24:08.441967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.444366 (+  2399us) service_pool.cc:224] Handling call
0709 12:24:08.489109 (+ 44743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.748094 (+1258985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.778291 (+ 30197us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400398854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:09.778570 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2893503744}
I20260709 12:24:09.859009 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:08.938441 26202 rpcz_store.cc:276] 0709 12:24:07.224563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.618769 (+394206us) service_pool.cc:224] Handling call
0709 12:24:07.746717 (+127948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.747124 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:08.569367 (+822243us) log.cc:844] Serialized 14 byte log entry
0709 12:24:08.897907 (+328540us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:08.897971 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:08.898024 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.898106 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.898240 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140704,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":36602,"thread_start_us":1067951,"threads_started":2}
I20260709 12:24:09.804287 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:09.817955 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76361.5us
I20260709 12:24:09.859782 26129 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:09.821362 27892 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:09.821853 26789 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:09.821970 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:09.874619 26791 raft_consensus.cc:2368] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:09.875108 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:09.822504 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:09.823143 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:09.823438 26051 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:09.827610 27834 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.769s	user 0.011s	sys 0.000s
W20260709 12:24:09.788913 27873 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:09.829888 26196 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:09.821249 27683 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:09.860648 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:09.860915 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:09.862332 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:09.864012 27864 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.761s	user 0.007s	sys 0.004s
I20260709 12:24:09.830614 27758 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.868203 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 170) took 1004 ms. Trace:
I20260709 12:24:09.875764 26789 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:09.880674 27890 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:24:09.802709 27900 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.880697 26791 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:09.899931 26122 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.899179 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:09.879310 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:09.905009 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:09.905951 26201 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
W20260709 12:24:09.868445 27761 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.858s	user 0.017s	sys 0.017s
W20260709 12:24:09.910094 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49526.2us
I20260709 12:24:09.878021 26120 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:09.851663 27685 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:09.902788 26054 rpcz_store.cc:276] 0709 12:24:08.864165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.892474 (+ 28309us) service_pool.cc:224] Handling call
0709 12:24:09.867994 (+975520us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400153454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:09.868138 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337269504}
I20260709 12:24:09.828452 27857 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:24:09.946508 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2204) took 1007 ms. Trace:
I20260709 12:24:09.948010 27899 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.947811 27582 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:09.883755 27778 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.954349 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:09.850490 27748 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:09.952898 26051 rpcz_store.cc:276] 0709 12:24:08.939113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.899214 (+960101us) service_pool.cc:224] Handling call
0709 12:24:09.944608 (+ 45394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.946156 (+  1548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.946180 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.946219 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:09.946246 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.946327 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.946442 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.909351 27864 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:09.950141 27823 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.916980 27685 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.976317 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 586) took 1540 ms. Trace:
I20260709 12:24:09.960686 27683 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:24:09.979161 26783 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:09.984841 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2207) took 1027 ms. Trace:
I20260709 12:24:09.996894 26051 rpcz_store.cc:276] 0709 12:24:08.957318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.963265 (+1005947us) service_pool.cc:224] Handling call
0709 12:24:09.984206 (+ 20941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.984428 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.984463 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.984519 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:09.984561 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.984633 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.984767 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:09.801600 26049 rpcz_store.cc:269] 0709 12:24:07.135640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.191890 (+ 56250us) service_pool.cc:224] Handling call
0709 12:24:09.655293 (+2463403us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b540006fc54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:09.655477 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4429123456}
W20260709 12:24:09.981987 27765 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.899s	user 0.000s	sys 0.014s
I20260709 12:24:09.977924 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:09.989560 26128 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:09.986109 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2205) took 1046 ms. Trace:
I20260709 12:24:10.007042 26054 rpcz_store.cc:276] 0709 12:24:08.939371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.960916 (+1021545us) service_pool.cc:224] Handling call
0709 12:24:09.985640 (+ 24724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.985794 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.985821 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.985877 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:09.985914 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.985960 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.986059 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.980885 26790 rpcz_store.cc:276] 0709 12:24:08.429165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.429299 (+   134us) service_pool.cc:224] Handling call
0709 12:24:08.429604 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.525262 (+ 95658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.525290 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.525724 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.859688 (+333964us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:08.951565 (+ 91877us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.102836 (+151271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.102912 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.968974 (+866062us) raft_consensus.cc:1685] finished
0709 12:24:09.969011 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.969135 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35045,"prepare.queue_time_us":432858,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":321705,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":28291,"thread_start_us":561953,"threads_started":3,"wal-append.queue_time_us":564263}
I20260709 12:24:09.829916 27896 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.012844 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2208) took 1051 ms. Trace:
I20260709 12:24:10.013271 26051 rpcz_store.cc:276] 0709 12:24:08.961678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.997926 (+1036248us) service_pool.cc:224] Handling call
0709 12:24:10.012388 (+ 14462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.012545 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.012570 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.012607 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.012639 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.012690 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.012791 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.900558 27834 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:09.988265 27582 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:09.914254 27761 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:10.009066 26200 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:24:10.017730 26200 pending_rounds.cc:85] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 5
I20260709 12:24:10.018035 26200 pending_rounds.cc:107] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:10.018317 26200 raft_consensus.cc:2889] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:10.010569 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:09.988484 26126 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.020529 26129 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:10.026387 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:10.011806 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.056461 26122 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:09.962239 27900 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.023569 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.013273 27772 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.028119 27847 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:10.057224 26051 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:10.061651 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:10.037638 26046 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.072506 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.073946 26120 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:10.076790 27748 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.056008 27701 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.078097 27701 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:10.062978 26045 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:10.081746 27834 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.086488 27881 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.091768 27766 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:10.067760 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 176) took 1201 ms. Trace:
I20260709 12:24:10.039640 27883 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.116258 26050 rpcz_store.cc:276] 0709 12:24:08.866059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.564414 (+698355us) service_pool.cc:224] Handling call
0709 12:24:09.564793 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.565182 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.565210 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.565991 (+   781us) log.cc:844] Serialized 26 byte log entry
0709 12:24:09.703756 (+137765us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:09.703792 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.703853 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.067519 (+363666us) raft_consensus.cc:1685] finished
0709 12:24:10.067541 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.067687 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12357,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":23284,"thread_start_us":137386,"threads_started":1,"wal-append.queue_time_us":441402}
I20260709 12:24:10.118513 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:10.067849 27773 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.375s	user 0.006s	sys 0.006s
I20260709 12:24:10.093187 27750 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.094380 27823 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:10.063076 26199 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:10.097935 27896 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.102483 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.130556 26203 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:10.130954 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.132395 26124 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:10.103382 27656 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:10.067082 27890 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.108327 26125 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:10.066006 26122 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:10.117717 27756 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.997s	user 0.005s	sys 0.004s
I20260709 12:24:10.122696 26139 tablet_copy_service.cc:342] P 9e23389325a6411c9b0634f4f6af54c3: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-f720b2e4986f496694c7578beef67cd4 received from {username='slave'} at 127.0.0.1:35926
I20260709 12:24:10.122674 27773 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:10.135094 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 224226us
W20260709 12:24:10.125627 27854 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.996s	user 0.009s	sys 0.003s
I20260709 12:24:10.139075 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.146386 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.147969 26124 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:10.157279 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:10.160285 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.138492 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.138530 27758 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:10.149600 26139 tablet_copy_service.cc:434] P 9e23389325a6411c9b0634f4f6af54c3: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-f720b2e4986f496694c7578beef67cd4 on tablet f720b2e4986f496694c7578beef67cd4 with peer 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:10.122989 27883 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.209575 26787 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:10.158947 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2210) took 1140 ms. Trace:
I20260709 12:24:10.210829 26126 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:10.205713 27907 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:10.308744 26049 rpcz_store.cc:276] 0709 12:24:09.008793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.012401 (+1003608us) service_pool.cc:224] Handling call
0709 12:24:10.034482 (+ 22081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.118579 (+ 84097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.120728 (+  2149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.131394 (+ 10666us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:10.133180 (+  1786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.137023 (+  3843us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.149569 (+ 12546us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.206743 27773 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:10.212044 27859 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.251s	user 0.010s	sys 0.002s
I20260709 12:24:10.144225 27756 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:10.190625 27701 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.169839 27854 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:10.222262 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:10.306313 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.319701 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.319896 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.306814 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.321343 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.153765 27889 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:10.320852 26124 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:10.328953 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.207789 27881 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.329147 27914 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.335865 27900 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.330628 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2214) took 1177 ms. Trace:
I20260709 12:24:10.336606 26054 rpcz_store.cc:276] 0709 12:24:09.154390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.313433 (+1159043us) service_pool.cc:224] Handling call
0709 12:24:10.327691 (+ 14258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.327895 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.327921 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.327966 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:10.327997 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.328070 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.330524 (+  2454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.338374 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.339295 27881 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.322307 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.323120 27748 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:10.190250 26782 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:10.324318 26049 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:10.332190 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.305925 27761 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.342430 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.342913 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.141672 27890 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.343510 26122 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:10.345269 26018 leader_election.cc:400] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:10.348168 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:10.316334 27859 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.318099 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.345613 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:10.360658 26051 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.361652 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.352895 26791 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.369964 26124 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:10.370100 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.373521 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.373558 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.374625 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:10.375485 27756 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.375252 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:10.375830 27761 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:10.349439 27773 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:10.376374 26784 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.376395 26120 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:10.377216 27875 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:10.377611 27875 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:10.377674 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:10.377568 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:10.377935 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:10.372139 27750 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:24:10.378746 27875 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.379740 27875 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.372406 27914 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:10.381244 27914 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:10.381774 27905 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:10.342223 27893 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:24:10.384291 26047 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:10.384876 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:10.383615 26756 leader_election.cc:400] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3.
I20260709 12:24:10.389616 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2157) took 1079 ms. Trace:
I20260709 12:24:10.383656 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:10.351341 27862 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.265s	user 0.008s	sys 0.004s
I20260709 12:24:10.388319 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.379900 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2156) took 1228 ms. Trace:
I20260709 12:24:10.392704 27896 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:10.390739 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:10.397450 26203 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.352828 27820 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:10.391911 27862 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.392395 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:10.334048 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:10.382287 27905 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.394989 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:10.393903 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.392565 27900 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.394913 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.350387 27685 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.401309 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:10.401978 27797 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:10.402055 26199 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.402545 27797 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:10.403395 26129 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.404584 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2150) took 1564 ms. Trace:
I20260709 12:24:10.404943 26128 rpcz_store.cc:276] 0709 12:24:08.840403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.840546 (+   143us) service_pool.cc:224] Handling call
0709 12:24:10.404389 (+1563843us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400186fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:10.404525 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523591040}
I20260709 12:24:10.406378 26128 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:10.407017 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2160) took 1095 ms. Trace:
I20260709 12:24:10.407564 27862 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.408682 26121 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:10.409161 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2152) took 1452 ms. Trace:
I20260709 12:24:10.397148 27896 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:10.409657 26126 rpcz_store.cc:276] 0709 12:24:08.957223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.979357 (+ 22134us) service_pool.cc:224] Handling call
0709 12:24:10.408972 (+1429615us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:10.409089 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439573248}
I20260709 12:24:10.411561 26786 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:10.412034 26126 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:10.387389 27632 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:24:10.403431 27797 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.413092 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1465) took 1001 ms. Trace:
I20260709 12:24:10.413621 27797 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:10.382373 27914 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:10.409742 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2162) took 1097 ms. Trace:
I20260709 12:24:10.415198 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.427977 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:10.386495 27750 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.349597 27881 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:10.407338 26128 rpcz_store.cc:276] 0709 12:24:09.311486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.405731 (+1094245us) service_pool.cc:224] Handling call
0709 12:24:10.406065 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.406749 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.406945 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.410817 27905 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:10.428124 26121 rpcz_store.cc:276] 0709 12:24:09.312020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.547677 (+235657us) service_pool.cc:224] Handling call
0709 12:24:10.409544 (+861867us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:10.409683 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939010304}
I20260709 12:24:10.342270 27748 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:10.430168 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:10.390084 26122 rpcz_store.cc:276] 0709 12:24:09.310672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.384531 (+1073859us) service_pool.cc:224] Handling call
0709 12:24:10.389559 (+  5028us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.430197 26129 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:10.340775 27889 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:10.419344 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 265) took 1324 ms (client timeout 1684 ms). Trace:
I20260709 12:24:10.432917 27685 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:10.451355 26196 rpcz_store.cc:269] 0709 12:24:09.094793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.094961 (+   168us) service_pool.cc:224] Handling call
0709 12:24:10.419165 (+1324204us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:10.419282 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535448320}
I20260709 12:24:10.395566 27656 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:24:10.399467 27766 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:10.410274 27896 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:10.450363 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.436744 27683 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.463907 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.465864 26121 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:10.437686 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:10.462790 27748 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:10.410295 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:10.435461 26049 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.417856 27663 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.416131 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 382) took 1117 ms. Trace:
I20260709 12:24:10.401314 26045 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:10.454581 27889 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.450400 27854 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:10.466454 27900 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.463515 27859 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.478302 27890 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.479192 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2213) took 1376 ms. Trace:
I20260709 12:24:10.557366 26048 rpcz_store.cc:276] 0709 12:24:09.102755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.355282 (+252527us) service_pool.cc:224] Handling call
0709 12:24:10.478984 (+1123702us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:10.479120 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515200256}
I20260709 12:24:10.485307 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 208) took 1073 ms. Trace:
I20260709 12:24:10.558624 26121 rpcz_store.cc:276] 0709 12:24:09.411927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.453651 (+1041724us) service_pool.cc:224] Handling call
0709 12:24:10.465382 (+ 11731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.474978 (+  9596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.485248 (+ 10270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.465482 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:10.486541 27772 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:10.486387 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48828.5us
I20260709 12:24:10.494643 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:10.413800 26126 rpcz_store.cc:276] 0709 12:24:09.411370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.411367 (+999997us) service_pool.cc:224] Handling call
0709 12:24:10.411702 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.412461 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.412692 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.496286 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:10.518738 26203 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
W20260709 12:24:10.580853 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 86372.7us
I20260709 12:24:10.483767 27905 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:10.579563 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 114153us
I20260709 12:24:10.484189 27798 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.594507 26054 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:10.611536 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.578270 27773 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:10.578060 27772 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.643038 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.582903 27589 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.647185 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:10.597155 27864 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:10.650192 26054 raft_consensus.cc:2368] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:10.685871 26054 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.674294 26785 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.720815 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:10.678171 27912 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:10.652432 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.653218 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:10.656991 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6713930839996465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
I20260709 12:24:10.639634 27683 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:10.672372 27766 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.741653 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.650341 27772 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.739203 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.490027 26787 rpcz_store.cc:276] 0709 12:24:09.298536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.312753 (+ 14217us) service_pool.cc:224] Handling call
0709 12:24:09.451604 (+138851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.308505 (+856901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.415756 (+107251us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b5400660154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:10.416060 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1637602048}
I20260709 12:24:10.679786 27864 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:10.741389 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:10.736400 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:10.755178 26051 raft_consensus.cc:2368] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:10.394086 26120 rpcz_store.cc:276] 0709 12:24:09.158414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.378811 (+1220397us) service_pool.cc:224] Handling call
0709 12:24:10.379154 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.379469 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.379501 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.379584 (+    83us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.379636 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.379709 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.379837 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1150,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":175}
W20260709 12:24:10.746104 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 151523us
I20260709 12:24:10.765473 26054 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:10.771906 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.580156 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:10.774148 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.777370 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:10.777465 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 196925us
I20260709 12:24:10.787020 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:10.740769 27683 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:10.791980 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:10.803704 26203 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:10.778009 26196 raft_consensus.cc:2376] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3. Re-sending same reply.
I20260709 12:24:10.746100 27900 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.761243 27864 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.882458 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:10.875080 26203 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:10.891911 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1462) took 1481 ms. Trace:
I20260709 12:24:10.893064 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.910398 26196 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:10.893679 26201 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:10.911861 26203 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:10.769399 27854 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.912513 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.913339 26196 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:10.915575 26129 rpcz_store.cc:276] 0709 12:24:09.410390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.024382 (+613992us) service_pool.cc:224] Handling call
0709 12:24:10.891629 (+867247us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:10.891822 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796231040}
I20260709 12:24:10.769755 27878 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:10.763499 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:10.876345 27875 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:10.894202 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 254) took 3461 ms. Trace:
I20260709 12:24:10.919552 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.747149 27766 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:10.877372 27683 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.932432 26195 rpcz_store.cc:276] 0709 12:24:07.410914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.809860 (+398946us) service_pool.cc:224] Handling call
0709 12:24:07.810172 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.353702 (+1543530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.872450 (+1518748us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002b3754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:10.872748 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3033395968}
I20260709 12:24:10.921784 26129 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:10.877521 27823 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:10.906018 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.911422 27663 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.956135 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1466) took 1544 ms. Trace:
I20260709 12:24:10.959538 26195 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:10.965794 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:10.968268 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:10.970068 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:10.970890 26195 raft_consensus.cc:2368] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:10.971269 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:10.892271 27890 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:24:10.959668 26128 rpcz_store.cc:276] 0709 12:24:09.411639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.437893 (+1026254us) service_pool.cc:224] Handling call
0709 12:24:10.467998 (+ 30105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.468275 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:10.662372 (+194097us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.955555 (+293183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.955638 (+    83us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:10.955688 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.955757 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.956068 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1206,"prepare.queue_time_us":171650,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":1595,"thread_start_us":476725,"threads_started":2}
I20260709 12:24:10.972111 26788 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:10.974922 27905 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:10.975107 26790 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:10.961683 26199 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.559s	user 0.014s	sys 0.000s
I20260709 12:24:10.955233 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.918592 27750 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:24:10.979645 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:10.956266 27823 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:10.983320 26195 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:10.939139 27854 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:10.984583 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 110857us
I20260709 12:24:10.981215 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:10.987275 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:10.986217 26120 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:10.990190 27923 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:24:10.991700 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.019731 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.020787 26195 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:11.036900 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44437.8us
I20260709 12:24:10.993939 27905 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.041957 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.048504 27927 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:24:11.049331 26194 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:11.049857 27927 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.050454 27927 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.052677 27823 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.041054 27797 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:24:11.062789 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 589) took 1243 ms. Trace:
I20260709 12:24:11.039361 27875 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:11.081631 27702 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.082808 26199 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:11.084036 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.084874 26199 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:11.085628 26199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:11.086177 26199 raft_consensus.cc:606] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:11.081638 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:11.088387 26197 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:11.089406 27656 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:11.090652 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2192) took 3270 ms. Trace:
I20260709 12:24:11.070669 27802 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.091832 27802 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.091246 26046 rpcz_store.cc:276] 0709 12:24:07.843221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.861899 (+ 18678us) service_pool.cc:224] Handling call
0709 12:24:07.891074 (+ 29175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.068525 (+2177451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.090247 (+1021722us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:11.090515 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4916621696}
I20260709 12:24:11.063504 26782 rpcz_store.cc:276] 0709 12:24:09.819472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.822454 (+  2982us) service_pool.cc:224] Handling call
0709 12:24:09.822776 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.353002 (+530226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.062410 (+709408us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:11.062693 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747257216}
I20260709 12:24:11.113814 26195 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:11.117417 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2154) took 2113 ms. Trace:
I20260709 12:24:11.117988 26125 rpcz_store.cc:276] 0709 12:24:09.003769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.012281 (+  8512us) service_pool.cc:224] Handling call
0709 12:24:11.117175 (+2104894us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54002c9cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:11.117336 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451187584}
I20260709 12:24:11.120529 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:11.116079 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.122531 26201 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:11.133846 26201 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.134498 26195 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:11.088122 27766 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:11.102288 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 374) took 2694 ms (client timeout 2097 ms). Trace:
I20260709 12:24:11.144572 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.093997 27883 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.149588 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.077430 27797 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:24:11.161202 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.160435 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.051725 27927 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:11.166474 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.073990 27905 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:11.169044 26127 raft_consensus.cc:2376] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3. Re-sending same reply.
W20260709 12:24:11.172315 26169 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 2.441s (SENT) [suppressed 45 similar messages]
I20260709 12:24:11.143493 27900 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.115309 27685 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:24:11.179553 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 194703us
I20260709 12:24:11.182770 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.171933 27892 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.179841 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.157541 27893 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:11.142139 27739 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:24:11.152199 27862 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:11.188920 27883 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:11.189216 27927 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.189999 26202 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.190260 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.227252 26125 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.190549 27905 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:11.171209 27868 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.201030 27893 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.202338 27854 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.203095 27802 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:11.147289 27890 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:24:11.182602 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.178700 27900 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:24:11.244117 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73199.4us
W20260709 12:24:11.148654 26783 rpcz_store.cc:269] 0709 12:24:08.407569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.419573 (+ 12004us) service_pool.cc:224] Handling call
0709 12:24:11.093257 (+2673684us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54005708d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:11.102068 (+  8811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3466168704}
I20260709 12:24:11.295444 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:11.220927 27931 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:11.244093 27918 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:24:11.243360 27893 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:11.304931 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 112876us
I20260709 12:24:11.237643 27868 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:11.297288 27914 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.027s	sys 0.059s
I20260709 12:24:11.183619 27875 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.300019 27847 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:11.306770 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:11.307039 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:11.307461 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:11.294314 27900 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.309124 27931 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:11.309180 26125 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
W20260709 12:24:11.297767 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 146426us
I20260709 12:24:11.320796 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 267) took 1435 ms. Trace:
I20260709 12:24:11.297456 27685 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:11.302035 26049 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.815s	user 0.004s	sys 0.009s
I20260709 12:24:11.299571 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:11.358441 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58969.3us
I20260709 12:24:11.325210 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.341269 26094 leader_election.cc:400] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:11.354647 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.325362 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:11.329066 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:11.362876 26050 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:11.363000 26196 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:11.347780 26054 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:11.340971 26200 rpcz_store.cc:276] 0709 12:24:09.885196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.885474 (+   278us) service_pool.cc:224] Handling call
0709 12:24:09.916523 (+ 31049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.018845 (+102322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.320518 (+1301673us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b5400399254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:11.320724 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141567360}
I20260709 12:24:11.360786 26051 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:11.325037 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.367043 26048 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:11.340399 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:11.312984 27914 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.027s	sys 0.060s
W20260709 12:24:11.338168 27909 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.850s	user 0.007s	sys 0.005s
I20260709 12:24:11.357110 26195 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:11.367959 26125 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:11.310384 27868 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.389046 27900 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.360008 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.395303 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:11.347586 27685 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:11.369997 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.339553 27847 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:11.332893 27889 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.776s	user 0.006s	sys 0.006s
I20260709 12:24:11.370962 27578 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.314996 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:11.372602 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1376) took 1682 ms. Trace:
I20260709 12:24:11.315616 26786 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:11.376925 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:11.300956 27934 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 12:24:11.380890 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2194) took 3536 ms (client timeout 1594 ms). Trace:
I20260709 12:24:11.381129 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.336495 27931 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:11.382149 27748 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.882s	user 0.012s	sys 0.000s
W20260709 12:24:11.331293 27896 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.855s	user 0.012s	sys 0.000s
I20260709 12:24:11.397629 27929 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.383157 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2212) took 2283 ms. Trace:
I20260709 12:24:11.383038 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.423835 26125 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:11.384994 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.385735 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:11.400590 27847 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.423601 26047 rpcz_store.cc:276] 0709 12:24:09.099918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.553768 (+453850us) service_pool.cc:224] Handling call
0709 12:24:11.382980 (+1829212us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:11.383102 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816656640}
I20260709 12:24:11.428118 26200 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:11.386736 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:11.386617 27683 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.387137 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 177) took 2520 ms. Trace:
I20260709 12:24:11.387600 27823 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:11.387835 27930 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:11.332121 27772 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.000s
I20260709 12:24:11.389535 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:11.324692 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.21299e+06us
W20260709 12:24:11.330914 27919 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.853s	user 0.003s	sys 0.010s
I20260709 12:24:11.318146 27878 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:11.389578 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.362078 26094 leader_election.cc:403] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:11.447409 26203 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:11.447743 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 106701us
I20260709 12:24:11.386610 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.358914 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:11.398663 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.449798 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:11.449939 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51380.9us
I20260709 12:24:11.450428 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.450672 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:11.451720 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.451977 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.402094 26198 rpcz_store.cc:276] 0709 12:24:09.690623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.760515 (+ 69892us) service_pool.cc:224] Handling call
0709 12:24:09.760961 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.761238 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.761276 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.801354 (+ 40078us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.199689 (+398335us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.199769 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.199851 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.337556 (+1137705us) spinlock_profiling.cc:243] Waited 93.1 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:11.372310 (+ 34754us) raft_consensus.cc:1685] finished
0709 12:24:11.372333 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.372541 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214065024,"thread_start_us":351981,"threads_started":1,"wal-append.queue_time_us":545829}
W20260709 12:24:11.408725 26053 rpcz_store.cc:269] 0709 12:24:07.843956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.844114 (+   158us) service_pool.cc:224] Handling call
0709 12:24:11.380638 (+3536524us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b540022ae54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:11.380805 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5094743296}
I20260709 12:24:11.452472 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.421373 27896 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.392102 27702 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:24:11.384753 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2167) took 1392 ms. Trace:
I20260709 12:24:11.404268 27889 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.402099 27766 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.471558 27953 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:11.431751 26202 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:11.531625 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.432116 26052 rpcz_store.cc:276] 0709 12:24:08.866288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.615577 (+749289us) service_pool.cc:224] Handling call
0709 12:24:09.615899 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.400494 (+784595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.386909 (+986415us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b540022b354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:11.387079 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788709760}
I20260709 12:24:11.435587 26784 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:11.445240 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55770.8us
I20260709 12:24:11.448602 27878 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.448681 27823 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:11.425598 27748 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.456192 27663 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.534759 27663 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:11.535223 27663 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.535334 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.535859 27748 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.447144 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.454111 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.449489 27675 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:11.454975 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:11.542011 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 87098.7us
I20260709 12:24:11.448698 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.456533 26122 rpcz_store.cc:276] 0709 12:24:09.992864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.465815 (+472951us) service_pool.cc:224] Handling call
0709 12:24:10.466098 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.594851 (+128753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.594890 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.594935 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.594967 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.384534 (+789567us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:11.384578 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.384700 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295566080}
I20260709 12:24:11.545598 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.547367 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.331562 27864 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.405443 27933 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.462668 26203 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:11.500422 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:11.553418 26200 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:11.513578 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.449353 27856 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.458405 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 194) took 1109 ms. Trace:
I20260709 12:24:11.537356 26127 raft_consensus.cc:2376] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4. Re-sending same reply.
I20260709 12:24:11.565927 26054 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:11.566627 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.454945 26050 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.458806 27798 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:11.460922 27802 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.539750 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.574216 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.574903 26121 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:11.574983 26048 raft_consensus.cc:2376] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5. Re-sending same reply.
I20260709 12:24:11.549405 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:11.550662 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 102012us
I20260709 12:24:11.576965 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.543802 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:11.454728 27934 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
W20260709 12:24:11.555050 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.578270 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2171) took 1582 ms. Trace:
I20260709 12:24:11.580418 26124 rpcz_store.cc:276] 0709 12:24:09.996033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.377308 (+381275us) service_pool.cc:224] Handling call
0709 12:24:11.578082 (+1200774us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54001f86d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:11.578212 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2281176960}
I20260709 12:24:11.581622 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.557983 27930 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.328135 27758 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:24:11.573792 27945 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:24:11.568406 26054 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:11.563417 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.531183 26126 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.561626 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:11.542747 27953 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.559598 27675 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.456681 27685 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.572665 27893 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.561419 27748 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.595749 27786 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.449131 27772 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.573853 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.584276 26127 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:11.586517 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.576082 26125 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:11.565605 26049 rpcz_store.cc:276] 0709 12:24:10.349246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.392132 (+ 42886us) service_pool.cc:224] Handling call
0709 12:24:11.456293 (+1064161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:11.553308 27929 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:11.473819 27766 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:11.597277 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.651875 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:11.561924 26120 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:11.574178 27949 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:24:11.561275 27545 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.570569 27890 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.607026 26124 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:11.607200 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:11.662367 26052 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:11.662602 27954 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:24:11.574437 27820 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.608836 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.666119 26048 raft_consensus.cc:2376] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4. Re-sending same reply.
I20260709 12:24:11.576803 27798 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:11.610378 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:11.611722 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 139468us
I20260709 12:24:11.669518 26051 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:11.584020 27900 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.613864 27934 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:24:11.573490 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:11.615087 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 79815.8us
I20260709 12:24:11.680315 27934 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.615590 26054 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:11.649585 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2151) took 2712 ms. Trace:
I20260709 12:24:11.595893 27758 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:11.682582 27758 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.599395 27750 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:11.647007 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.683756 27758 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.684675 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.685894 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.686441 26125 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:11.588872 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:11.595494 27883 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:24:11.646869 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.688037 26127 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:11.688189 27896 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.688459 26787 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:11.688908 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.689118 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.647030 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.654686 26129 raft_consensus.cc:2368] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4.
I20260709 12:24:11.659897 27766 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.700208 27883 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.701635 27883 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.702816 27883 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.703197 27893 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.700306 27578 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:11.704802 27578 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:11.705250 27578 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.683764 27750 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.709877 27663 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.713131 26051 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.712093 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:11.666260 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80424.8us
W20260709 12:24:11.608264 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 161158us
I20260709 12:24:11.668556 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.717635 26122 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.606240 27865 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:24:11.616744 27786 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:11.680747 27934 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:11.680789 27798 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.681293 26123 rpcz_store.cc:276] 0709 12:24:08.902007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.911403 (+  9396us) service_pool.cc:224] Handling call
0709 12:24:08.958382 (+ 46979us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.438244 (+479862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.443003 (+  4759us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.205869 (+762866us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:10.206388 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.393169 (+186781us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.740759 (+347590us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.745937 (+  5178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.764989 (+ 19052us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.330354 (+565365us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:11.603839 (+273485us) raft_consensus.cc:1685] finished
0709 12:24:11.606526 (+  2687us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.614419 (+  7893us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":238417,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":28120,"raft.queue_time_us":287139,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":58520,"spinlock_wait_cycles":236073984,"thread_start_us":788022,"threads_started":3,"wal-append.queue_time_us":248141}
I20260709 12:24:11.609248 27953 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.658424 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:11.748282 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 89979.8us
W20260709 12:24:11.695317 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 106529us
I20260709 12:24:11.750056 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:11.719084 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:11.664927 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 212500us
I20260709 12:24:11.679908 27862 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.745419 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.704141 27883 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.689316 27545 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.761411 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.752029 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.765864 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:11.767000 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:11.770347 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 100279us
I20260709 12:24:11.733831 27786 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.771885 26195 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:11.777000 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.734035 27683 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.784762 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.738966 27750 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.783087 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.771464 27893 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:11.763597 27905 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:11.756716 27862 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:11.812141 26122 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:11.779523 27632 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:11.789132 27588 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:11.814409 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:11.775177 27962 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.777791 27889 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.758154 27589 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:11.819471 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:11.824452 27967 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.823827 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.826586 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.790657 27900 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:11.827243 27875 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:11.812873 27957 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:11.828307 27969 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:11.780577 27739 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:24:11.798657 27905 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.819507 27632 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.833182 26122 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:11.801723 27896 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:11.816102 27750 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:11.841454 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 91538.8us
I20260709 12:24:11.804561 27797 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:11.830044 27802 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.819481 27683 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:11.840015 27748 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:11.832312 27864 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.886070 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.857723 27905 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.837250 27900 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.838006 27961 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.841121 27938 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:11.848393 27758 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.899793 27938 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:11.900192 27773 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:11.872520 27959 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:11.900817 27773 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:11.900885 27938 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.902396 27938 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.904412 27896 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:11.878479 26122 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:11.905479 27896 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.851775 27656 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.906914 27656 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.907326 27772 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.855679 27969 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.910938 27969 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.913710 27970 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:11.914512 27970 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.877583 27802 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:11.916554 27802 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.917860 27632 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.885284 27683 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.922199 27683 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:11.859781 27663 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.798079 27934 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.923081 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.924132 26125 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:11.889210 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71029.3us
W20260709 12:24:11.893352 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108668us
I20260709 12:24:11.893491 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.879627 27960 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:24:11.887928 27766 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.898041 27900 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.844883 27962 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:11.885565 27957 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:11.864593 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:11.928099 26196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:11.915427 27970 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.880800 27958 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.937918 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:11.953248 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:11.955694 27663 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:11.956394 27663 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:11.957007 26052 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.957376 27663 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.959425 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.930217 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:11.905516 27868 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.003s
I20260709 12:24:11.959324 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.961042 26051 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:11.960731 27868 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:11.964668 26046 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:11.964974 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 100481us
I20260709 12:24:11.922231 27964 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.977739 26203 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:11.977964 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.978940 26122 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:11.980091 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.981323 26048 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:11.983183 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.985805 26048 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:11.877202 27797 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.963912 27868 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:11.988763 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.992206 26122 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:11.856524 27862 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.938119 27959 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.887290 27931 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:24:11.988873 26051 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:11.987865 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:11.859104 27545 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.943574 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.994414 26048 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.996625 26122 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:12.004508 26194 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:12.004791 27957 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:12.005997 27798 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.008095 27967 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.008664 27967 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:11.998287 27959 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.009754 27967 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:12.009958 27962 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:12.010793 27937 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.654s	user 0.004s	sys 0.001s
I20260709 12:24:12.011137 27959 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:12.013597 27797 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.014544 27797 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.008980 27589 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:12.008683 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.997964 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:11.928908 27847 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.017747 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:12.071614 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:11.990514 27656 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:12.061440 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52837.1us
I20260709 12:24:12.077623 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.072986 26122 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:12.066859 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 120073us
I20260709 12:24:11.942706 27972 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:24:12.089149 26791 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Advancing to term 3
I20260709 12:24:11.966157 27926 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:24:11.991010 27578 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.062837 27862 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.967275 27878 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:12.109836 26201 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:12.064395 27974 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:24:12.100626 26124 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.099576 27981 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.070785 27969 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.075565 27588 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.106320 26199 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:12.098615 26785 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.051286 27797 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:11.996196 27976 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 12:24:11.998487 27964 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.129266 26123 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:12.151944 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:11.993891 27958 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.157578 26048 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:12.172691 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.132957 27798 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:12.133579 27588 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:12.178591 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.139065 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:12.183938 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:12.178380 27931 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.173882 27896 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:12.192481 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.187359 27896 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:12.182600 27968 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:24:12.152793 27739 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:12.178611 27656 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.175920 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:12.200982 27971 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.212391 27971 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.200721 27663 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.221167 27971 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.220481 27930 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:24:12.222950 27930 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.223503 27930 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:12.224573 27588 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.225633 27588 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.225836 27958 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:12.226747 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:12.227428 26199 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.229220 27578 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.202124 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.216315 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.199957 27931 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.244632 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.245528 26790 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:12.247421 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.248217 26790 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
W20260709 12:24:12.250736 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 153069us
I20260709 12:24:12.251564 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.251914 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.252570 26790 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:12.253059 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:12.253803 27926 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.254413 27926 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:12.254869 27926 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:12.255358 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.256441 27956 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.256946 27956 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.257407 27956 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.255712 27926 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.270007 27956 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.272920 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:12.274161 27750 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:12.274607 27750 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:12.275031 27750 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.276783 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:12.277658 27961 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.278074 27961 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.278419 27961 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.275938 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:12.283273 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.284045 27976 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.284466 27976 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.284816 27976 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.287405 27961 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.287150 27750 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:12.291424 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 477066us
I20260709 12:24:12.292613 26094 leader_election.cc:400] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:12.293077 26094 leader_election.cc:403] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:12.293738 27683 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:12.294416 27864 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:12.295336 27864 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:12.295683 27864 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:12.296541 27864 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:12.297986 27864 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.299306 27961 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.300359 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.301406 27683 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.302006 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.303404 27683 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.303689 26094 leader_election.cc:400] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:12.305040 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.306173 27823 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:12.306850 27927 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.306855 27823 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:12.308954 27683 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.308547 27823 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.309441 27683 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.312143 27683 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.316092 27683 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.316699 27683 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.321717 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.324858 27956 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.327973 27959 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:12.329162 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45975.9us
I20260709 12:24:12.202606 27772 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:12.340636 27966 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:24:12.204815 27960 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:12.218104 27766 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.347065 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.207252 27739 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.236277 27854 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:12.238498 27868 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.354014 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:12.356595 26169 leader_election.cc:400] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4.
I20260709 12:24:12.362967 26169 leader_election.cc:403] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:12.352886 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 272236us
I20260709 12:24:12.363458 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.334261 27578 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:24:12.334692 27896 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:12.364801 26048 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:12.366112 26094 leader_election.cc:400] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:12.354929 27959 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:12.431182 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 74811.6us
I20260709 12:24:12.422590 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.429270 26094 leader_election.cc:403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:12.359344 27938 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:12.430306 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.412992 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.381667 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:12.446800 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80859.4us
I20260709 12:24:12.448680 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.450376 26198 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:12.364787 27772 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:12.452111 26196 raft_consensus.cc:606] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:12.452766 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 214624us
I20260709 12:24:12.236609 27930 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.365458 27847 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:12.355979 27854 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:12.523685 27966 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.524454 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.355211 27656 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.163s
I20260709 12:24:12.371348 27958 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.162s
I20260709 12:24:12.450625 27750 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.576606 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:12.459774 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:12.362310 27931 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.191s
I20260709 12:24:12.431703 27959 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:12.547195 27578 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:12.518204 27683 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:12.462198 27878 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:12.521955 27896 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4.
I20260709 12:24:12.552284 27868 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:12.581670 27773 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.003s
I20260709 12:24:12.554198 27772 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:12.460500 27972 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.564139 27938 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:24:12.570683 27964 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.576617 27865 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:12.644162 27802 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.727s	user 0.006s	sys 0.006s
I20260709 12:24:12.602214 27992 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.548213 27966 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.594262 27797 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.599574 27847 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.599298 27750 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.598692 27578 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:12.701143 27578 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.702529 27578 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.702594 27675 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.703799 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2175) took 2704 ms. Trace:
I20260709 12:24:12.568572 27663 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.643776 27930 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:12.705787 27997 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:24:12.707149 27987 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:24:12.603305 26049 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:12.645197 27683 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.602784 27892 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:12.714499 27892 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:12.710917 27993 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:24:12.708205 26126 rpcz_store.cc:276] 0709 12:24:09.999904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.605916 (+606012us) service_pool.cc:224] Handling call
0709 12:24:10.717177 (+111261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.608378 (+891201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.703497 (+1095119us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54001c4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:12.703728 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661328640}
I20260709 12:24:12.728782 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:12.729238 27953 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.957s	user 0.009s	sys 0.001s
W20260709 12:24:12.729628 27883 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.966s	user 0.007s	sys 0.004s
I20260709 12:24:12.730261 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:12.730046 27772 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.731272 27772 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.731393 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:12.733451 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:12.743173 26054 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
W20260709 12:24:12.015611 27889 consensus_queue.cc:1175] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 4 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:12.715372 27892 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:11.924386 27893 consensus_queue.cc:1175] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 4 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:24:12.722174 27786 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.932s	user 0.008s	sys 0.001s
I20260709 12:24:12.722450 26199 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:12.734552 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:12.735865 27862 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.002s
W20260709 12:24:12.736287 27588 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 12:24:12.791250 26121 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:12.738917 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:12.739785 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:12.740618 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:12.739849 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:12.741551 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1478) took 1358 ms. Trace:
I20260709 12:24:12.797171 26125 rpcz_store.cc:276] 0709 12:24:11.382681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.696320 (+313639us) service_pool.cc:224] Handling call
0709 12:24:11.923944 (+227624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.924613 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.741326 (+816713us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540013f450. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:12.741486 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522753408}
I20260709 12:24:12.798713 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:12.799767 26125 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:24:12.721679 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1011) took 1372 ms. Trace:
I20260709 12:24:12.800209 26050 rpcz_store.cc:276] 0709 12:24:11.349550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.383532 (+ 33982us) service_pool.cc:224] Handling call
0709 12:24:11.454781 (+ 71249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.568169 (+113388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.721463 (+1153294us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54000f8550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:12.721605 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163114368}
I20260709 12:24:12.748477 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:12.744532 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:12.755991 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:12.647921 27773 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.647022 27854 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:12.716775 27875 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.872s	user 0.012s	sys 0.015s
W20260709 12:24:12.748278 27900 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.000s
I20260709 12:24:12.797158 26789 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
W20260709 12:24:12.805495 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.809000 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:12.817814 26125 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:12.810058 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 361483us
I20260709 12:24:12.756716 26049 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:12.709554 27959 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:12.816246 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.816167 26783 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.672649 27802 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:12.826666 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.841696 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.716671 27847 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.844970 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:12.791018 27974 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4.
I20260709 12:24:12.813933 27739 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:12.863284 26125 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:12.873915 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:12.856292 27926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.016s	sys 0.047s
I20260709 12:24:12.872362 27959 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:12.874949 27926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.016s	sys 0.047s
W20260709 12:24:12.866631 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.820736 27875 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.696923 27992 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.847087 26126 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:12.831295 27854 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.878924 26125 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:12.801159 27786 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:12.885026 26125 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:12.747669 27683 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:12.729631 27953 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.864195 27750 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.825295 27900 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.876709 27748 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.923287 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:12.933329 26047 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:12.873974 27878 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:12.939360 26128 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:12.901683 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:12.935369 27976 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.650s	user 0.006s	sys 0.006s
W20260709 12:24:12.938203 26123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.015s	sys 0.042s
W20260709 12:24:12.954305 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.957975 27683 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:12.950007 26124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.000s	sys 0.014s
W20260709 12:24:12.959988 26124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.000s	sys 0.014s
I20260709 12:24:12.963754 27758 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.964206 27758 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:12.965066 27758 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:12.966341 27758 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.936720 28002 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.903506 27750 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:12.953271 26123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.015s	sys 0.042s
I20260709 12:24:12.932475 27862 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:24:12.950362 27748 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:12.972754 27942 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.990705 28002 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:12.985663 27992 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:12.994403 26126 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:12.991014 27878 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:12.994705 27966 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.006155 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:12.957051 27976 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.015918 27878 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.003754 27773 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:13.007937 27962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.011s	sys 0.019s
I20260709 12:24:12.997176 28002 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.019981 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:13.027480 27962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.012s	sys 0.020s
I20260709 12:24:13.007943 27989 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:24:13.058923 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 482373us
I20260709 12:24:13.022362 28012 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:13.065130 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:13.033679 27847 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.070151 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:13.070953 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51111.1us
I20260709 12:24:13.067214 26120 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:13.057309 27786 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:13.074244 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.069202 27900 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.095240 27750 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:13.097774 27905 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.171s	user 0.007s	sys 0.005s
W20260709 12:24:13.121191 28003 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:24:13.121537 27905 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.099098 28008 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:13.099618 26126 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:13.099881 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1387) took 1742 ms (client timeout 1550 ms). Trace:
W20260709 12:24:13.124547 28009 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:24:13.124873 26195 rpcz_store.cc:269] 0709 12:24:11.369486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.369649 (+   163us) service_pool.cc:224] Handling call
0709 12:24:13.099615 (+1729966us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.099782 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826318720}
I20260709 12:24:13.125687 28008 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.126094 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:13.126323 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.126546 27953 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.126816 26195 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.127462 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:13.128165 26124 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:13.130690 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:13.111416 28012 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:13.072192 27545 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.077504 27702 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:13.103814 27976 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.054744 27748 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.104436 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 223) took 1215 ms. Trace:
I20260709 12:24:13.141083 26121 rpcz_store.cc:276] 0709 12:24:11.889499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.892989 (+  3490us) service_pool.cc:224] Handling call
0709 12:24:13.104047 (+1211058us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.104332 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1989412096}
I20260709 12:24:13.142957 26121 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:13.143172 27878 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:13.143927 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.105825 26125 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:24:13.144379 26125 pending_rounds.cc:85] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 4
I20260709 12:24:13.144626 27981 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:13.145002 26199 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:13.145615 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 996) took 1213 ms. Trace:
W20260709 12:24:13.145787 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.145982 26199 rpcz_store.cc:276] 0709 12:24:11.932094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.852195 (+920101us) service_pool.cc:224] Handling call
0709 12:24:13.145551 (+293356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.147006 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.145013 27981 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:13.148394 27656 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.148721 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.148523 27981 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.148990 27656 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.149400 26199 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:13.149777 27981 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.150703 28012 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:13.151113 27656 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.075611 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:13.108523 28004 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:24:13.110738 28006 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:24:13.160472 27905 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:13.111980 28005 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:13.112342 27847 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:13.170032 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 232) took 1048 ms. Trace:
I20260709 12:24:13.170982 26128 rpcz_store.cc:276] 0709 12:24:12.121379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.145746 (+ 24367us) service_pool.cc:224] Handling call
0709 12:24:12.188739 (+ 42993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.942008 (+753269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.169807 (+227799us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54001d1fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:13.169977 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725448576}
I20260709 12:24:13.114482 27685 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.115715 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.125555 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:13.126133 28008 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:13.102270 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1391) took 1715 ms. Trace:
I20260709 12:24:13.137529 26124 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:13.133008 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 237484us
W20260709 12:24:13.138013 27996 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:24:13.138170 26195 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:13.127566 26783 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:13.138098 26786 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:13.144646 26125 pending_rounds.cc:107] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:13.152935 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.184444 26125 raft_consensus.cc:2889] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.184589 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:13.153215 26121 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:13.153542 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78001.5us
I20260709 12:24:13.152398 27758 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.151731 28012 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.156303 27878 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:13.158782 27931 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:13.153404 27702 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.194177 27772 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:24:13.164942 27798 raft_consensus.cc:993] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:24:13.165510 27972 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:13.169967 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.170343 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 279) took 1312 ms. Trace:
I20260709 12:24:13.189558 26126 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.178241 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:13.180032 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 992) took 1249 ms (client timeout 1574 ms). Trace:
I20260709 12:24:13.179811 26201 rpcz_store.cc:276] 0709 12:24:11.386699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.389903 (+  3204us) service_pool.cc:224] Handling call
0709 12:24:11.390186 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.116555 (+726369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.101919 (+985364us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:13.102178 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654358400}
I20260709 12:24:13.210187 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.182197 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.186343 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:13.184734 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.211628 26195 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:13.212937 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.213630 26195 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.194346 27758 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:13.217859 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.208418 26202 rpcz_store.cc:269] 0709 12:24:11.930897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.931013 (+   116us) service_pool.cc:224] Handling call
0709 12:24:13.115557 (+1184544us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:13.179966 (+ 64409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30983936}
I20260709 12:24:13.193897 27976 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:13.197116 27545 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:13.195537 27982 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:24:13.191979 27905 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:13.198787 27878 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:13.199855 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.205542 26200 rpcz_store.cc:276] 0709 12:24:11.856919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.857064 (+   145us) service_pool.cc:224] Handling call
0709 12:24:11.857420 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.015524 (+158104us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:12.059987 (+ 44463us) log.cc:844] Serialized 14 byte log entry
0709 12:24:12.236756 (+176769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.236812 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:12.236865 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.168006 (+931141us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400335c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:13.168049 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.168170 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44269,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":7576,"raft.queue_time_us":104,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":95653,"spinlock_wait_cycles":363133312,"thread_start_us":176358,"threads_started":1,"wal-append.queue_time_us":305329}
I20260709 12:24:13.227571 26195 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:13.238389 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.239812 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.161201 28016 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.200534 27702 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:13.218259 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:13.191151 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.191970 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:13.240500 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.207895 27990 raft_consensus.cc:1081] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:24:13.207629 27972 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.241489 26791 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:13.238662 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.223414 27905 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.276413 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.242205 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.277585 26202 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:13.220474 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.251832 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:13.223184 27854 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.219065 27890 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.240366 26195 raft_consensus.cc:2376] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4. Re-sending same reply.
I20260709 12:24:13.219252 27856 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:13.223366 27750 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.271097 27966 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:13.280730 28016 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:13.286711 27750 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.279150 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.223006 27953 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:24:13.273865 26199 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.233505 28008 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.289717 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.236860 27878 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:13.279834 27905 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:13.284914 27957 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:13.286787 27758 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.289297 27750 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.309127 27750 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:13.309548 27750 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.310182 28008 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.310461 27750 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.311898 27750 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.304536 26201 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:13.316457 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.303627 26789 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:13.321370 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.324898 26127 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:13.326099 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:13.333774 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:13.325376 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.339177 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.339780 26201 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:13.299242 27966 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.340674 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.341331 26200 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:13.289563 27890 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.348848 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.350126 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.332132 27862 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.360034 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.360881 26200 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:13.363099 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.335995 27957 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.366716 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.367332 26201 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:13.367005 27957 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.367830 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 288) took 1124 ms. Trace:
I20260709 12:24:13.368188 26201 rpcz_store.cc:276] 0709 12:24:12.243058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.366471 (+1123413us) service_pool.cc:224] Handling call
0709 12:24:13.367764 (+  1293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.369081 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.372531 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.344702 27966 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.376806 28027 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:13.336758 27953 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:13.377048 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.355933 27685 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.356101 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:13.381618 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.315526 27878 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.343485 28016 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.387844 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:13.387367 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48812.9us
I20260709 12:24:13.386461 27748 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 12:24:13.407128 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.384716 27900 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.391047 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1006) took 2067 ms. Trace:
I20260709 12:24:13.440052 27900 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:13.440559 26046 rpcz_store.cc:276] 0709 12:24:11.323601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.324406 (+   805us) service_pool.cc:224] Handling call
0709 12:24:11.324753 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.965143 (+640390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.380648 (+1415505us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b540022c254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:13.387802 (+  7154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471024384}
W20260709 12:24:13.437544 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65545.1us
I20260709 12:24:13.442636 27900 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.438078 26201 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:13.364158 27862 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:13.438419 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49003us
I20260709 12:24:13.439242 28029 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:13.436684 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.385661 28027 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:13.444613 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52589.9us
I20260709 12:24:13.443747 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 281) took 1211 ms. Trace:
I20260709 12:24:13.446324 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:13.388856 26123 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:13.449777 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:13.450510 26123 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:13.450590 28023 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:24:13.449766 28024 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:24:13.453015 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:13.453596 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:13.454139 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 602) took 1097 ms. Trace:
I20260709 12:24:13.454576 26123 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 1.
I20260709 12:24:13.454802 27932 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:13.455312 27932 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.455911 27932 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.446193 26201 rpcz_store.cc:276] 0709 12:24:12.232335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.388780 (+1156445us) service_pool.cc:224] Handling call
0709 12:24:13.435886 (+ 47106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.443453 (+  7567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.443691 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.385136 27685 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.467401 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.436676 27856 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.389191 28025 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:24:13.494194 28026 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:13.494575 26201 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:13.500924 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 283) took 1267 ms. Trace:
I20260709 12:24:13.501315 26201 rpcz_store.cc:276] 0709 12:24:12.233626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.492800 (+1259174us) service_pool.cc:224] Handling call
0709 12:24:13.494241 (+  1441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.500645 (+  6404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.500868 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.502321 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.500969 27890 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.503914 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 285) took 1263 ms. Trace:
I20260709 12:24:13.458629 27748 consensus_queue.cc:1243] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:13.466001 27900 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 12:24:13.504705 28032 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:24:13.504714 27890 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:13.505379 28019 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:24:13.506067 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:13.462612 28027 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:24:13.454610 26782 rpcz_store.cc:276] 0709 12:24:12.356672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.360111 (+  3439us) service_pool.cc:224] Handling call
0709 12:24:13.155644 (+795533us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:13.155800 (+   156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.181950 (+ 26150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.185631 (+  3681us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.207457 (+ 21826us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.298100 (+ 90643us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:13.299602 (+  1502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.317864 (+ 18262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.453818 (+135954us) raft_consensus.cc:1685] finished
0709 12:24:13.453852 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.454073 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14844,"raft.queue_time_us":24939,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":11249,"spinlock_wait_cycles":18225920,"thread_start_us":57541,"threads_started":1,"wal-append.queue_time_us":119337}
I20260709 12:24:13.510052 27685 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.511140 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.511935 27953 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:13.512682 27856 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.469103 28000 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.518160 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.519896 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.520051 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.517159 26201 rpcz_store.cc:276] 0709 12:24:12.240104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.503162 (+1263058us) service_pool.cc:224] Handling call
0709 12:24:13.503456 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.503624 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.503649 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:13.503691 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.503724 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.503775 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.503867 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.522537 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:13.529881 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:13.516192 28026 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.520905 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.522200 28029 consensus_queue.cc:1243] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:13.531180 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.532351 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.532166 28000 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.533087 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.518020 28016 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.537312 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.538129 26054 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:13.519213 27683 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:24:13.540573 28030 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.540729 28027 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:13.525324 27890 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.546379 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:13.548578 27972 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.549713 28026 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.554872 27956 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:13.615171 27956 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.615482 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.616204 27956 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.617611 27956 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.620446 27702 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.621310 28000 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.621976 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44736 (request call id 21) took 1694 ms. Trace:
I20260709 12:24:13.622370 26196 rpcz_store.cc:276] 0709 12:24:11.927830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.927964 (+   134us) service_pool.cc:224] Handling call
0709 12:24:13.621772 (+1693808us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.621856 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173202688}
I20260709 12:24:13.622848 27702 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.623659 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.561297 27966 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.656490 26196 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:13.657501 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.658273 26196 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:13.562559 27932 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.659221 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.565941 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:13.569124 27974 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.663677 27974 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:13.569936 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:13.666918 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 97045us
I20260709 12:24:13.668111 27930 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.668720 27930 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.670179 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.671190 27930 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.671778 27930 raft_consensus.cc:2764] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:13.672292 27959 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:13.672739 27959 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.673110 27959 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.673273 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.674686 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.676589 27958 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:13.677840 27958 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:13.678763 27758 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:13.679230 27758 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:13.680353 27758 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.681972 27758 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:13.683988 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.688761 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.689510 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:13.691125 27900 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:13.691588 27900 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:13.693005 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 991) took 1763 ms (client timeout 1705 ms). Trace:
I20260709 12:24:13.693555 26200 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:13.693442 26203 rpcz_store.cc:269] 0709 12:24:11.929858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.929965 (+   107us) service_pool.cc:224] Handling call
0709 12:24:13.692758 (+1762793us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.692929 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38116096}
I20260709 12:24:13.695014 26203 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:13.695113 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 280) took 1477 ms. Trace:
I20260709 12:24:13.695685 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1394) took 1138 ms. Trace:
I20260709 12:24:13.695498 26200 rpcz_store.cc:276] 0709 12:24:12.218084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.385695 (+1167611us) service_pool.cc:224] Handling call
0709 12:24:13.693361 (+307666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.694065 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.694925 (+   860us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400301450. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.695054 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603158784}
I20260709 12:24:13.691989 27900 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.688676 26790 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:13.559614 26199 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:13.574223 27984 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.229s	user 0.001s	sys 0.002s
I20260709 12:24:13.696861 27942 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:13.721165 27942 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.696219 26203 rpcz_store.cc:276] 0709 12:24:12.557301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.694418 (+1137117us) service_pool.cc:224] Handling call
0709 12:24:13.694791 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.695420 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.695634 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.699342 26046 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:13.707248 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 993) took 1776 ms. Trace:
I20260709 12:24:13.729569 26198 rpcz_store.cc:276] 0709 12:24:11.931205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.931304 (+    99us) service_pool.cc:224] Handling call
0709 12:24:11.952109 (+ 20805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.450887 (+498778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.706970 (+1256083us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:13.707182 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1145095936}
W20260709 12:24:13.730224 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.576781 27802 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.578513 26197 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:13.582604 27934 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.590751 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 230) took 1581 ms. Trace:
I20260709 12:24:13.748484 26120 rpcz_store.cc:276] 0709 12:24:12.009231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.116165 (+106934us) service_pool.cc:224] Handling call
0709 12:24:12.180962 (+ 64797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.077521 (+896559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.590468 (+512947us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:13.590656 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025639680}
I20260709 12:24:13.749981 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:13.750818 26120 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:13.752571 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.756136 27748 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.756709 27748 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.757601 26194 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
W20260709 12:24:13.758235 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 995) took 1826 ms (client timeout 1910 ms). Trace:
I20260709 12:24:13.758889 27748 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.590902 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:13.759672 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 168837us
I20260709 12:24:13.591823 27981 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:13.592077 27931 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:13.596771 26048 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:13.596817 26052 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:13.600618 27914 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:13.608961 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.596704 27953 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.607687 28027 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:13.613217 28030 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.769985 28030 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.767580 28027 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.771199 28027 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.627982 27890 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.634596 26787 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:13.645138 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2186) took 2481 ms (client timeout 3258 ms). Trace:
I20260709 12:24:13.652607 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:13.655229 27972 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.708595 27862 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:13.661921 27975 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.632033 25949 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 283) took 6872 ms. Trace:
I20260709 12:24:13.707223 27766 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:13.791426 27766 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:13.791697 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.763078 27748 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:13.792582 27766 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.793587 27766 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:13.793286 27748 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.795523 27748 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.796967 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:13.797775 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:13.798210 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:13.798897 26050 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:13.798929 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:13.799767 26201 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:13.789791 27889 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:13.710100 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.713053 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2234) took 2116 ms. Trace:
I20260709 12:24:13.801263 26047 rpcz_store.cc:276] 0709 12:24:11.601753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.610121 (+  8368us) service_pool.cc:224] Handling call
0709 12:24:13.712859 (+2102738us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400153454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.712985 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2905528064}
I20260709 12:24:13.802700 26047 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:13.713745 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 236) took 1398 ms. Trace:
I20260709 12:24:13.804399 26125 rpcz_store.cc:276] 0709 12:24:12.314957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.886955 (+571998us) service_pool.cc:224] Handling call
0709 12:24:13.105322 (+218367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.184990 (+ 79668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.713560 (+528570us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540013ef50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.713692 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501203840}
I20260709 12:24:13.805847 26791 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:13.807785 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:13.808547 26047 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:13.732761 26200 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:13.734289 27967 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.715173 27896 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:13.574996 27905 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.739583 27685 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:13.740123 26049 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:13.779603 27953 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:13.740455 26785 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 LEARNER]: Advancing to term 3
I20260709 12:24:13.708472 26045 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:13.741072 27683 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:13.745029 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.760993 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:13.759732 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:13.766270 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.774452 26789 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:13.772248 27914 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.762677 27931 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:13.777391 28026 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.778633 27981 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:13.780946 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:13.712564 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:13.733050 27932 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:13.783483 26127 rpcz_store.cc:269] 0709 12:24:11.174882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.688655 (+513773us) service_pool.cc:224] Handling call
0709 12:24:13.644886 (+1956231us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:13.645066 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3724802816}
W20260709 12:24:13.661309 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 95450.7us
I20260709 12:24:13.715291 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 284) took 1478 ms. Trace:
I20260709 12:24:13.785970 27900 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.760459 27938 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.713732 26126 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:13.720413 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:13.811347 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1397) took 1253 ms. Trace:
I20260709 12:24:13.800052 26050 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:13.813375 26196 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:13.568938 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:13.773010 27927 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:13.773202 27878 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:13.763413 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 990) took 1834 ms (client timeout 1893 ms). Trace:
W20260709 12:24:13.758621 26194 rpcz_store.cc:269] 0709 12:24:11.931781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.005861 (+ 74080us) service_pool.cc:224] Handling call
0709 12:24:13.510971 (+1505110us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:13.758156 (+247185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36328192}
I20260709 12:24:13.666406 27974 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:13.788080 27893 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.800737 27889 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:13.824397 27967 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:13.814818 27748 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:13.848757 26788 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:24:13.852532 26202 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:13.870766 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:13.815106 27966 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:13.874327 26790 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:13.821802 27930 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:13.824046 27896 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:13.826900 27685 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:14.007122 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.235754 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 666844us
W20260709 12:24:13.960839 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.236611 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:14.167420 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:13.933440 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173775us
W20260709 12:24:14.171936 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.241499 26200 rpcz_store.cc:276] 0709 12:24:12.558418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.696658 (+1138240us) service_pool.cc:224] Handling call
0709 12:24:13.732593 (+ 35935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.811017 (+ 78424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.811272 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:13.893785 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 132869us
I20260709 12:24:14.243875 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:13.789294 25949 rpcz_store.cc:276] 0709 12:24:06.793706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.823931 (+ 30225us) service_pool.cc:224] Handling call
0709 12:24:13.631956 (+6808025us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:08.714491 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:08.723970 (+  9479us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:08.724007 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:24:08.724026 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:08.724708 (+   682us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:08.724782 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:08.724825 (+    43us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:24:08.754360 (+ 29535us) tablet.cc:701] Row locks acquired
0709 12:24:08.754398 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:24:08.837843 (+ 83445us) write_op.cc:270] Start()
0709 12:24:08.850784 (+ 12941us) write_op.cc:276] Timestamp: P: 1783599848830677 usec, L: 0
0709 12:24:08.850822 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:24:08.965098 (+114276us) log.cc:844] Serialized 1696 byte log entry
0709 12:24:10.070453 (+1105355us) op_driver.cc:464] REPLICATION: finished
0709 12:24:10.335473 (+265020us) write_op.cc:301] APPLY: starting
0709 12:24:10.335554 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:10.335735 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:10.335849 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:10.336569 (+   720us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:10.336803 (+   234us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:24:10.336861 (+    58us) write_op.cc:312] APPLY: finished
0709 12:24:10.337125 (+   264us) log.cc:844] Serialized 41 byte log entry
0709 12:24:10.369099 (+ 31974us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:10.383823 (+ 14724us) write_op.cc:454] Released schema lock
0709 12:24:10.387715 (+  3892us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:10.414689 (+ 26974us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:10.414753 (+    64us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":580994,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":264754,"apply.run_cpu_time_us":2642,"apply.run_wall_time_us":79122,"num_ops":4,"prepare.queue_time_us":809004,"prepare.run_cpu_time_us":32815,"prepare.run_wall_time_us":462328,"raft.queue_time_us":203930,"raft.run_cpu_time_us":15866,"raft.run_wall_time_us":79196,"replication_time_us":1217173,"thread_start_us":1081402,"threads_started":4,"wal-append.queue_time_us":478924,"wal-append.run_cpu_time_us":19119,"wal-append.run_wall_time_us":1891088}]]}
W20260709 12:24:14.248035 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80763us
I20260709 12:24:14.259238 26200 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:14.261653 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:14.264256 26050 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:14.051131 26054 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:14.234364 27893 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.267844 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1403) took 1615 ms. Trace:
I20260709 12:24:14.265415 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:14.268965 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:14.270483 26050 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:14.268447 26200 rpcz_store.cc:276] 0709 12:24:12.652109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.258716 (+1606607us) service_pool.cc:224] Handling call
0709 12:24:14.259054 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.259978 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.267785 (+  7807us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:14.271965 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.004s	sys 0.015s
I20260709 12:24:13.844165 26045 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:14.250347 27758 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:14.041255 27862 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.053804 27999 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.180161 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:13.934942 27982 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:14.177443 27892 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:14.244544 27798 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.008s	sys 0.012s
I20260709 12:24:14.324816 26201 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:14.042701 27975 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:14.242579 27927 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.245030 28016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.004s	sys 0.014s
I20260709 12:24:14.054040 27864 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.942977 27802 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.244925 27856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.029s	sys 0.072s
W20260709 12:24:14.272398 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.005s	sys 0.015s
W20260709 12:24:14.376144 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 929798us
I20260709 12:24:14.377674 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:14.162236 27980 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.873641 27981 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:14.343174 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.055846 28026 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.056260 28000 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:14.354847 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.283748 27958 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.259892 28008 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:14.285719 27974 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.285277 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2208) took 1169 ms. Trace:
W20260709 12:24:14.364259 26198 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.007s	sys 0.030s
I20260709 12:24:14.259677 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:14.266830 28029 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:14.244386 26199 rpcz_store.cc:276] 0709 12:24:12.237296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.174462 (+937166us) service_pool.cc:224] Handling call
0709 12:24:13.715227 (+540765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.055081 27942 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.233600 27748 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.367681 27798 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.008s	sys 0.012s
I20260709 12:24:14.371920 26123 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:14.365796 26783 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:14.387763 26198 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.008s	sys 0.030s
W20260709 12:24:14.240653 26197 rpcz_store.cc:269] 0709 12:24:11.929410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.937379 (+  7969us) service_pool.cc:224] Handling call
0709 12:24:13.199678 (+1262299us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:13.761882 (+562204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10610560}
W20260709 12:24:14.369772 28016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.005s	sys 0.015s
I20260709 12:24:14.359797 27932 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:14.278254 27938 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.369450 27927 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:14.375233 27856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.030s	sys 0.073s
I20260709 12:24:14.367462 27975 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.222219 27931 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.260819 27930 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.403622 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:14.352393 27862 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.363229 27982 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:14.386219 27966 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:14.429033 28002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.329s	user 0.010s	sys 0.035s
I20260709 12:24:14.389194 27970 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.359512 27999 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.429003 28027 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.000s
I20260709 12:24:14.393790 28008 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.411000 27953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.020s	sys 0.015s
W20260709 12:24:14.459138 26047 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.009s	sys 0.024s
I20260709 12:24:14.390165 28029 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:14.384672 27802 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.434348 27748 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:14.447973 27905 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:14.458811 27739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.015s	sys 0.020s
I20260709 12:24:14.389056 26126 rpcz_store.cc:276] 0709 12:24:13.108395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.125288 (+ 16893us) service_pool.cc:224] Handling call
0709 12:24:14.278001 (+1152713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.451694 27975 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:14.490758 26047 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.009s	sys 0.024s
I20260709 12:24:14.398185 27958 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.476449 27956 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.005s
I20260709 12:24:14.384893 27864 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:14.389140 28000 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.456393 27889 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.479887 27931 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:14.386981 27980 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.374912 27896 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.480367 27982 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:14.486601 28002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.382s	user 0.010s	sys 0.036s
I20260709 12:24:14.476787 27856 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.482790 26120 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:14.506669 27739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.016s	sys 0.020s
I20260709 12:24:14.452270 27932 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.493818 27953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.022s	sys 0.017s
I20260709 12:24:14.486096 28016 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.497048 28029 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.493686 28008 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:14.537704 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1401) took 1959 ms. Trace:
I20260709 12:24:14.586230 26201 rpcz_store.cc:276] 0709 12:24:12.578561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.802905 (+1224344us) service_pool.cc:224] Handling call
0709 12:24:13.823230 (+ 20325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.365064 (+541834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.537366 (+172302us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400335254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:14.537616 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152891776}
I20260709 12:24:14.499100 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:14.498929 27914 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.505765 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:14.584672 27957 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.028s	sys 0.011s
W20260709 12:24:14.597738 27957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.029s	sys 0.011s
I20260709 12:24:14.607734 27862 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.574251 27856 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.540489 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 185758us
I20260709 12:24:14.479463 27981 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.557670 28036 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.299s	user 0.000s	sys 0.002s
I20260709 12:24:14.578572 26200 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:14.580565 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 336799us
I20260709 12:24:14.560846 27893 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.496220 27970 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.574827 27896 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.591719 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:14.636178 26201 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:14.600591 27938 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.623047 27930 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.624429 28000 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.625280 27932 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.625510 27864 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.626299 27958 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:14.627645 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 393) took 2697 ms. Trace:
W20260709 12:24:14.603175 28038 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:24:14.631475 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:14.631639 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:14.621897 27766 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.828s	user 0.011s	sys 0.000s
I20260709 12:24:14.614101 27802 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.613479 27942 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:14.650774 27934 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.891s	user 0.013s	sys 0.000s
I20260709 12:24:14.673903 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:14.654045 27868 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.044s	sys 0.076s
W20260709 12:24:14.704855 28040 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:24:14.705724 27970 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:14.707166 27856 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.708761 27975 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:14.711051 27975 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:14.696485 28039 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 12:24:14.716823 27932 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:14.703184 27868 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.045s	sys 0.076s
W20260709 12:24:14.675762 27959 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.002s	user 0.007s	sys 0.006s
I20260709 12:24:14.664500 27981 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.702617 27958 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:14.746243 27930 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:14.699698 28001 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.730975 28049 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 12:24:14.725080 28043 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:24:14.720521 28055 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:14.736217 28008 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.720192 27982 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.782727 27982 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.448523 26045 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:14.830623 26046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.101s	user 0.021s	sys 0.035s
W20260709 12:24:14.832647 26048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.024s	sys 0.027s
W20260709 12:24:14.906162 26048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.024s	sys 0.028s
I20260709 12:24:14.906505 26045 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.907709 26203 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:14.834025 26052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47798
I20260709 12:24:14.909756 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 200) took 1381 ms. Trace:
I20260709 12:24:14.774590 28001 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.845070 26127 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.003s	sys 0.024s
W20260709 12:24:14.845366 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.846251 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.017s	sys 0.036s
I20260709 12:24:14.846803 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2196) took 2407 ms. Trace:
W20260709 12:24:14.847615 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.007s	sys 0.009s
I20260709 12:24:14.848868 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1400) took 2280 ms. Trace:
I20260709 12:24:14.848438 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:14.724589 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.948415 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 997) took 2061 ms. Trace:
I20260709 12:24:14.949812 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2259) took 1174 ms. Trace:
I20260709 12:24:14.905557 27685 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.950448 26047 rpcz_store.cc:276] 0709 12:24:13.775453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.809628 (+ 34175us) service_pool.cc:224] Handling call
0709 12:24:13.809994 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.810242 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:14.504792 (+694550us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.948991 (+444199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.949080 (+    89us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:14.949187 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.949325 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.949595 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":767979,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":123,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":193,"thread_start_us":1086310,"threads_started":2}
I20260709 12:24:14.947770 27766 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:14.952469 27766 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:14.951864 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2211) took 1707 ms. Trace:
I20260709 12:24:14.956394 26052 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.735181 26789 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:14.957131 26052 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:14.957257 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.008s	sys 0.009s
W20260709 12:24:14.958446 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.667021 26787 rpcz_store.cc:276] 0709 12:24:11.930321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.943005 (+ 12684us) service_pool.cc:224] Handling call
0709 12:24:12.060690 (+117685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.785833 (+1725143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.627338 (+841505us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54003101d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:14.627536 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3618965376}
I20260709 12:24:14.962541 26787 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEARNER]: Advancing to term 4
W20260709 12:24:14.962682 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 321126us
I20260709 12:24:14.727785 26788 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:14.595979 26197 raft_consensus.cc:2376] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3. Re-sending same reply.
I20260709 12:24:14.970634 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:14.858960 26199 raft_consensus.cc:2376] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2. Re-sending same reply.
W20260709 12:24:14.972532 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1022) took 1453 ms (client timeout 1784 ms). Trace:
I20260709 12:24:14.721827 27748 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:14.973079 26051 rpcz_store.cc:269] 0709 12:24:13.519076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.519583 (+   507us) service_pool.cc:224] Handling call
0709 12:24:14.972059 (+1452476us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:14.972314 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2996805376}
I20260709 12:24:14.773756 27930 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.943117 26128 rpcz_store.cc:276] 0709 12:24:12.439398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.188692 (+749294us) service_pool.cc:224] Handling call
0709 12:24:13.509335 (+320643us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:13.509534 (+   199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.509901 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.509933 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.510448 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.593317 (+ 82869us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:13.593355 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.593420 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.845719 (+1252299us) raft_consensus.cc:1685] finished
0709 12:24:14.845786 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.846464 (+   678us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2647,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":7699328,"thread_start_us":82484,"threads_started":1,"wal-append.queue_time_us":547065}
I20260709 12:24:14.978834 27930 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.869271 27758 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.750224 27927 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.980913 26121 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7.
I20260709 12:24:14.982108 27927 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.768250 27958 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.982986 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2215) took 1275 ms. Trace:
I20260709 12:24:14.983475 26121 rpcz_store.cc:276] 0709 12:24:13.707167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.720110 (+ 12943us) service_pool.cc:224] Handling call
0709 12:24:14.982611 (+1262501us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:14.982856 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665822080}
I20260709 12:24:14.871057 27856 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.984725 27856 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.985249 27958 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:14.745276 27959 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.873904 27931 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.882928 27773 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.889797 27967 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.890836 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 287) took 2648 ms. Trace:
I20260709 12:24:14.771873 27980 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:14.892659 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 282) took 2659 ms. Trace:
I20260709 12:24:14.896579 26784 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Advancing to term 4
I20260709 12:24:14.859958 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:14.901683 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 290) took 2159 ms. Trace:
W20260709 12:24:14.905509 26046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.022s	sys 0.035s
W20260709 12:24:14.841238 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.801335 28008 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.734162 28029 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:14.922472 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1399) took 2356 ms. Trace:
I20260709 12:24:14.944190 26198 rpcz_store.cc:276] 0709 12:24:12.568506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.730517 (+1162011us) service_pool.cc:224] Handling call
0709 12:24:13.741911 (+ 11394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.742106 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:14.430476 (+688370us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.847745 (+417269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.847870 (+   125us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:14.848082 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.848250 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.848526 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":761846,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":46585,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":287,"thread_start_us":1087459,"threads_started":2,"wal-append.queue_time_us":354587}
I20260709 12:24:14.949488 26194 rpcz_store.cc:276] 0709 12:24:12.868776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.329389 (+1460613us) service_pool.cc:224] Handling call
0709 12:24:14.374810 (+ 45421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.448080 (+ 73270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.451584 (+  3504us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.509251 (+ 57667us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.924227 (+414976us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:14.924371 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.924498 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.930242 (+  5744us) raft_consensus.cc:1685] finished
0709 12:24:14.930267 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.930510 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3104,"raft.queue_time_us":112353,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":15869,"thread_start_us":354083,"threads_started":1,"wal-append.queue_time_us":382113}
I20260709 12:24:14.956579 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 242) took 1755 ms. Trace:
W20260709 12:24:14.848479 26757 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.009s	sys 0.010s
I20260709 12:24:14.857628 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:14.948686 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:14.967451 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.017s	sys 0.036s
W20260709 12:24:14.967998 26127 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.004s	sys 0.024s
I20260709 12:24:14.970680 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:14.874873 27953 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.987497 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 243) took 1714 ms. Trace:
I20260709 12:24:15.046870 26128 rpcz_store.cc:276] 0709 12:24:13.270566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.977745 (+1707179us) service_pool.cc:224] Handling call
0709 12:24:14.978382 (+   637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.979400 (+  1018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.979427 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.979497 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.979625 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.984381 (+  4756us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.984614 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.957155 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 402) took 1140 ms. Trace:
I20260709 12:24:14.952064 27685 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:14.975085 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:14.993938 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:14.993446 27856 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:14.982869 26125 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:15.051045 26195 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:15.051308 26125 pending_rounds.cc:85] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 3
I20260709 12:24:15.051716 26125 pending_rounds.cc:107] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:24:15.051932 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1398) took 2485 ms. Trace:
I20260709 12:24:14.998292 27773 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.997421 26202 rpcz_store.cc:276] 0709 12:24:12.232990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.390997 (+1158007us) service_pool.cc:224] Handling call
0709 12:24:13.436981 (+ 45984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.241077 (+804096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.891976 (+650899us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:14.892431 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954999168}
I20260709 12:24:14.994817 26196 rpcz_store.cc:276] 0709 12:24:12.241993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.660170 (+1418177us) service_pool.cc:224] Handling call
0709 12:24:13.773087 (+112917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.240909 (+467822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.889897 (+648988us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b5400398d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:14.890579 (+   682us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92098304}
I20260709 12:24:15.005136 26200 rpcz_store.cc:276] 0709 12:24:12.742242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.309329 (+1567087us) service_pool.cc:224] Handling call
0709 12:24:14.309800 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.615987 (+306187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.901127 (+285140us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400396f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:14.901459 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617311616}
I20260709 12:24:14.952127 27900 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.010428 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:14.955403 26124 rpcz_store.cc:276] 0709 12:24:13.243990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.547918 (+1303928us) service_pool.cc:224] Handling call
0709 12:24:14.548395 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.553915 (+  5520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.553964 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.573115 (+ 19151us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.931382 (+358267us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:14.931467 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.931590 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.933976 (+  2386us) raft_consensus.cc:1685] finished
0709 12:24:14.934000 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.951635 (+ 17635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4889,"raft.queue_time_us":7350,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6896,"thread_start_us":84793,"threads_started":1,"wal-append.queue_time_us":85413}
I20260709 12:24:14.979797 27758 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:15.015758 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49159.8us
I20260709 12:24:15.020617 27999 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.021111 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 197) took 1497 ms. Trace:
I20260709 12:24:15.024688 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:15.025628 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.954416 27766 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:14.910853 26048 rpcz_store.cc:276] 0709 12:24:13.528686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.762614 (+233928us) service_pool.cc:224] Handling call
0709 12:24:13.763029 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.763446 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.763487 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.764033 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.907010 (+1142977us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.907752 (+   742us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.908127 (+   375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.909337 (+  1210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.909400 (+    63us) raft_consensus.cc:1685] finished
0709 12:24:14.909413 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.909642 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7269,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":979,"raft.queue_time_us":4285,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":414,"thread_start_us":1119678,"threads_started":1,"wal-append.queue_time_us":942408}
W20260709 12:24:15.031069 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.049376 28027 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.050015 28016 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.050407 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:15.052012 26125 raft_consensus.cc:2889] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.000692 28075 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:14.987018 25950 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 4, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 4 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:15.461249 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 604118us
I20260709 12:24:15.045081 26203 rpcz_store.cc:276] 0709 12:24:12.566330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.725265 (+1158935us) service_pool.cc:224] Handling call
0709 12:24:14.906450 (+1181185us) spinlock_profiling.cc:243] Waited 8.74 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:14.907001 (+   551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.912103 (+  5102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.922380 (+ 10277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20109568}
W20260709 12:24:15.134451 26757 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.010s	sys 0.010s
I20260709 12:24:15.190275 26129 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:15.450220 26050 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:15.427541 28048 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
W20260709 12:24:15.427228 28073 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 12:24:15.466876 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.005s	sys 0.030s
W20260709 12:24:15.467461 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.006s	sys 0.030s
W20260709 12:24:15.469238 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.06559e+06us
W20260709 12:24:15.425678 28065 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:24:15.361292 26053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.123s	user 0.000s	sys 0.017s
W20260709 12:24:15.474066 26053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.633s	user 0.000s	sys 0.021s
I20260709 12:24:15.477780 26201 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.478968 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 999) took 2609 ms. Trace:
W20260709 12:24:15.302062 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.052976 26047 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:15.096096 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.479677 26201 rpcz_store.cc:276] 0709 12:24:12.869401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.362293 (+2492892us) service_pool.cc:224] Handling call
0709 12:24:15.477055 (+114762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.478557 (+  1502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.478860 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.053187 27931 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.498519 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2210) took 2374 ms. Trace:
I20260709 12:24:15.498967 26125 rpcz_store.cc:276] 0709 12:24:13.124563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.981392 (+1856829us) service_pool.cc:224] Handling call
0709 12:24:14.982002 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.498232 (+516230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.498444 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:15.497340 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 399) took 1724 ms (client timeout 1960 ms). Trace:
I20260709 12:24:15.500098 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:15.500963 26198 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:15.050069 27868 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.502801 26198 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:15.502547 27868 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.503449 27931 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:15.503582 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 291) took 1939 ms. Trace:
I20260709 12:24:15.497483 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:15.450814 26052 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.052502 26195 rpcz_store.cc:276] 0709 12:24:12.566058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.709255 (+1143197us) service_pool.cc:224] Handling call
0709 12:24:15.006807 (+1297552us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:15.050498 (+ 43691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.051581 (+  1083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.051853 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56944512}
W20260709 12:24:15.505977 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.505535 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:15.218030 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.000s	sys 0.011s
I20260709 12:24:15.504813 26200 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.504156 26198 rpcz_store.cc:276] 0709 12:24:13.564578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.501835 (+1937257us) service_pool.cc:224] Handling call
0709 12:24:15.502398 (+   563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.503278 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.503495 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.503362 26197 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:15.515503 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 206) took 1091 ms. Trace:
W20260709 12:24:15.516172 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.291895 26782 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:15.176069 27878 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.789s	user 0.016s	sys 0.007s
W20260709 12:24:15.399364 28078 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 12:24:15.516081 26050 rpcz_store.cc:276] 0709 12:24:14.402441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.970043 (+567602us) service_pool.cc:224] Handling call
0709 12:24:15.494189 (+524146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:15.522831 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.001s	sys 0.017s
I20260709 12:24:15.520072 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1006) took 2584 ms. Trace:
W20260709 12:24:15.504956 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.6027e+06us
I20260709 12:24:15.052603 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.191082 26049 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:15.528153 26200 rpcz_store.cc:276] 0709 12:24:12.930324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.503765 (+2573441us) service_pool.cc:224] Handling call
0709 12:24:15.504213 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.514021 (+  9808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.514247 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:15.095750 26785 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.252s	user 0.011s	sys 0.007s
I20260709 12:24:15.045562 27748 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.527050 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1028) took 2005 ms. Trace:
I20260709 12:24:15.048045 26791 rpcz_store.cc:276] 0709 12:24:13.827009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.827207 (+   198us) service_pool.cc:224] Handling call
0709 12:24:13.827607 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.487066 (+659459us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:14.867980 (+380914us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.955561 (+ 87581us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.955654 (+    93us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:14.955789 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.956605 (+   816us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54006bf8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:14.956673 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.956909 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22188,"prepare.queue_time_us":269667,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":27982,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":30331,"spinlock_wait_cycles":1515662080,"thread_start_us":422243,"threads_started":2,"wal-append.queue_time_us":48794}
I20260709 12:24:15.079705 26194 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:15.500635 28063 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:15.539875 26789 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:15.082434 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:24:15.191514 27980 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:15.426038 28060 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.546371 27766 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.546803 26785 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:15.546931 26125 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.546603 28063 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.554082 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:15.559896 28063 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.547843 27999 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.560889 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:15.563839 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 298) took 1278 ms. Trace:
W20260709 12:24:15.499856 26786 rpcz_store.cc:269] 0709 12:24:13.772558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.785287 (+ 12729us) service_pool.cc:224] Handling call
0709 12:24:15.024285 (+1238998us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:15.497240 (+472955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12301184}
I20260709 12:24:15.564437 26202 rpcz_store.cc:276] 0709 12:24:14.275017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.524688 (+1249671us) service_pool.cc:224] Handling call
0709 12:24:15.547222 (+ 22534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.547698 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.547725 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.553356 (+  5631us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.553419 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.553498 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.553655 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.565852 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1504) took 1995 ms. Trace:
I20260709 12:24:15.292339 27953 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.536710 27959 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.533181 27856 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:15.568483 26125 rpcz_store.cc:276] 0709 12:24:13.565923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.499817 (+1933894us) service_pool.cc:224] Handling call
0709 12:24:15.519750 (+ 19933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.553678 (+ 33928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.561550 (+  7872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.573308 26051 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:15.537307 26053 rpcz_store.cc:276] 0709 12:24:13.521130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.521244 (+   114us) service_pool.cc:224] Handling call
0709 12:24:13.532339 (+ 11095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.532571 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:13.840711 (+308140us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.475709 (+1634998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.475871 (+   162us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:15.476011 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.476145 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.526971 (+ 50826us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":957706,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":1434,"raft.queue_time_us":213,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":227,"thread_start_us":1939497,"threads_started":2,"wal-append.queue_time_us":1660493}
I20260709 12:24:15.579969 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:15.577548 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:15.504578 28027 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.582229 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1038) took 1261 ms. Trace:
I20260709 12:24:15.582651 26051 rpcz_store.cc:276] 0709 12:24:14.321089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.497024 (+1175935us) service_pool.cc:224] Handling call
0709 12:24:15.582162 (+ 85138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:15.583575 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.041s	sys 0.034s
W20260709 12:24:15.584168 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.042s	sys 0.034s
I20260709 12:24:15.560892 28029 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.584021 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:15.566699 26128 raft_consensus.cc:606] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:15.547493 28075 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.572503 27878 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.564545 27980 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:15.577803 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 998) took 2698 ms. Trace:
I20260709 12:24:15.588701 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.600947 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.602051 26195 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:15.599922 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1037) took 1274 ms. Trace:
I20260709 12:24:15.577348 27900 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:15.602561 26047 rpcz_store.cc:276] 0709 12:24:14.320742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.578592 (+1257850us) service_pool.cc:224] Handling call
0709 12:24:15.595494 (+ 16902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.602903 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:15.602440 26787 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.639s	user 0.014s	sys 0.000s
I20260709 12:24:15.560950 27999 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.611761 27962 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:15.611699 27900 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.613268 26125 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:15.616878 26051 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:24:15.619292 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.620450 27959 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:15.597222 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 613) took 1858 ms. Trace:
I20260709 12:24:15.627832 26047 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.628968 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1506) took 2060 ms. Trace:
I20260709 12:24:15.626117 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:15.629377 26125 rpcz_store.cc:276] 0709 12:24:13.568504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.597117 (+2028613us) service_pool.cc:224] Handling call
0709 12:24:15.612910 (+ 15793us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.628668 (+ 15758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.628912 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.630378 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1035) took 1360 ms. Trace:
I20260709 12:24:15.608134 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1020) took 1267 ms. Trace:
I20260709 12:24:15.633284 26051 rpcz_store.cc:276] 0709 12:24:14.265297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.586483 (+1321186us) service_pool.cc:224] Handling call
0709 12:24:15.616596 (+ 30113us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.625269 (+  8673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.625492 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.579099 28016 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.636055 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1036) took 1312 ms. Trace:
I20260709 12:24:15.638096 26125 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:15.641783 26051 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:15.642418 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 205) took 1240 ms. Trace:
I20260709 12:24:15.642725 26051 rpcz_store.cc:276] 0709 12:24:14.402158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.637004 (+1234846us) service_pool.cc:224] Handling call
0709 12:24:15.641477 (+  4473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.642163 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.642342 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.643697 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 245) took 2049 ms. Trace:
W20260709 12:24:15.635983 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.644084 26125 rpcz_store.cc:276] 0709 12:24:13.589284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.637335 (+2048051us) service_pool.cc:224] Handling call
0709 12:24:15.637771 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.638530 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.638735 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.637032 26047 rpcz_store.cc:276] 0709 12:24:14.320412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.605170 (+1284758us) service_pool.cc:224] Handling call
0709 12:24:15.609866 (+  4696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.632019 (+ 22153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.632284 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.635063 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 241) took 2467 ms. Trace:
I20260709 12:24:15.648458 26050 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:15.647341 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 202) took 2030 ms. Trace:
I20260709 12:24:15.599457 27878 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.630367 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:15.649708 26054 rpcz_store.cc:276] 0709 12:24:13.564353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.674365 (+110012us) service_pool.cc:224] Handling call
0709 12:24:15.569459 (+1895094us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54001b3fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.594694 (+ 25235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847638784}
I20260709 12:24:15.628863 26785 rpcz_store.cc:276] 0709 12:24:13.738868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.739952 (+  1084us) service_pool.cc:224] Handling call
0709 12:24:13.740311 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.563432 (+1823121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.593804 (+ 30372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.653522 26048 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:15.627238 27856 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.045766 26126 rpcz_store.cc:276] 0709 12:24:13.201531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.507189 (+1305658us) service_pool.cc:224] Handling call
0709 12:24:14.507584 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.507976 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.508006 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.508562 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.936195 (+427633us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:14.936344 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.936459 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.938710 (+  2251us) raft_consensus.cc:1685] finished
0709 12:24:14.939712 (+  1002us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.956346 (+ 16634us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44576,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":6025,"thread_start_us":426916,"threads_started":1,"wal-append.queue_time_us":427257}
I20260709 12:24:15.635588 27868 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.599284 26194 rpcz_store.cc:276] 0709 12:24:12.869119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.052161 (+2183042us) service_pool.cc:224] Handling call
0709 12:24:15.052870 (+   709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.546059 (+493189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.567861 (+ 21802us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.652842 26787 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:15.647377 27959 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.660717 27931 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.609207 28060 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.670442 26051 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:15.660005 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 204) took 1258 ms. Trace:
I20260709 12:24:15.460732 26046 rpcz_store.cc:276] 0709 12:24:13.523998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.726245 (+202247us) service_pool.cc:224] Handling call
0709 12:24:13.727812 (+  1567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.728189 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.728218 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.728779 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.020628 (+1291849us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.020707 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.020807 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.020845 (+    38us) raft_consensus.cc:1685] finished
0709 12:24:15.020860 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.021040 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37020,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":1277740,"threads_started":1,"wal-append.queue_time_us":996627}
I20260709 12:24:15.636711 26199 rpcz_store.cc:276] 0709 12:24:14.335598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.560463 (+1224865us) service_pool.cc:224] Handling call
0709 12:24:15.602805 (+ 42342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.671299 26048 rpcz_store.cc:276] 0709 12:24:14.401836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.585311 (+1183475us) service_pool.cc:224] Handling call
0709 12:24:15.659937 (+ 74626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.673113 26125 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:15.625380 25950 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 2, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:15.647929 26127 rpcz_store.cc:276] 0709 12:24:13.167441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.233158 (+1065717us) service_pool.cc:224] Handling call
0709 12:24:14.233648 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.233886 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:15.050288 (+816402us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.601577 (+551289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.609963 (+  8386us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:15.629058 (+ 19095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.633342 (+  4284us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.634988 (+  1646us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":595,"prepare.queue_time_us":522653,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":11299,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8500,"thread_start_us":1316587,"threads_started":2,"wal-append.queue_time_us":589549}
I20260709 12:24:15.671203 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 207) took 1268 ms. Trace:
I20260709 12:24:15.675709 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 246) took 2085 ms. Trace:
I20260709 12:24:15.676146 26125 rpcz_store.cc:276] 0709 12:24:13.589602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.647716 (+2058114us) service_pool.cc:224] Handling call
0709 12:24:15.667172 (+ 19456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.675058 (+  7886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.675255 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.675892 26051 rpcz_store.cc:276] 0709 12:24:14.402738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.647754 (+1245016us) service_pool.cc:224] Handling call
0709 12:24:15.659637 (+ 11883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.670964 (+ 11327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.671153 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.684795 27773 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.694001 27773 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:15.694597 27773 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:15.677091 26783 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.147s	user 0.012s	sys 0.009s
I20260709 12:24:15.677834 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 620) took 1024 ms. Trace:
I20260709 12:24:15.700709 28085 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:15.706193 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 97755.3us
I20260709 12:24:15.707299 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:15.709137 27685 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.001s
I20260709 12:24:15.711154 27773 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:15.655936 27962 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:15.714007 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.714465 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.717459 27878 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.718461 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:15.718796 27900 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.719352 27748 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:15.719729 27748 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.720081 27748 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.720456 28016 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.721563 28075 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:15.722805 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.723294 27959 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.725592 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:15.670488 27931 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.728364 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
W20260709 12:24:15.729074 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 405) took 1790 ms (client timeout 1641 ms). Trace:
W20260709 12:24:15.729472 26783 rpcz_store.cc:269] 0709 12:24:13.939026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.042172 (+103146us) service_pool.cc:224] Handling call
0709 12:24:15.728961 (+1686789us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.730225 27931 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.730875 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2212) took 2427 ms. Trace:
I20260709 12:24:15.730641 27980 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.731248 26121 rpcz_store.cc:276] 0709 12:24:13.303095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.010138 (+1707043us) service_pool.cc:224] Handling call
0709 12:24:15.010759 (+   621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.011370 (+   611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.011409 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.030792 (+ 19383us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.678612 (+647820us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:15.678668 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.678759 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.714804 (+ 36045us) raft_consensus.cc:1685] finished
0709 12:24:15.714826 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.730788 (+ 15962us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":372,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":8750,"thread_start_us":217322,"threads_started":1,"wal-append.queue_time_us":675195}
I20260709 12:24:15.733455 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 240) took 2566 ms. Trace:
W20260709 12:24:15.734261 27932 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.931s	user 0.007s	sys 0.015s
I20260709 12:24:15.734736 27932 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:15.734965 27748 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.738680 27748 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.739526 27748 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:15.739902 27748 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:15.741865 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61734us
I20260709 12:24:15.742092 27748 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:15.742637 26782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 404) took 1852 ms (client timeout 1556 ms). Trace:
W20260709 12:24:15.743173 26782 rpcz_store.cc:269] 0709 12:24:13.932389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.933969 (+  1580us) service_pool.cc:224] Handling call
0709 12:24:15.742362 (+1808393us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.742528 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347953024}
I20260709 12:24:15.744001 26126 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:15.744716 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2214) took 2059 ms. Trace:
I20260709 12:24:15.745033 26126 rpcz_store.cc:276] 0709 12:24:13.685430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.677422 (+1991992us) service_pool.cc:224] Handling call
0709 12:24:15.743756 (+ 66334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.744437 (+   681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.744641 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:15.748178 26129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 221) took 3872 ms (client timeout 2025 ms). Trace:
W20260709 12:24:15.748723 26129 rpcz_store.cc:269] 0709 12:24:11.875763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.922741 (+ 46978us) service_pool.cc:224] Handling call
0709 12:24:15.747904 (+3825163us) spinlock_profiling.cc:243] Waited 3.13 s on lock 0x7b540010e0d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.748085 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7197610240}
I20260709 12:24:15.733994 26120 rpcz_store.cc:276] 0709 12:24:13.167076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.834497 (+667421us) service_pool.cc:224] Handling call
0709 12:24:14.173532 (+339035us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.845046 (+671514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.733027 (+887981us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:15.733291 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643243776}
I20260709 12:24:15.752933 26122 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:15.753556 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:15.754691 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 237) took 3439 ms. Trace:
I20260709 12:24:15.755119 26122 rpcz_store.cc:276] 0709 12:24:12.315644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.890740 (+575096us) service_pool.cc:224] Handling call
0709 12:24:15.752420 (+2861680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.753327 (+   907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.754353 (+  1026us) spinlock_profiling.cc:243] Waited 2.86 s on lock 0x7b540010e0d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.754486 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6580114688}
I20260709 12:24:15.756903 26121 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:15.756995 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 247) took 2072 ms (client timeout 1781 ms). Trace:
W20260709 12:24:15.757516 26123 rpcz_store.cc:269] 0709 12:24:13.684586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.712292 (+ 27706us) service_pool.cc:224] Handling call
0709 12:24:15.756391 (+2044099us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54001860d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.756813 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902547840}
I20260709 12:24:15.758361 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2218) took 2029 ms. Trace:
I20260709 12:24:15.759057 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:15.758839 26121 rpcz_store.cc:276] 0709 12:24:13.729410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.734023 (+2004613us) service_pool.cc:224] Handling call
0709 12:24:15.756552 (+ 22529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.757364 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.758182 (+   818us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b54001860d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.758304 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50825472}
I20260709 12:24:15.761507 27758 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:15.762002 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:15.762706 26121 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.703969 26787 rpcz_store.cc:276] 0709 12:24:14.650599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.961540 (+310941us) service_pool.cc:224] Handling call
0709 12:24:14.962141 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.669884 (+707743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.674630 (+  4746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.765430 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:15.765933 26045 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.858s	user 0.012s	sys 0.004s
I20260709 12:24:15.766682 26045 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.767059 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:15.767338 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1025) took 2247 ms. Trace:
W20260709 12:24:15.770463 27750 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:24:15.767720 26045 rpcz_store.cc:276] 0709 12:24:13.520092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.522624 (+  2532us) service_pool.cc:224] Handling call
0709 12:24:13.538632 (+ 16008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.708357 (+169725us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b54000d73a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:24:15.767021 (+2058664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.767265 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143056640}
I20260709 12:24:15.787129 26121 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:15.788533 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:15.795573 26129 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:15.793428 26786 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:15.800688 25950 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:15.783744 28027 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.809396 26121 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:15.782785 27748 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.788028 27758 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:15.772120 28085 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.829939 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:15.838037 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.811026 27953 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.839462 26123 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.820736 27766 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:15.840967 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1007) took 2909 ms. Trace:
I20260709 12:24:15.840210 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:15.845692 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 104671us
I20260709 12:24:15.846827 26129 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.847681 26203 rpcz_store.cc:276] 0709 12:24:12.931080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.507036 (+2575956us) service_pool.cc:224] Handling call
0709 12:24:15.576792 (+ 69756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.840611 (+263819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.840643 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.840685 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.840718 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.840776 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.840901 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22067,"raft.queue_time_us":56240,"spinlock_wait_cycles":12035968,"thread_start_us":190639,"threads_started":1}
I20260709 12:24:15.857985 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1016) took 1523 ms. Trace:
I20260709 12:24:15.843262 25950 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 3, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:15.855180 27999 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.846383 27758 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:15.859694 27766 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:15.859753 26196 rpcz_store.cc:276] 0709 12:24:14.334439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.525313 (+1190874us) service_pool.cc:224] Handling call
0709 12:24:15.573374 (+ 48061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.857595 (+284221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.857657 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.857715 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:15.857748 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.857807 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.857930 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33217,"raft.queue_time_us":74606,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":16792,"thread_start_us":202746,"threads_started":1}
I20260709 12:24:15.858995 27953 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.884344 26121 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:15.863639 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1409) took 2925 ms. Trace:
W20260709 12:24:15.872605 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1354417210000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 12:24:15.889190 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 341131us
W20260709 12:24:15.873991 28081 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:24:15.890178 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1354417210000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 12:24:15.890748 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.890839 27589 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:24:15.891353 27766 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.892787 27856 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:15.867157 28060 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:15.935420 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0211417209998217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
I20260709 12:24:15.960618 27589 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:15.962292 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 147998us
I20260709 12:24:15.963483 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:15.890559 26201 rpcz_store.cc:276] 0709 12:24:12.938839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.496364 (+2557525us) service_pool.cc:224] Handling call
0709 12:24:15.564349 (+ 67985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.860976 (+296627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.861009 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.863349 (+  2340us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:15.863391 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.863452 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.863583 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24838,"raft.queue_time_us":61884,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":12259,"thread_start_us":216516,"threads_started":1}
I20260709 12:24:15.895090 27932 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:15.896157 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:15.896534 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 297) took 2109 ms. Trace:
I20260709 12:24:15.896210 27931 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:15.896311 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:15.898896 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:15.903239 26052 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:15.910089 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.912899 27962 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.871668 28016 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:15.929203 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.980818 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:47798 (request call id 2265) took 2192 ms. Trace:
W20260709 12:24:15.917512 28079 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.305s	user 0.005s	sys 0.000s
I20260709 12:24:15.920146 27959 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.920953 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:15.921272 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:15.923033 27798 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.923811 27976 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.924024 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.924427 28001 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.925006 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:15.925570 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:16.032945 26788 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.927290 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.888939 27975 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:15.894199 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2261) took 2118 ms. Trace:
I20260709 12:24:15.936605 27900 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.936924 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:15.937325 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:15.937996 25950 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 3, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:15.965291 28089 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.965739 27953 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.873059 28027 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.887553 27905 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.969771 26123 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.961640 27589 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.968743 26198 rpcz_store.cc:276] 0709 12:24:13.787620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.522213 (+1734593us) service_pool.cc:224] Handling call
0709 12:24:15.582384 (+ 60171us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.624939 (+ 42555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.624979 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.643544 (+ 18565us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.857858 (+214314us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:15.857887 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.857940 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.896350 (+ 38410us) raft_consensus.cc:1685] finished
0709 12:24:15.896368 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.896482 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12350,"raft.queue_time_us":27865,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":21126,"thread_start_us":190290,"threads_started":1,"wal-append.queue_time_us":232435}
I20260709 12:24:15.981238 26052 rpcz_store.cc:276] 0709 12:24:13.777150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.551077 (+773927us) service_pool.cc:224] Handling call
0709 12:24:15.746724 (+1195647us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b540022bd54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439ca3d 00007f5b7936ccd0 00007f5b728f1254 00007f5b728f11e2 00007f5b728f1111 00007f5b728f108c 00007f5b728f0342 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:15.903092 (+156368us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4c00229a80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72671a20 00007f5b72682a29 00007f5b7268313f 00007f5b726472ab 00007f5b72674f3c 00007f5b726753fe 00007f5b7290288b 00007f5b74301495 00007f5b7435230f 00007f5b7439ca55 00007f5b7936ccd0
0709 12:24:15.969786 (+ 66694us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186437632}
W20260709 12:24:15.986824 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.986678 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:15.988329 26122 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.990074 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:15.992259 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:15.992870 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:15.993777 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 97561.5us
I20260709 12:24:15.996488 27773 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:15.997231 27931 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:15.997845 28091 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.071874 28091 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:15.999245 28088 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:16.076121 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:16.076864 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.076443 28088 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.078351 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.079443 28088 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:16.016605 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53494.5us
I20260709 12:24:16.016945 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1505) took 2450 ms. Trace:
I20260709 12:24:16.017769 27758 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:16.021773 26784 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.005s	user 0.012s	sys 0.000s
I20260709 12:24:16.022599 27748 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:16.023064 28016 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.023819 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:16.024428 28085 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:15.977528 28075 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:16.030480 26129 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:16.039674 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:16.047971 26121 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:16.060730 27868 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:16.063795 28090 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:16.064602 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 205184us
I20260709 12:24:16.065492 26783 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:16.065903 26120 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.110674 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.066385 27927 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:16.044523 27856 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:16.068849 28087 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.070422 27976 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.084383 26124 rpcz_store.cc:276] 0709 12:24:13.566526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.500408 (+1933882us) service_pool.cc:224] Handling call
0709 12:24:15.500886 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.501481 (+   595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.501522 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.502136 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.598330 (+ 96194us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:15.598370 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.598432 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.016620 (+418188us) raft_consensus.cc:1685] finished
0709 12:24:16.016653 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.016859 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26878,"raft.queue_time_us":5662,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":21764,"thread_start_us":95650,"threads_started":1,"wal-append.queue_time_us":183621}
I20260709 12:24:16.082732 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2216) took 2375 ms. Trace:
W20260709 12:24:16.087321 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44256.9us
I20260709 12:24:16.087612 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:16.088549 26053 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:16.052806 27798 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.103711 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.104769 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2213) took 2419 ms. Trace:
I20260709 12:24:16.044380 26787 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:16.105901 26791 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:16.109143 26788 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:16.067566 28089 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:16.112718 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.104434 28088 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:16.115079 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.059077 27931 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:16.046280 27900 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.125699 26125 rpcz_store.cc:276] 0709 12:24:13.707471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.693696 (+1986225us) service_pool.cc:224] Handling call
0709 12:24:15.694184 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.694607 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.694638 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.695182 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.888474 (+193292us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:15.888529 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.888641 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.082372 (+193731us) raft_consensus.cc:1685] finished
0709 12:24:16.082408 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.082659 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":191664,"threads_started":1,"wal-append.queue_time_us":342761}
I20260709 12:24:16.129786 26128 rpcz_store.cc:276] 0709 12:24:13.684923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.657825 (+1972902us) service_pool.cc:224] Handling call
0709 12:24:15.686262 (+ 28437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.706172 (+ 19910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.706202 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.706755 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.861051 (+154296us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:15.861093 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.861193 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.104469 (+243276us) raft_consensus.cc:1685] finished
0709 12:24:16.104494 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.104687 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3704,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":161,"thread_start_us":153364,"threads_started":1,"wal-append.queue_time_us":342594}
I20260709 12:24:16.108615 26785 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.112835 28001 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.088033 28091 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:16.138592 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 207156us
I20260709 12:24:16.144392 27931 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.155926 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.056600 27959 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.159713 27959 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.088194 28027 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.146262 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.160524 28063 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:16.161080 27748 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:16.161722 27748 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:16.162281 28027 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.162982 28027 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:16.153288 26121 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:16.162834 27748 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.128535 27750 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:16.039861 26049 rpcz_store.cc:276] 0709 12:24:13.775925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.254626 (+478701us) service_pool.cc:224] Handling call
0709 12:24:14.344667 (+ 90041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.568685 (+1224018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.893875 (+325190us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54002e2554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:15.894124 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679211776}
I20260709 12:24:16.153342 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:16.152417 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 90610us
W20260709 12:24:16.161113 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.169992 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.135994 26784 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:24:16.166301 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:16.137153 27927 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:16.144903 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.115780 28029 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:16.138072 27856 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.138343 26046 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
W20260709 12:24:16.249336 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.174018 26129 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:16.304551 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2220) took 2565 ms. Trace:
W20260709 12:24:16.303046 26020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.707s (SENT) [suppressed 14 similar messages]
I20260709 12:24:16.257081 26121 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
W20260709 12:24:16.246912 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80700.3us
I20260709 12:24:16.299611 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2217) took 2591 ms. Trace:
I20260709 12:24:16.291776 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:16.242880 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 72970us
I20260709 12:24:16.341859 26122 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:16.342608 26121 pending_rounds.cc:85] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 2
I20260709 12:24:16.345652 26121 pending_rounds.cc:107] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:24:16.205765 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.300426 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:16.300190 28086 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:24:16.165093 28075 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.247779 27878 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:16.149570 27868 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.144301 27905 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.250s
I20260709 12:24:16.171587 28001 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.457881 28029 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.460096 28001 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.398821 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.395449 27959 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.460793 28029 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.341694 26126 rpcz_store.cc:276] 0709 12:24:13.739426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.745738 (+2006312us) service_pool.cc:224] Handling call
0709 12:24:15.746141 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.746410 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.746436 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.746953 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:24:16.130544 (+383591us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:16.130581 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.130639 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.304321 (+173682us) raft_consensus.cc:1685] finished
0709 12:24:16.304340 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.304485 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":383066,"threads_started":1,"wal-append.queue_time_us":521349}
I20260709 12:24:16.462983 28016 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:16.463407 26169 leader_election.cc:400] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7.
W20260709 12:24:16.436605 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.420886 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.453711 26045 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:16.247836 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:16.363379 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.345345 26127 rpcz_store.cc:276] 0709 12:24:13.707772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.900375 (+2192603us) service_pool.cc:224] Handling call
0709 12:24:15.900897 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.901348 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.901388 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.902109 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 12:24:16.109865 (+207756us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:16.278937 (+169072us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.299111 (+ 20174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.299242 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.299303 (+    61us) raft_consensus.cc:1685] finished
0709 12:24:16.299322 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.299520 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59558,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":25669,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":150,"thread_start_us":243826,"threads_started":2,"wal-append.queue_time_us":230944}
I20260709 12:24:16.466144 27953 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.228973 27750 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:16.466923 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 237584us
I20260709 12:24:16.467140 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.447109 26120 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:16.468545 27685 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.469020 27685 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.469493 27959 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:16.471272 28084 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.471699 28084 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:16.472078 28084 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.304404 28097 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.453480 27927 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:16.454448 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.465377 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 192760us
W20260709 12:24:16.464799 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 98216.6us
I20260709 12:24:16.465034 26121 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:16.463541 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.457352 28075 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.473336 27878 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.420673 27798 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:16.461279 26128 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:16.468045 26127 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:16.478969 27953 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:16.479674 27900 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.480451 27953 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:16.480867 27953 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:16.481017 28089 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:16.482367 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:16.485551 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:16.487035 26127 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:24:16.488161 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:16.488505 26127 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.491968 26791 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:16.492957 26126 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.494788 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:16.456395 28099 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:16.495787 28016 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:16.496387 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:16.473598 26051 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.498373 28016 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:16.498957 28108 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:16.499303 28108 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:16.500173 28108 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.501245 28108 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:16.464815 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 309036us
I20260709 12:24:16.476835 27976 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:16.477315 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:16.477460 27878 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.515317 27931 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.517699 27931 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.485437 27750 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.472710 28090 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:16.486253 27953 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.492830 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:16.466104 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 257918us
I20260709 12:24:16.167881 26786 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.504614 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.524551 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:16.463778 26169 leader_election.cc:403] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:16.338268 26784 pending_rounds.cc:85] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 2
I20260709 12:24:16.509824 26051 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:16.510807 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:16.512279 26123 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:16.526893 26128 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:16.513126 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.341310 28008 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:16.529186 26124 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.529889 26122 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.472986 27589 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:16.531365 26198 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:16.534876 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.537436 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.539345 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.542876 26790 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.537163 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.528765 26784 pending_rounds.cc:107] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:24:16.547622 27856 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:16.545959 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.555550 26199 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:16.546228 26202 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:16.546641 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.547220 27932 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:16.549113 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.562474 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.551260 27685 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:16.553380 27868 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.553694 27953 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.570672 27959 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:16.522390 28090 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:16.545857 26203 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:16.554298 28060 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:16.554904 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.576009 26129 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:16.547997 26784 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:16.565579 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.563164 27932 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:16.570163 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:16.573220 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:16.576469 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.572053 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.547716 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:16.580801 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:16.541649 27748 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:16.582124 26789 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:16.585556 26126 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:16.589709 27685 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.590373 27685 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.589630 28090 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.590742 27685 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.591363 28090 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.592255 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 406) took 2551 ms. Trace:
I20260709 12:24:16.592546 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:16.589644 26197 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:16.592689 26784 rpcz_store.cc:276] 0709 12:24:14.040886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.045835 (+  4949us) service_pool.cc:224] Handling call
0709 12:24:14.895614 (+849779us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:14.896050 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.591948 (+1695898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.592155 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17647360}
W20260709 12:24:16.595220 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.596413 28085 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:16.575392 28060 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:16.596827 27900 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:16.597656 28060 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.599313 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.596827 28085 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:16.600591 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.600886 28085 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.601270 26053 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:16.601562 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.556006 28008 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.602262 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.602867 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.603039 26046 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 1.
I20260709 12:24:16.603464 26196 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:16.604116 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.604480 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.604599 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.604946 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:16.605038 26048 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:16.605258 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:16.604616 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.604121 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.606019 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.606248 26195 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:16.606739 26052 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:16.606931 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:16.607359 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:16.607759 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.608021 26197 raft_consensus.cc:2376] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4. Re-sending same reply.
I20260709 12:24:16.596295 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.602342 26203 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:16.608610 27975 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 12:24:16.635090 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79605us
W20260709 12:24:16.638118 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.585417 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.587263 27748 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.606948 27976 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:24:16.643065 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:16.644481 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.644502 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.588248 27905 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.650585 26790 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.650285 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.610230 27868 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:16.656123 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 192869us
I20260709 12:24:16.658260 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:16.640781 28060 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.629783 26783 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:16.660666 26195 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:16.643743 27856 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:16.668517 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:16.674093 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.674748 27856 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:16.589564 27932 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.676802 27932 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.678422 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:16.680543 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 84350.2us
I20260709 12:24:16.676612 27856 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:24:16.682140 27750 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.682518 27750 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.682832 27750 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.684423 27758 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.681493 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.685583 26120 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:16.680419 28104 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:16.674970 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.653941 27748 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.643374 27685 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:16.693754 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98459.3us
W20260709 12:24:16.699182 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61379us
I20260709 12:24:16.701143 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.702711 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.703521 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.692615 28001 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:16.686932 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.709942 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.712410 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:16.710551 26124 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:16.693099 27856 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:16.705669 26128 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:16.693223 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.700953 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:16.700686 27868 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.714457 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.717973 26199 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:16.718331 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.659300 28084 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:16.722349 26199 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:16.722527 26122 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:16.705890 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.721546 26786 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:16.756686 26049 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:16.752708 28084 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.707218 27953 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:16.752734 27685 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.755511 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.756608 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.753970 26199 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.757437 28029 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.794296 28029 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.719565 28114 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:24:16.801039 28114 consensus_queue.cc:1243] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:16.782006 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.802636 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.803195 26128 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.803815 27878 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.805270 26123 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:16.805716 28085 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:16.807296 27878 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:16.800761 26782 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:16.738565 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.725636 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:16.816960 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.785036 28113 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:16.789588 26047 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:16.821676 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.792507 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:16.787043 26048 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:16.788867 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.830283 26053 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.789562 26050 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:16.727280 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.781405 26129 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:16.783684 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.784003 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.831894 26121 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.709995 27589 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.830561 26054 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:16.832955 26124 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:16.855067 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 154184us
I20260709 12:24:16.788059 26199 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:16.834219 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107020us
W20260709 12:24:16.860176 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.860313 28115 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.860253 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.857577 28027 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.861429 26129 raft_consensus.cc:2376] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5. Re-sending same reply.
I20260709 12:24:16.861559 28027 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.720703 27976 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.133s
I20260709 12:24:16.865427 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:16.821571 27953 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:16.720613 28001 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:16.752887 28008 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.877672 28008 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:16.878100 28008 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.878419 28008 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.867457 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:16.880970 26052 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:16.881258 28016 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:16.881772 28016 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:16.882946 28016 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.830224 28085 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.890952 26195 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:16.872805 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.891726 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66627.9us
I20260709 12:24:16.873332 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.788751 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:16.876639 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.885710 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.886813 27878 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:16.854768 27975 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.872032 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.875831 27868 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.892395 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.892483 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:16.893059 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:16.903858 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71211.9us
I20260709 12:24:16.899813 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:16.895637 26050 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:16.865974 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.908428 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.783290 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:16.904541 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.892198 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.919229 26195 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:16.907745 27905 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7.
I20260709 12:24:16.867887 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:16.909291 26201 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:16.893654 28016 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.909766 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.928331 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.929397 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:16.930724 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.934897 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.905335 27798 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:16.927439 27856 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:16.936625 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.868780 27589 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:16.941655 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.944018 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:16.944572 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.920688 27959 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:16.940512 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.942260 26195 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:16.943377 26128 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:16.952713 26203 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:16.955052 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:16.933416 26783 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.933604 28119 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.936638 28115 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:16.936872 27868 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:16.961114 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:16.908411 27975 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.956218 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:16.960803 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 95476.8us
I20260709 12:24:16.967018 26121 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:16.966528 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.949146 27959 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:16.954674 27932 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.956827 27878 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.972107 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:16.957741 28114 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.969118 26194 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:16.974229 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:16.969411 27976 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:16.961907 28097 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:24:16.975229 27953 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:16.967756 28104 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.964541 28117 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.003336 27976 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:16.976848 28119 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:16.972455 27685 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:16.961891 27931 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.979494 26053 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
W20260709 12:24:17.042630 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 86488.6us
I20260709 12:24:16.961990 28090 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.991707 27878 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.049628 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.959851 28027 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:16.991458 26198 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:16.995908 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.982048 26127 raft_consensus.cc:2376] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2. Re-sending same reply.
I20260709 12:24:17.001356 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:16.981137 28115 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:16.965097 27868 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:17.013147 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.065279 26121 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:17.038084 27773 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:17.023728 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.066736 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 74820.7us
I20260709 12:24:16.974026 27959 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:17.041340 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74919.2us
I20260709 12:24:16.961998 28084 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.000932 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:16.975888 28001 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.052923 27878 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.107610 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.064584 27685 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:17.048993 28090 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:17.083262 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.039745 28121 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:17.113617 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.106881 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 182260us
I20260709 12:24:17.008752 27932 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.121135 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.065727 28027 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.067785 27868 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.125121 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:17.170174 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47225.2us
I20260709 12:24:17.117903 26124 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:17.125125 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 142477us
I20260709 12:24:17.173015 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.129146 26127 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 2: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:17.174683 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.175817 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:17.177412 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.046038 27931 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.178086 27931 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.176934 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.184810 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:17.186121 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.186858 26127 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
W20260709 12:24:17.135010 28120 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:24:17.135879 28075 raft_consensus.cc:993] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:24:17.135996 28115 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.189743 27962 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.190675 27962 raft_consensus.cc:1081] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
W20260709 12:24:17.138892 28118 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:24:17.194870 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.194140 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:17.140092 27975 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:17.196617 27975 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:17.197408 27975 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.197906 27962 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:17.198496 27962 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:17.199108 27962 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.106205 27856 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.141147 28097 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:17.201922 28097 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.202440 28097 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:17.202940 28097 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.144397 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:17.187959 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.115818 28085 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.165578 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:17.161144 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.210599 28060 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:17.208814 26127 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:17.211629 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.220002 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.153308 26199 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:17.168604 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 164713us
I20260709 12:24:17.139748 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:17.205157 26049 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:17.041728 28117 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.208168 27773 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:17.221779 26202 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:17.138897 28089 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:17.225414 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.218765 28085 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:17.225121 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:17.153724 28090 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.257491 26121 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:17.147034 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:17.169142 26786 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:17.221670 28084 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:17.308529 26121 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:17.308199 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48604.6us
I20260709 12:24:17.301272 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:17.328209 26126 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.231150 28089 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:17.337057 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 292397us
I20260709 12:24:17.329950 26126 pending_rounds.cc:85] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 4
I20260709 12:24:17.337855 26126 pending_rounds.cc:107] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:17.338263 26126 raft_consensus.cc:2889] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.152040 28122 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.169s
I20260709 12:24:17.307559 27927 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.131785 27976 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.342092 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:17.342317 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:17.324380 28060 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.337978 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:17.342855 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:17.323132 28084 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.346380 26127 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:17.322098 27589 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:17.357296 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.345783 27927 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.384909 27927 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.384639 28127 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:24:17.388738 26049 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.390231 27962 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.391345 26049 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:17.392398 28134 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:17.392809 28134 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:17.392984 27932 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:17.393723 27748 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.694s	user 0.007s	sys 0.004s
I20260709 12:24:17.394289 27748 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.337337 28091 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 12:24:17.230268 28121 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:17.358282 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:17.668785 27750 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.986s	user 0.006s	sys 0.017s
W20260709 12:24:17.395356 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53389.4us
I20260709 12:24:17.670693 27750 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.671589 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:17.546720 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 198001us
I20260709 12:24:17.673230 27750 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:17.674279 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.660514 28119 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:17.466215 27962 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:17.445019 26049 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:24:17.677662 28119 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:17.471055 28132 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 12:24:17.679077 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 320895us
W20260709 12:24:17.506175 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.476177 28008 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
W20260709 12:24:17.444676 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 119837us
W20260709 12:24:17.660153 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55763.3us
W20260709 12:24:17.643270 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 432636us
W20260709 12:24:17.458214 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 238336us
I20260709 12:24:17.425860 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:17.688184 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 262380us
I20260709 12:24:17.480882 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:17.693481 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.607715 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.470274 26045 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:17.474376 26783 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 12:24:17.697297 26200 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:17.710659 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1421) took 1052 ms. Trace:
I20260709 12:24:17.442058 27976 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.711143 26200 rpcz_store.cc:276] 0709 12:24:16.658554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.658725 (+   171us) service_pool.cc:224] Handling call
0709 12:24:17.480681 (+821956us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:17.710498 (+229817us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31548288}
I20260709 12:24:17.491568 28128 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:17.676542 26049 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 1
I20260709 12:24:17.472074 27758 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:17.707237 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:17.708436 26129 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:17.709064 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:17.721028 26123 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
W20260709 12:24:17.703783 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 360970us
I20260709 12:24:17.722306 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.714291 26128 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:17.490311 28067 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:17.713733 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:17.704450 27798 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.472420 28141 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.317s
I20260709 12:24:17.675364 28084 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.691043 28119 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:17.350400 27931 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.461833 28133 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.717943 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:17.776628 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58798.6us
I20260709 12:24:17.777954 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:17.727545 28109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.012s	sys 0.016s
W20260709 12:24:17.782476 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59448.6us
W20260709 12:24:17.782651 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 68913.8us
I20260709 12:24:17.791849 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.691286 28008 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.705068 28145 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:17.720580 26049 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:24:17.714802 28128 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:17.712594 26783 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
W20260709 12:24:17.727698 28016 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.782s	user 0.011s	sys 0.000s
W20260709 12:24:17.779560 27868 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.005s
W20260709 12:24:17.794196 28109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.014s	sys 0.036s
I20260709 12:24:17.761888 27589 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.683400 28097 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.707074 28115 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:24:17.828027 26129 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:17.438760 27685 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:17.829583 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 122407us
I20260709 12:24:17.830009 28067 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:17.800062 26049 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.776409 27798 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.490728 28089 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.719465 27758 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:17.393762 28134 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.856227 27758 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.775267 28133 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.780627 27932 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.822708 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:17.783306 27962 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:17.839692 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 126034us
I20260709 12:24:17.783955 28117 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:17.793731 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:17.851397 28143 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.163s
I20260709 12:24:17.874620 28143 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:17.874985 28143 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:17.876216 28131 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.875875 28143 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:17.876824 28131 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.877347 28143 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.879149 28131 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:17.880381 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.832120 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 432) took 1003 ms. Trace:
I20260709 12:24:17.799635 28135 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.884965 28151 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.884698 28135 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.885462 28151 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.885860 28151 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.887243 28135 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:17.848284 26053 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:17.831416 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53595.4us
I20260709 12:24:17.819430 28128 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.799227 28145 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.725117 28060 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 12:24:17.832513 27773 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:17.839628 27589 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:17.829375 27750 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:17.860317 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 68550.8us
I20260709 12:24:17.855818 28136 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:17.873131 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:17.859978 28133 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.897292 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.868500 28140 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.168s
W20260709 12:24:17.847046 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.854647 27798 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.888664 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:17.841257 27868 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.855902 28089 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.901582 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:17.904317 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.873462 27927 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:17.908846 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.904995 28016 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.895243 27773 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:17.911382 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:17.893559 28145 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:17.901633 28090 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.576s	user 0.003s	sys 0.007s
I20260709 12:24:17.896611 28136 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.882812 26783 rpcz_store.cc:276] 0709 12:24:16.828549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.828809 (+   260us) service_pool.cc:224] Handling call
0709 12:24:17.827642 (+998833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.920116 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:17.903820 28142 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.902395 26197 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:17.919519 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.924773 28097 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:17.916898 28067 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:17.903288 28141 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:24:17.916734 28140 consensus_queue.cc:1243] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:17.924100 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:17.926060 28153 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:24:17.968143 28134 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.896785 28119 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:17.980862 28155 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:24:17.981961 28141 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:17.982424 28141 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:17.918905 28117 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.927141 28147 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:17.857821 28127 consensus_queue.cc:1175] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:17.907346 28131 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:17.933157 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.931028 28154 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.931236 28090 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.934361 28164 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.931257 27962 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.935869 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.936532 27856 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:17.937474 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1540) took 1077 ms. Trace:
W20260709 12:24:17.938104 27878 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.000s
I20260709 12:24:17.937929 28145 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:17.939229 28029 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:17.940680 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:17.945417 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:17.947809 27927 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:17.957206 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1530) took 1174 ms. Trace:
I20260709 12:24:17.960384 28121 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:17.924955 28136 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.000105 26786 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.922287 27748 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:17.917784 27750 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:17.923228 27773 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.048565 28166 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.048877 27750 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.049176 28166 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.051339 28166 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:17.974677 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 182922us
I20260709 12:24:18.053020 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.053123 28090 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:17.974694 27932 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:17.974988 28168 raft_consensus.cc:2764] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:18.053889 27932 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:18.054366 27932 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.055130 28152 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:18.055567 28152 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.055919 28152 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.970603 28142 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.057931 28142 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:18.059190 28145 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:18.040339 28008 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:18.033496 28154 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:18.063400 28154 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:17.983445 28141 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.054387 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:18.064973 28141 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.066175 27927 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.066618 27927 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:18.067060 27927 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.032030 26128 rpcz_store.cc:276] 0709 12:24:16.860025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.480078 (+620053us) service_pool.cc:224] Handling call
0709 12:24:17.605079 (+125001us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.723501 (+118422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.937186 (+213685us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:17.937412 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250502016}
I20260709 12:24:18.036211 26124 rpcz_store.cc:276] 0709 12:24:16.782861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.906257 (+123396us) service_pool.cc:224] Handling call
0709 12:24:17.956974 (+1050717us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54000b95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:17.957119 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426557824}
W20260709 12:24:18.036827 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 96273.2us
I20260709 12:24:17.917459 28016 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:18.043363 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107591us
I20260709 12:24:18.085615 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:18.002915 28060 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.999594 28117 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:17.925567 27589 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:18.073824 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.089118 27589 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.090611 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:18.074174 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.071702 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:18.091710 26052 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:18.087335 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:18.036789 28131 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:18.095239 28119 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.109738 28119 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:18.110173 28119 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.042443 28143 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:18.108686 28029 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.113343 28029 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.114338 28029 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.095153 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:18.096383 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:18.116617 26051 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:18.116940 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:18.118444 26121 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.119354 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.039942 28147 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:18.095546 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:18.036312 27878 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.126596 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.125492 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.127233 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:18.130072 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 228548us
I20260709 12:24:18.125885 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.086580 28016 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.131870 26198 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:18.132742 27856 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:24:18.137218 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.138085 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.138746 26194 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:18.140497 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.123669 28121 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.141286 28162 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:18.141415 28121 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:18.142098 28121 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.141870 28162 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.128719 28140 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.144389 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 265) took 1081 ms. Trace:
I20260709 12:24:18.096917 27748 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.123817 28089 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:18.145788 28089 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.148563 28027 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.148999 28027 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:18.151794 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 206445us
I20260709 12:24:18.152705 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.152889 28084 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:18.153549 28089 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:18.155786 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.156713 28131 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.157755 28001 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:18.158644 28001 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:18.160435 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:18.162521 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.144883 26122 rpcz_store.cc:276] 0709 12:24:17.063177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.481394 (+418217us) service_pool.cc:224] Handling call
0709 12:24:17.886342 (+404948us) spinlock_profiling.cc:243] Waited 7.12 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:17.886561 (+   219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.886926 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.886959 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.887501 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:24:18.002960 (+115459us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:18.003021 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.003114 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.144038 (+140924us) raft_consensus.cc:1685] finished
0709 12:24:18.144058 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.144324 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15331,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":14090,"spinlock_wait_cycles":16385792,"thread_start_us":115038,"threads_started":1,"wal-append.queue_time_us":208064}
W20260709 12:24:18.165118 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136553us
I20260709 12:24:18.105933 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:18.170636 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:18.113384 28143 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:18.125764 28087 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:18.150934 27927 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.180903 26125 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:18.088099 28060 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.118719 28097 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:24:18.090690 28135 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:24:18.184580 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.135088 28147 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:18.109158 27798 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.171041 28145 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:18.192157 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:18.192420 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70845.3us
I20260709 12:24:18.045051 28164 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.086939 28133 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.115939 28152 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.107148 28117 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.197008 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.197511 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.182562 28087 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.214450 28087 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.214845 28128 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.216415 26052 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:18.218997 27927 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.201181 28151 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.222685 28151 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.185973 26049 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:18.195737 28067 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:18.225383 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:18.226334 26049 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:18.228261 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:18.228852 28145 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:18.229991 28145 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.126224 28136 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:18.193918 27959 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.069s	user 0.002s	sys 0.010s
I20260709 12:24:18.207865 27953 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.207993 28143 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.211015 27878 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:18.149863 28027 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.213833 28164 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.231835 26195 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:18.198264 28142 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.219815 27773 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.203239 26046 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.240097 26201 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:18.234905 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.281538 26196 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:18.243819 28027 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.245814 28147 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.246688 28152 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.247347 27975 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:18.253248 28129 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.101s	user 0.003s	sys 0.001s
I20260709 12:24:18.253836 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.251897 28119 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.302587 26199 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:18.255823 28175 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:18.280525 28122 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.257175 27798 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:18.299355 26049 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:18.234633 27685 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:18.258883 28016 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.238106 27959 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.261166 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2232) took 1429 ms. Trace:
W20260709 12:24:18.265056 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.271450 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.272024 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.317936 26127 rpcz_store.cc:276] 0709 12:24:16.831778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.223266 (+391488us) service_pool.cc:224] Handling call
0709 12:24:17.258676 (+ 35410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.401050 (+142374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.260919 (+859869us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:18.261099 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190322944}
I20260709 12:24:18.238914 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:18.302776 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49009.6us
I20260709 12:24:18.255512 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 263) took 1421 ms. Trace:
I20260709 12:24:18.319064 26121 rpcz_store.cc:276] 0709 12:24:16.834239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.330285 (+496046us) service_pool.cc:224] Handling call
0709 12:24:17.335672 (+  5387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.118871 (+783199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.255210 (+136339us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:18.255449 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799715584}
I20260709 12:24:18.319463 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.256714 28142 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.309716 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.324617 28142 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:18.324700 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 85852.3us
I20260709 12:24:18.325063 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.325290 27748 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.326257 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.327268 27589 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:18.327550 27748 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.327684 27589 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.309645 28122 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.324045 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.311137 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2284) took 1460 ms. Trace:
I20260709 12:24:18.328959 26201 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.330389 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:18.330731 28140 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:18.270979 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:18.318517 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46580.6us
I20260709 12:24:18.332532 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.329228 26047 rpcz_store.cc:276] 0709 12:24:16.851257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.918115 (+ 66858us) service_pool.cc:224] Handling call
0709 12:24:16.948459 (+ 30344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.960057 (+ 11598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.960083 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:16.964581 (+  4498us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.038878 (+ 74297us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:17.038918 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.039011 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.252682 (+1213671us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:18.310829 (+ 58147us) raft_consensus.cc:1685] finished
0709 12:24:18.310849 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.311072 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4404,"raft.queue_time_us":86563,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":58220,"spinlock_wait_cycles":7896960,"thread_start_us":59559,"threads_started":1,"wal-append.queue_time_us":85469}
I20260709 12:24:18.322242 27750 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:18.328168 27589 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.333724 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.333595 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.334440 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:18.337742 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 66910.7us
I20260709 12:24:18.338490 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:18.307341 28117 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.339507 26199 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:18.345062 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.346235 26120 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:18.348502 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.348960 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:18.352557 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.353574 26124 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:18.355386 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:18.356295 26123 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.366861 26123 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:18.369993 26129 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.373507 26129 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:18.375600 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.376680 26129 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:18.380421 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.381258 26121 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:18.381851 26050 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.302093 28145 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:24:18.393952 26047 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:18.317294 28091 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:18.432001 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.441985 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.340417 28154 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.332101 27798 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.342190 28152 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:18.337011 27748 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.378453 28090 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.341217 28117 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.457844 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.338392 27750 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.460381 26129 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.380775 28175 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:18.380046 27975 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.463052 26127 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.464839 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.508002 26199 raft_consensus.cc:2376] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4. Re-sending same reply.
I20260709 12:24:18.396890 28164 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.378185 28166 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 12:24:18.528226 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.504627 26047 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.443761 27900 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.505721 26121 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:18.499351 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 156841us
I20260709 12:24:18.443732 26782 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.399649 28091 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:18.532114 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:18.378036 28131 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 12:24:18.465137 28090 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.509214 27750 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.512964 28075 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:18.465366 27976 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.505295 28175 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.544010 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.510221 28117 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.511634 27975 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.341986 28168 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.170s
I20260709 12:24:18.505487 28067 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:18.459969 28152 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:18.549234 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.527174 28060 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.512403 26786 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:18.549590 28140 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 12:24:18.561041 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.562307 28188 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:18.563045 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.536118 28164 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.564481 28162 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.565258 28029 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.577587 26122 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.576813 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.460776 28154 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.550112 28090 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.585597 27773 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.543205 28133 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:18.633376 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:18.533242 27932 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.563573 28119 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:18.618856 26195 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:18.638422 26054 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:18.618912 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 284512us
I20260709 12:24:18.571918 27975 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.569764 28175 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.570349 27976 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.540757 27900 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.613847 28152 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.612331 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.616257 28167 raft_consensus.cc:2764] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:18.619117 28060 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.618845 27959 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:18.639232 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.624264 27953 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.640733 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:18.626134 27685 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:18.667829 27773 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:18.674933 26122 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:18.634896 28168 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:18.617338 28067 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:18.685166 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:18.701260 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.701910 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:18.703403 26122 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:18.696468 27773 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:18.697158 28168 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:18.697276 28192 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:18.699541 27685 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.687954 26127 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.647748 28145 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:18.640053 28189 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:18.648383 28016 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:18.781739 28168 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.785264 28114 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.790741 28154 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:18.818172 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.627308 28188 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.625881 28091 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:18.792755 28087 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.001s
W20260709 12:24:18.896960 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.897014 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 257849us
I20260709 12:24:18.900266 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.686736 28060 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.909114 26199 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:18.910959 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.860539 28114 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:18.927212 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.895356 28085 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.959913 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:18.960896 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46245.4us
I20260709 12:24:18.917820 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:18.963717 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.968864 26195 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:18.902851 28154 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.957834 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:18.970202 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52470.1us
I20260709 12:24:19.001262 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.003049 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:18.861670 28189 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:19.015223 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.905799 27931 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:19.020892 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.020s	sys 0.034s
W20260709 12:24:19.022231 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.020s	sys 0.034s
W20260709 12:24:19.022640 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 689668us
W20260709 12:24:19.025017 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 124841us
I20260709 12:24:18.902307 28084 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:18.899255 28090 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.025712 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:18.967499 28145 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:18.971143 28075 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.028378 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.028218 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:18.903309 28087 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:18.919526 28193 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.917040 28091 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.001590 28179 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.146698 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 437) took 1452 ms. Trace:
W20260709 12:24:19.071238 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46052.4us
I20260709 12:24:19.147111 26782 rpcz_store.cc:276] 0709 12:24:17.694695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.695731 (+  1036us) service_pool.cc:224] Handling call
0709 12:24:17.779281 (+ 83550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.547932 (+768651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.146426 (+598494us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54004ffe54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:19.146618 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378214144}
I20260709 12:24:19.108785 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.149106 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:18.915324 27773 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:18.957986 28151 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.107712 28133 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.126318 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.126490 28145 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.036962 27878 raft_consensus.cc:2764] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:24:19.127856 28090 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:19.128330 27589 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.002s
I20260709 12:24:19.021132 27900 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:19.026983 28119 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:19.113440 28121 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.971s	user 0.014s	sys 0.001s
I20260709 12:24:19.132895 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.133538 27932 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.133371 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:19.116734 27758 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:19.138298 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2288) took 2170 ms (client timeout 2046 ms). Trace:
I20260709 12:24:19.139717 27976 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.043615 28029 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.017894 28067 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.141414 28188 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.142261 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2296) took 1067 ms. Trace:
I20260709 12:24:19.143059 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.143450 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2272) took 2355 ms. Trace:
I20260709 12:24:19.016952 28154 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.048671 28164 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.033174 28084 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.144697 28143 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:19.032918 28060 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.145579 28202 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:19.174253 28117 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.001s
W20260709 12:24:19.174438 26201 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.845s	user 0.008s	sys 0.002s
I20260709 12:24:19.176950 26127 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:19.177102 28189 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:19.187867 26786 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:19.188602 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:19.189209 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.189808 26784 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:19.190659 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.192456 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:19.033290 27868 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:19.321137 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.566097 26053 rpcz_store.cc:269] 0709 12:24:16.968346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.049286 (+ 80940us) service_pool.cc:224] Handling call
0709 12:24:19.138022 (+2088736us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:19.138187 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817764736}
W20260709 12:24:19.566797 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.473042 26129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.006s
W20260709 12:24:19.567945 26129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.013s	sys 0.006s
I20260709 12:24:19.567726 28117 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.568993 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2247) took 1057 ms. Trace:
I20260709 12:24:19.569383 26129 rpcz_store.cc:276] 0709 12:24:18.512017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.516760 (+  4743us) service_pool.cc:224] Handling call
0709 12:24:18.539751 (+ 22991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.540016 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:18.893188 (+353172us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.568664 (+675476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.568717 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:19.568763 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.568820 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.568940 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":84645,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1845,"raft.queue_time_us":89,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":1014108,"threads_started":2}
I20260709 12:24:19.570437 26201 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:19.570854 26783 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:19.571247 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 318) took 1375 ms. Trace:
I20260709 12:24:19.584568 26201 rpcz_store.cc:276] 0709 12:24:18.196025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.323794 (+127769us) service_pool.cc:224] Handling call
0709 12:24:19.571193 (+1247399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.586731 28154 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.602703 28084 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.571758 28087 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:19.283686 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.029s	sys 0.048s
W20260709 12:24:19.603679 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.041s	sys 0.048s
I20260709 12:24:19.572484 26051 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.604748 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.605365 26200 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.605829 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:19.469357 28027 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.169s	user 0.015s	sys 0.004s
I20260709 12:24:19.435447 28195 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.607065 28195 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.608426 28029 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:19.609058 28029 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.609941 28195 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.574659 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.612051 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:19.575747 26198 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:19.229588 26790 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:19.614410 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.614166 28114 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.615306 26198 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:19.615600 28114 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.616200 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.617285 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.618422 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.618693 28122 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.619360 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.620064 26198 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:19.577472 26046 rpcz_store.cc:276] 0709 12:24:16.787946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.791599 (+  3653us) service_pool.cc:224] Handling call
0709 12:24:16.791955 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.295054 (+1503099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.143222 (+848168us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:19.143390 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3221276032}
I20260709 12:24:19.579804 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.580554 27589 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.565500 26050 rpcz_store.cc:276] 0709 12:24:18.075010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.075208 (+   198us) service_pool.cc:224] Handling call
0709 12:24:18.096682 (+ 21474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.394234 (+297552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.142030 (+747796us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54000ac3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:19.142198 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655370496}
I20260709 12:24:19.570257 28117 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.573194 28202 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.587592 27931 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.654543 28175 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.957s	user 0.006s	sys 0.007s
W20260709 12:24:19.598256 28191 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.094s	user 0.002s	sys 0.003s
W20260709 12:24:19.554613 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 457206us
I20260709 12:24:19.691280 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1549) took 1339 ms. Trace:
I20260709 12:24:19.621955 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.622411 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.577088 28119 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:19.693130 28119 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:19.694042 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.697141 26203 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:19.697608 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.698020 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.699142 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.699673 26203 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:19.623767 28136 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.624083 28029 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:19.691659 26120 rpcz_store.cc:276] 0709 12:24:18.351680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.352582 (+   902us) service_pool.cc:224] Handling call
0709 12:24:18.353004 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.353281 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.353318 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:18.353911 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:24:18.440544 (+ 86633us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:18.440591 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.440689 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.597535 (+1156846us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:19.691009 (+ 93474us) raft_consensus.cc:1685] finished
0709 12:24:19.691031 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.691224 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12515584,"thread_start_us":86115,"threads_started":1,"wal-append.queue_time_us":109539}
I20260709 12:24:19.702147 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.703357 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:19.631090 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.632547 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:19.225790 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.022s	sys 0.059s
W20260709 12:24:19.704934 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.165s	user 0.030s	sys 0.083s
W20260709 12:24:19.705318 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.17319e+06us
W20260709 12:24:19.706277 28152 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.023s	user 0.002s	sys 0.007s
I20260709 12:24:19.706691 28152 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.571317 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.707806 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:19.708437 26788 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:19.709507 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2262) took 1002 ms. Trace:
I20260709 12:24:19.710005 26129 rpcz_store.cc:276] 0709 12:24:18.706948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.571068 (+864120us) service_pool.cc:224] Handling call
0709 12:24:19.708250 (+137182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.651515 26122 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.567533 28143 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:19.657289 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.589193 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.642177 28121 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.221657 28151 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.213781 28075 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.460s
I20260709 12:24:19.672475 27878 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:24:19.673451 28001 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:24:19.675377 27931 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.673072 27798 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.676107 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.675897 28084 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:19.717041 28213 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.220417 28133 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.676226 28163 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.676482 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:19.677845 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1443) took 1552 ms (client timeout 1962 ms). Trace:
I20260709 12:24:19.681067 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.757845 28202 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.675865 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.682531 28128 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.684372 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:19.715327 28175 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.703959 26785 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:19.713564 26201 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 2.
I20260709 12:24:19.714247 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.718812 26789 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:19.709430 27975 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.001s	user 0.010s	sys 0.003s
I20260709 12:24:19.774502 27975 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.721107 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:19.719328 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:19.666714 26200 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
W20260709 12:24:19.735386 26757 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.623s (SENT)
I20260709 12:24:19.776404 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:19.776587 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57296.5us
I20260709 12:24:19.777427 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:19.736333 26053 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
W20260709 12:24:19.714787 28197 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.006s	user 0.012s	sys 0.021s
I20260709 12:24:19.739876 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:19.676178 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.746167 26195 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.747892 26051 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.788872 26198 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:19.751855 26196 rpcz_store.cc:269] 0709 12:24:18.125427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.125642 (+   215us) service_pool.cc:224] Handling call
0709 12:24:19.677654 (+1552012us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400397954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:19.677789 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341923584}
I20260709 12:24:19.753657 26047 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:19.745393 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:19.756554 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.756810 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.768419 26790 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:19.797225 26202 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:19.768615 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2300) took 1689 ms. Trace:
I20260709 12:24:19.800755 26052 rpcz_store.cc:276] 0709 12:24:18.078773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.132948 (+ 54175us) service_pool.cc:224] Handling call
0709 12:24:18.161521 (+ 28573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.216925 (+ 55404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.768374 (+1551449us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b5400227754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:19.768556 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125574400}
W20260709 12:24:19.769894 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 85633.8us
I20260709 12:24:19.689744 27976 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:19.769645 27589 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:19.773975 26046 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.784766 26201 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.809484 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:19.726459 28166 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.002s	sys 0.039s
I20260709 12:24:19.794488 26050 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.795127 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:19.777204 28196 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:19.750741 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1433) took 3059 ms. Trace:
I20260709 12:24:19.806762 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1548) took 1458 ms. Trace:
I20260709 12:24:19.767236 28117 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:19.812282 26196 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:19.757910 28133 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.815069 26789 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:19.815032 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 313) took 1681 ms. Trace:
W20260709 12:24:19.795835 28197 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.012s	sys 0.021s
I20260709 12:24:19.816155 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.767570 28163 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.741053 28164 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:24:19.805644 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:19.767767 28213 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:19.804975 28091 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.003s
I20260709 12:24:19.784443 28001 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.826928 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2240) took 1458 ms. Trace:
I20260709 12:24:19.803742 28029 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:19.828461 27750 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.254s	user 0.012s	sys 0.000s
I20260709 12:24:19.828835 26194 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 2: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.753330 27798 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.829926 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.830758 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.833235 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.793545 28152 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:19.872447 26047 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.872525 26199 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:19.837607 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.845541 28147 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.834371 26128 rpcz_store.cc:276] 0709 12:24:18.348292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.350582 (+  2290us) service_pool.cc:224] Handling call
0709 12:24:18.351085 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.531629 (+180544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.531664 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:18.532129 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:24:18.684519 (+152390us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:18.684574 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.793296 (+1108722us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:19.793361 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.793422 (+    61us) raft_consensus.cc:1685] finished
0709 12:24:19.793442 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.806680 (+ 13238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23643,"raft.queue_time_us":19667,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8676,"spinlock_wait_cycles":370569472,"thread_start_us":123987,"threads_started":1,"wal-append.queue_time_us":467179}
I20260709 12:24:19.844525 26203 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
W20260709 12:24:19.820648 28085 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.002s
I20260709 12:24:19.803764 28122 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:19.808169 28128 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.817538 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:19.822844 28193 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.000s
I20260709 12:24:19.850484 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:19.851424 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 314) took 1715 ms. Trace:
I20260709 12:24:19.908183 26196 rpcz_store.cc:276] 0709 12:24:18.135799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.790215 (+1654416us) service_pool.cc:224] Handling call
0709 12:24:19.806380 (+ 16165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.851028 (+ 44648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.851348 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:19.824672 28166 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.002s	sys 0.041s
W20260709 12:24:19.853942 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48396.7us
I20260709 12:24:19.786295 28202 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.809046 26787 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:19.872117 28136 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.858481 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.862722 26124 rpcz_store.cc:276] 0709 12:24:18.368816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.368993 (+   177us) service_pool.cc:224] Handling call
0709 12:24:18.369482 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.459574 (+ 90092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.459621 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:18.504112 (+ 44491us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.037910 (+533798us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:24:19.193365 (+155455us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b1800304ea0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:19.649045 (+455680us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:19.649086 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.649169 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.826707 (+177538us) raft_consensus.cc:1685] finished
0709 12:24:19.826731 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.826866 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89549,"raft.queue_time_us":141768,"raft.run_cpu_time_us":2281,"raft.run_wall_time_us":511251,"spinlock_wait_cycles":13152512,"thread_start_us":455461,"threads_started":1,"wal-append.queue_time_us":1246248}
I20260709 12:24:19.843432 26201 rpcz_store.cc:276] 0709 12:24:18.133899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.784265 (+1650366us) service_pool.cc:224] Handling call
0709 12:24:19.784601 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.814769 (+ 30168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.814973 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.927150 26053 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:19.874676 26120 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:19.830200 27589 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:19.842885 28067 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:24:19.858330 28196 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.901638 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.864822 28133 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.793081 28143 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.906497 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.937191 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:19.937894 26047 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:19.866204 28164 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:19.910353 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 92899.1us
I20260709 12:24:19.913640 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:19.849933 28117 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.922164 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:19.927512 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.919557 28193 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.872608 28029 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.947751 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:19.934072 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.937922 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:19.904136 28085 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:19.964493 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2230) took 3133 ms (client timeout 3234 ms). Trace:
W20260709 12:24:19.964962 26125 rpcz_store.cc:269] 0709 12:24:16.830910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.194494 (+363584us) service_pool.cc:224] Handling call
0709 12:24:19.964303 (+2769809us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b54001f81d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:19.964425 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260903680}
I20260709 12:24:19.939965 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.940114 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.868209 28001 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.928480 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.900786 28147 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.833920 26197 rpcz_store.cc:276] 0709 12:24:16.685651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.875498 (+1189847us) service_pool.cc:224] Handling call
0709 12:24:17.875962 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.961484 (+ 85522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.725107 (+1763623us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b540039ab54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:19.745307 (+ 20200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12308864}
I20260709 12:24:19.982719 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:19.964828 27856 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:19.983454 26782 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:19.983630 27856 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:19.984594 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:19.985369 26782 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:19.985040 27856 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.986234 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:19.986795 26782 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.916863 28211 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.934721 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.989065 26128 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:19.927559 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 188386us
I20260709 12:24:19.989065 28154 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:19.948645 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:19.940680 28164 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:19.947103 28145 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:19.991724 28145 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:19.950121 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:19.919318 28121 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:19.872818 27750 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.932152 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.931041 28114 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.997042 26050 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:19.996479 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:19.999253 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:20.001942 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51861us
I20260709 12:24:19.959079 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1455) took 1572 ms. Trace:
W20260709 12:24:19.937492 28179 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.003s
W20260709 12:24:19.961115 28216 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:24:19.961700 28214 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:24:20.008957 28179 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.961895 28143 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.010176 27750 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.010955 26048 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:20.011309 27773 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:20.012512 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:20.013408 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:20.013643 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:20.014533 26048 raft_consensus.cc:2410] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:20.015353 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:20.015507 28152 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:20.015882 28152 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:19.962285 28207 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:24:20.016525 26046 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
I20260709 12:24:20.017752 26046 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.017822 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2248) took 1477 ms. Trace:
I20260709 12:24:20.018317 26121 rpcz_store.cc:276] 0709 12:24:18.540005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.540153 (+   148us) service_pool.cc:224] Handling call
0709 12:24:18.540476 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.553881 (+ 13405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.553916 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:18.569438 (+ 15522us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.123104 (+553666us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:19.123153 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.123226 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.017560 (+894334us) raft_consensus.cc:1685] finished
0709 12:24:20.017610 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.017758 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":584,"raft.queue_time_us":7552,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":14560,"thread_start_us":484678,"threads_started":1,"wal-append.queue_time_us":1122496}
W20260709 12:24:20.019866 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2259) took 1313 ms (client timeout 1727 ms). Trace:
W20260709 12:24:20.020318 26122 rpcz_store.cc:269] 0709 12:24:18.705987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.133130 (+427143us) service_pool.cc:224] Handling call
0709 12:24:20.019673 (+886543us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b540006fed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.019810 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183011072}
I20260709 12:24:20.020533 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.021442 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.022713 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.023633 28128 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:20.023806 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:19.963521 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:20.024784 26051 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:20.025853 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:20.026652 26197 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:20.027490 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:19.966200 26203 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:20.007447 27748 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:20.029174 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:20.030375 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:19.996784 28175 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:19.974932 28157 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 2.066s	user 0.000s	sys 0.011s
I20260709 12:24:20.032972 28163 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.977082 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.034332 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.034896 26787 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:20.035771 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:20.036581 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 264) took 3202 ms. Trace:
I20260709 12:24:20.007822 28220 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:19.979158 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:20.037397 26782 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:20.038761 27798 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:20.038949 26194 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:19.964130 28151 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:20.004655 26127 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.041021 28147 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.008109 28114 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.036593 26052 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:20.005088 26120 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:20.044905 28114 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:19.964694 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.031939 26203 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:20.045887 26783 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:20.046422 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1458) took 1151 ms. Trace:
I20260709 12:24:20.046839 26203 rpcz_store.cc:276] 0709 12:24:18.894793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.031314 (+1136521us) service_pool.cc:224] Handling call
0709 12:24:20.031619 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.045847 (+ 14228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.046357 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.006525 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:19.970161 27878 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.993091 28145 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:19.936399 28136 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:19.971521 27975 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:20.039907 26125 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:20.045221 28001 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.005740 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:20.049314 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46515.6us
I20260709 12:24:20.037101 26126 rpcz_store.cc:276] 0709 12:24:16.834526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.329290 (+494764us) service_pool.cc:224] Handling call
0709 12:24:17.329711 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:17.457911 (+128200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.457951 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:17.463474 (+  5523us) log.cc:844] Serialized 52 byte log entry
0709 12:24:17.697577 (+234103us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:17.697678 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.697753 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.974395 (+2276642us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:20.036299 (+ 61904us) raft_consensus.cc:1685] finished
0709 12:24:20.036321 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.036523 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18936,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4790,"spinlock_wait_cycles":14066944,"thread_start_us":103405,"threads_started":1,"wal-append.queue_time_us":300259}
I20260709 12:24:20.028043 26197 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.055312 26127 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:20.052508 26051 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:20.053397 27878 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.053869 26787 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.056556 26122 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.058709 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:20.059499 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.078039 26126 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:20.056918 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:20.067384 26121 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.175442 26122 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:20.129251 28152 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:20.175556 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 116941us
I20260709 12:24:20.070726 26194 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.176939 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.131534 27878 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:20.008136 26198 rpcz_store.cc:276] 0709 12:24:18.387021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.819335 (+1432314us) service_pool.cc:224] Handling call
0709 12:24:19.847241 (+ 27906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.847499 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:19.914870 (+ 67371us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.958747 (+ 43877us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.958798 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:19.958841 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.958905 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.959022 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34259,"prepare.queue_time_us":125380,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":23914,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":25272,"thread_start_us":72464,"threads_started":2,"wal-append.queue_time_us":35805}
I20260709 12:24:20.038800 26782 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.098400 28147 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:20.180989 28147 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:20.064884 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:20.181943 28147 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.184006 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2306) took 1033 ms. Trace:
I20260709 12:24:20.184419 26048 rpcz_store.cc:276] 0709 12:24:19.151086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.183123 (+1032037us) service_pool.cc:224] Handling call
0709 12:24:20.183412 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.183593 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.183625 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.183679 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.183729 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.183814 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.183931 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.176882 28152 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:20.101643 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.178267 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.186236 26122 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:20.057314 28151 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:20.100390 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.174706 26126 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:20.179116 26194 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:20.202049 26194 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:20.203487 26194 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:20.204895 26194 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:20.177623 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:20.056221 26053 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:20.206591 27878 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.206682 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.207114 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:20.209647 28084 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:20.190114 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.190565 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.191311 27748 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.196635 28151 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:20.197007 26121 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:20.200112 28175 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.209663 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.094962 28001 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:20.210260 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.213493 26122 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:20.221946 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.222882 26122 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:20.213239 27685 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:20.213416 28085 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:20.226465 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:20.232429 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.240506 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.226192 27748 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.242004 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:20.220942 26782 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:20.242784 26122 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:20.243229 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:20.243548 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:20.241778 27748 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:20.244378 28001 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.244524 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.244956 28001 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:20.251264 26122 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:20.252799 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2260) took 1546 ms. Trace:
I20260709 12:24:20.254771 26122 rpcz_store.cc:276] 0709 12:24:18.706320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.243330 (+1537010us) service_pool.cc:224] Handling call
0709 12:24:20.252724 (+  9394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.266644 26053 pending_rounds.cc:85] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 1
I20260709 12:24:20.267189 26053 pending_rounds.cc:107] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:24:20.267474 26053 raft_consensus.cc:2889] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:20.276520 28127 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:20.277180 28127 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:20.278543 28127 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:20.281848 28029 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.283392 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2255) took 1559 ms. Trace:
I20260709 12:24:20.284657 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1039) took 1063 ms. Trace:
I20260709 12:24:20.285241 26202 rpcz_store.cc:276] 0709 12:24:19.220864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.569891 (+349027us) service_pool.cc:224] Handling call
0709 12:24:20.284329 (+714438us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b5400302d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.284569 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184741760}
I20260709 12:24:20.287312 28127 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:20.283962 26125 rpcz_store.cc:276] 0709 12:24:18.704547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.263213 (+1558666us) service_pool.cc:224] Handling call
0709 12:24:20.263659 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.263878 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.263907 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.263951 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.263994 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.264075 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.264211 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:20.295083 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.296921 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2299) took 2212 ms (client timeout 1555 ms). Trace:
W20260709 12:24:20.297518 26054 rpcz_store.cc:269] 0709 12:24:18.077978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.078158 (+   180us) service_pool.cc:224] Handling call
0709 12:24:20.289976 (+2211818us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400228654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.290177 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182317952}
I20260709 12:24:20.299970 26125 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:20.301944 26129 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.310173 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 270) took 1594 ms. Trace:
I20260709 12:24:20.310573 26129 rpcz_store.cc:276] 0709 12:24:18.707887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.291015 (+1583128us) service_pool.cc:224] Handling call
0709 12:24:20.301773 (+ 10758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.302510 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.302726 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.311172 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2263) took 1593 ms. Trace:
I20260709 12:24:20.311554 26125 rpcz_store.cc:276] 0709 12:24:18.707253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.289476 (+1582223us) service_pool.cc:224] Handling call
0709 12:24:20.299632 (+ 10156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.300460 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.300663 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.222524 28175 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.258490 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.318830 26129 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.232779 27685 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:20.246285 28085 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.327024 26125 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.315980 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2302) took 1166 ms. Trace:
I20260709 12:24:20.318301 27748 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.343675 26125 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:20.340595 26782 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:20.335439 28152 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.333804 26786 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:20.351629 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.361780 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.362452 26789 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:20.354053 26124 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:20.355690 26125 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:20.343977 27748 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:20.322542 28151 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.375231 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.375979 26789 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:20.039912 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:20.367991 25950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 292) took 9703 ms. Trace:
I20260709 12:24:20.368889 26124 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:20.345572 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:20.386524 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.392863 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2310) took 1171 ms. Trace:
I20260709 12:24:20.337044 26046 rpcz_store.cc:276] 0709 12:24:19.148064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.018671 (+870607us) service_pool.cc:224] Handling call
0709 12:24:20.018986 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.019247 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:20.152404 (+133157us) log.cc:844] Serialized 14 byte log entry
0709 12:24:20.247064 (+ 94660us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.288802 (+ 41738us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:20.292369 (+  3567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.299690 (+  7321us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.314198 (+ 14508us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":156386,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":39,"raft.queue_time_us":2623,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":8351,"thread_start_us":203874,"threads_started":2,"wal-append.queue_time_us":163402}
W20260709 12:24:20.400025 28234 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:24:20.402987 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:20.370066 28242 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:24:20.406610 26050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:24:20.365453 27685 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.406944 26124 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:20.450613 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.450812 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.418218 26790 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:20.451099 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.451633 26782 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:20.452530 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:20.453259 26782 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:20.453398 26790 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.454293 26782 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Advancing to term 5
I20260709 12:24:20.455595 26790 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.403334 26054 rpcz_store.cc:276] 0709 12:24:19.221350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.298524 (+1077174us) service_pool.cc:224] Handling call
0709 12:24:20.312145 (+ 13621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.312526 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.312556 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.312989 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.377669 (+ 64680us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:20.377709 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.377772 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.392548 (+ 14776us) raft_consensus.cc:1685] finished
0709 12:24:20.392587 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.392775 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1844,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2494,"thread_start_us":45225,"threads_started":1,"wal-append.queue_time_us":68635}
I20260709 12:24:20.401892 27878 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.458292 26050 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:20.460057 26050 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:20.385852 28067 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:20.491353 26126 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
W20260709 12:24:20.476987 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.363040 28152 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.477274 28001 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.513624 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.570605 26790 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:20.589926 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:20.591548 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.381815 25950 rpcz_store.cc:276] 0709 12:24:10.664766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.665064 (+   298us) service_pool.cc:224] Handling call
0709 12:24:20.367918 (+9702854us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:16.572140 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:16.753732 (+181592us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:16.753790 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:24:16.753820 (+    30us) tablet.cc:662] Decoding operations
0709 12:24:16.754620 (+   800us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:16.754689 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:16.754722 (+    33us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:24:16.755376 (+   654us) tablet.cc:701] Row locks acquired
0709 12:24:16.755423 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:24:16.755601 (+   178us) write_op.cc:270] Start()
0709 12:24:16.755754 (+   153us) write_op.cc:276] Timestamp: P: 1783599856755534 usec, L: 0
0709 12:24:16.755773 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:24:16.859211 (+103438us) log.cc:844] Serialized 2079 byte log entry
0709 12:24:17.678912 (+819701us) spinlock_profiling.cc:243] Waited 8.99 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b
0709 12:24:17.973197 (+294285us) op_driver.cc:464] REPLICATION: finished
0709 12:24:18.041492 (+ 68295us) write_op.cc:301] APPLY: starting
0709 12:24:18.041607 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:18.041834 (+   227us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:18.041880 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:18.046224 (+  4344us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:18.046448 (+   224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:24:18.046494 (+    46us) write_op.cc:312] APPLY: finished
0709 12:24:18.046822 (+   328us) log.cc:844] Serialized 47 byte log entry
0709 12:24:18.094936 (+ 48114us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:18.107228 (+ 12292us) write_op.cc:454] Released schema lock
0709 12:24:18.107271 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:18.107650 (+   379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:18.107678 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2763883,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":67990,"apply.run_cpu_time_us":3228,"apply.run_wall_time_us":66168,"num_ops":5,"prepare.queue_time_us":268359,"prepare.run_cpu_time_us":56008,"prepare.run_wall_time_us":1281854,"raft.queue_time_us":38616,"raft.run_cpu_time_us":25914,"raft.run_wall_time_us":61319,"replication_time_us":1217267,"spinlock_wait_cycles":20687744,"thread_start_us":1276035,"threads_started":4,"wal-append.queue_time_us":1037653,"wal-append.run_cpu_time_us":12509,"wal-append.run_wall_time_us":1197699}]]}
I20260709 12:24:20.401656 27748 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:20.500958 28067 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.770109 26790 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:20.641425 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.773679 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148111us
W20260709 12:24:20.737922 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.741374 26050 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.783622 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:20.785234 26200 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:20.785831 26050 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:20.747295 26787 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.001s
W20260709 12:24:20.600601 28224 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:24:20.673550 26049 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.005s
W20260709 12:24:20.644771 28244 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:24:20.787979 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:20.636811 28247 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:24:20.787881 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:20.641062 28245 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:24:20.791057 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:47804 (request call id 1064) took 1209 ms. Trace:
I20260709 12:24:20.793315 26050 rpcz_store.cc:276] 0709 12:24:19.577501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.374681 (+797180us) service_pool.cc:224] Handling call
0709 12:24:20.786427 (+411746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.754410 26782 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:20.678557 27748 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:20.784198 28084 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.003s
I20260709 12:24:20.799098 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:20.623102 26046 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:20.802021 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.802474 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:20.804654 26202 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:20.803298 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.781620 27773 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:20.812575 26201 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:20.812486 27773 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.815268 27773 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:20.807104 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:20.817412 26790 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 2: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:20.818338 27931 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:20.818616 26790 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.818885 27931 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.819375 27931 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:20.810294 28215 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.045s	user 0.002s	sys 0.000s
I20260709 12:24:20.792057 26049 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:20.821441 27931 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.815670 28117 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:20.816390 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:20.822535 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.822749 26202 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:20.824393 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 635) took 1115 ms. Trace:
I20260709 12:24:20.820008 26790 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEARNER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:20.827925 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.829221 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 269) took 2121 ms (client timeout 1623 ms). Trace:
W20260709 12:24:20.829828 26123 rpcz_store.cc:269] 0709 12:24:18.707584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.970825 (+263241us) service_pool.cc:224] Handling call
0709 12:24:20.828948 (+1858123us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.829135 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134075776}
I20260709 12:24:20.787014 26787 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:20.835240 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:20.836563 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1065) took 1258 ms. Trace:
I20260709 12:24:20.806773 28084 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:20.794450 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:20.838670 26197 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:20.838800 26790 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:20.834543 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:20.841265 26782 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:20.842474 28067 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:20.844528 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1057) took 1662 ms. Trace:
I20260709 12:24:20.844916 26053 rpcz_store.cc:276] 0709 12:24:19.201737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.265890 (+1064153us) service_pool.cc:224] Handling call
0709 12:24:20.266330 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.285366 (+ 19036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.285417 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.286017 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.330958 (+ 44941us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:20.330996 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.331084 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.844325 (+513241us) raft_consensus.cc:1685] finished
0709 12:24:20.844345 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.844473 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":44486,"threads_started":1,"wal-append.queue_time_us":79970}
I20260709 12:24:20.846674 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:20.824812 26788 rpcz_store.cc:276] 0709 12:24:19.709315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.717805 (+  8490us) service_pool.cc:224] Handling call
0709 12:24:20.259285 (+541480us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:20.259564 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.260166 (+   602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.260208 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.260852 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.323980 (+ 63128us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:20.324029 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.324091 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.824080 (+499989us) raft_consensus.cc:1685] finished
0709 12:24:20.824106 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.824325 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4429,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":190,"spinlock_wait_cycles":3116928,"thread_start_us":62599,"threads_started":1,"wal-append.queue_time_us":145303}
I20260709 12:24:20.832646 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:20.849162 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 273) took 1249 ms. Trace:
I20260709 12:24:20.849526 26121 rpcz_store.cc:276] 0709 12:24:19.600131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.020193 (+420062us) service_pool.cc:224] Handling call
0709 12:24:20.848972 (+828779us) spinlock_profiling.cc:243] Waited 46.1 ms on lock 0x7b54001d1fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.849106 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105963904}
I20260709 12:24:20.850662 26787 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:24:20.852208 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2261) took 2145 ms (client timeout 2361 ms). Trace:
I20260709 12:24:20.837026 26046 rpcz_store.cc:276] 0709 12:24:19.577828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.459234 (+881406us) service_pool.cc:224] Handling call
0709 12:24:20.588857 (+129623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.807306 (+218449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.836499 (+ 29193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.844002 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1056) took 1662 ms. Trace:
W20260709 12:24:20.852761 26127 rpcz_store.cc:269] 0709 12:24:18.706650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.611748 (+905098us) service_pool.cc:224] Handling call
0709 12:24:20.851976 (+1240228us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54003d1654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.852130 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900579200}
I20260709 12:24:20.847926 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:20.848690 26788 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:20.589025 27878 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 12:24:20.857728 26050 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:20.858287 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.860049 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1070) took 1236 ms. Trace:
I20260709 12:24:20.860561 26050 rpcz_store.cc:276] 0709 12:24:19.624054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.807705 (+1183651us) service_pool.cc:224] Handling call
0709 12:24:20.856950 (+ 49245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.859775 (+  2825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.859991 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.853859 26048 rpcz_store.cc:276] 0709 12:24:19.181441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.186227 (+1004786us) service_pool.cc:224] Handling call
0709 12:24:20.214523 (+ 28296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.221802 (+  7279us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:20.241316 (+ 19514us) log.cc:844] Serialized 14 byte log entry
0709 12:24:20.314577 (+ 73261us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.320555 (+  5978us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.320710 (+   155us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:20.320984 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 12:24:20.321186 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.321249 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.843793 (+522544us) raft_consensus.cc:1685] finished
0709 12:24:20.843814 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.843942 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":846,"prepare.queue_time_us":20380,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":6237,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":6641,"spinlock_wait_cycles":33932928,"thread_start_us":71889,"threads_started":1,"wal-append.queue_time_us":92276}
W20260709 12:24:20.871846 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 204192us
W20260709 12:24:20.872210 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.873950 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.876142 26052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.028s	sys 0.037s
W20260709 12:24:20.876938 26052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.028s	sys 0.037s
W20260709 12:24:20.877707 26756 connection.cc:256] RPC call timeout handler was delayed by -1.0863274039998032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 12:24:20.878340 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:20.886168 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:20.885524 27931 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:20.908403 27931 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.908555 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2264) took 1243 ms. Trace:
I20260709 12:24:20.908885 27931 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.909017 26129 rpcz_store.cc:276] 0709 12:24:19.665301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.322519 (+657218us) service_pool.cc:224] Handling call
0709 12:24:20.322811 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.330599 (+  7788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.330636 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.331224 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.499032 (+167808us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:20.499102 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.499202 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.908316 (+409114us) raft_consensus.cc:1685] finished
0709 12:24:20.908338 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.908476 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4337,"raft.queue_time_us":21905,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":13724,"thread_start_us":148126,"threads_started":1,"wal-append.queue_time_us":543748}
I20260709 12:24:20.912621 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:20.912760 26785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 444) took 1721 ms (client timeout 2057 ms). Trace:
I20260709 12:24:20.913578 26046 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:20.913321 26785 rpcz_store.cc:269] 0709 12:24:19.191848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.192128 (+   280us) service_pool.cc:224] Handling call
0709 12:24:20.912555 (+1720427us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.912691 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299687168}
I20260709 12:24:20.914722 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:20.915558 26046 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:20.915727 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:20.916417 26049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1059) took 1599 ms (client timeout 1705 ms). Trace:
W20260709 12:24:20.905258 26199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1031) took 1807 ms (client timeout 1662 ms). Trace:
W20260709 12:24:20.917742 26199 rpcz_store.cc:269] 0709 12:24:19.097884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.159976 (+ 62092us) service_pool.cc:224] Handling call
0709 12:24:19.630892 (+470916us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:20.904923 (+1274031us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400378554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.905145 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425423360}
I20260709 12:24:20.885464 27773 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:24:20.916471 26791 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.919210 28213 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.920954 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1556) took 1233 ms. Trace:
I20260709 12:24:20.921324 26126 rpcz_store.cc:276] 0709 12:24:19.687089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.206448 (+519359us) service_pool.cc:224] Handling call
0709 12:24:20.920771 (+714323us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.920897 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625294080}
I20260709 12:24:20.922465 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2266) took 1256 ms. Trace:
W20260709 12:24:20.922572 26167 connection.cc:256] RPC call timeout handler was delayed by -1.0536210579998624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260709 12:24:20.922866 26120 rpcz_store.cc:276] 0709 12:24:19.667227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.928225 (+260998us) service_pool.cc:224] Handling call
0709 12:24:20.922310 (+994085us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b54001f68d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.922412 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87127936}
I20260709 12:24:20.923889 26122 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:20.917008 26049 rpcz_store.cc:269] 0709 12:24:19.317616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.680768 (+363152us) service_pool.cc:224] Handling call
0709 12:24:20.904535 (+1223767us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:20.916359 (+ 11824us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619585920}
I20260709 12:24:20.906651 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1062) took 1330 ms. Trace:
I20260709 12:24:20.924541 28257 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:20.925457 26167 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.570s (SENT)
I20260709 12:24:20.924798 26052 rpcz_store.cc:276] 0709 12:24:19.576717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.299049 (+722332us) service_pool.cc:224] Handling call
0709 12:24:20.314657 (+ 15608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.317303 (+  2646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.317347 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.319409 (+  2062us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.877313 (+557904us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:20.877369 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.877469 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.906434 (+ 28965us) raft_consensus.cc:1685] finished
0709 12:24:20.906451 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.906588 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2242,"raft.queue_time_us":2367,"spinlock_wait_cycles":5002112,"thread_start_us":539432,"threads_started":1,"wal-append.queue_time_us":574366}
W20260709 12:24:20.927676 26167 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.614s (SENT)
I20260709 12:24:20.925817 26049 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.927652 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1073) took 1289 ms. Trace:
I20260709 12:24:20.930213 26052 rpcz_store.cc:276] 0709 12:24:19.638177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.926817 (+1288640us) service_pool.cc:224] Handling call
0709 12:24:20.927081 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.927360 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.927417 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.927460 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:20.927495 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.927534 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.927612 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129}
I20260709 12:24:20.929109 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:20.931996 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:20.938596 26791 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:20.941143 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1075) took 1302 ms. Trace:
I20260709 12:24:20.944299 26052 rpcz_store.cc:276] 0709 12:24:19.638924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.931838 (+1292914us) service_pool.cc:224] Handling call
0709 12:24:20.932107 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.940547 (+  8440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.940594 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.940654 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:20.940721 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.940803 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.940956 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162}
I20260709 12:24:20.947773 26052 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:24:20.948195 26052 pending_rounds.cc:85] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 6
I20260709 12:24:20.946626 28151 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:20.946650 28127 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.949970 28127 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:20.950800 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1072) took 1323 ms. Trace:
I20260709 12:24:20.951261 26049 rpcz_store.cc:276] 0709 12:24:19.624677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.925064 (+1300387us) service_pool.cc:224] Handling call
0709 12:24:20.925413 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.933307 (+  7894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.947925 (+ 14618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.952994 26052 pending_rounds.cc:107] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:24:20.953364 26052 raft_consensus.cc:2889] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:20.931263 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:20.955968 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:20.957365 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1076) took 1002 ms. Trace:
I20260709 12:24:20.957864 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:20.957846 26052 rpcz_store.cc:276] 0709 12:24:19.954672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.945435 (+990763us) service_pool.cc:224] Handling call
0709 12:24:20.945737 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.957046 (+ 11309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.957296 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.956722 27931 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:20.960637 28151 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:20.977460 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:20.967235 26054 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:20.978976 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1066) took 1400 ms. Trace:
I20260709 12:24:20.979437 26054 rpcz_store.cc:276] 0709 12:24:19.578147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.966508 (+1388361us) service_pool.cc:224] Handling call
0709 12:24:20.966873 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.978624 (+ 11751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.978893 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.955833 28067 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:24:20.958985 28127 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:20.963912 26046 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:20.971904 28257 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:20.964776 28151 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:20.967518 26049 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
W20260709 12:24:20.997572 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57467.3us
I20260709 12:24:20.990257 27878 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:24:20.998699 27878 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:20.998952 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:20.999303 27878 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:20.993201 26047 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:20.962757 27798 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:20.992662 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:20.962342 27773 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:20.966167 27931 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:20.991884 28196 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:21.009843 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52112.6us
I20260709 12:24:20.990620 27748 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:24:21.007386 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:20.981583 28145 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.005352 26049 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 4
I20260709 12:24:21.001725 28151 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:20.995931 28257 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:20.997391 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2304) took 1847 ms. Trace:
I20260709 12:24:21.010080 28242 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.014020 27798 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:21.014254 28001 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:21.014523 26049 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 12:24:21.002408 28254 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:24:21.003490 28261 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:21.016538 27931 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:21.045279 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.046149 26199 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:21.033423 28193 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.052664 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:21.054436 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.050463 28257 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.055284 26787 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:21.055721 28257 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.057895 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.047354 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 630) took 1337 ms. Trace:
I20260709 12:24:21.060842 26784 rpcz_store.cc:276] 0709 12:24:19.706497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.706618 (+   121us) service_pool.cc:224] Handling call
0709 12:24:19.706928 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.758153 (+ 51225us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:19.828274 (+ 70121us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.971279 (+143005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.995504 (+ 24225us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.008287 (+ 12783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.032155 (+1023868us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5400409a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:21.035265 (+  3110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.043724 (+  8459us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":75136,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3096,"raft.queue_time_us":10731,"spinlock_wait_cycles":45243520,"thread_start_us":118318,"threads_started":1,"wal-append.queue_time_us":124043,"wal-append.run_cpu_time_us":1421,"wal-append.run_wall_time_us":1001429}
I20260709 12:24:21.039348 26049 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.063603 28257 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:21.064755 27798 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.016113 28145 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.069844 28145 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.042265 28263 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:21.077112 28263 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:21.077504 28263 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:21.064047 28257 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:21.036968 28001 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:21.083742 28001 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:21.083470 28257 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.084860 28257 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.085119 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:21.080312 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 69005.4us
I20260709 12:24:21.090070 28122 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.090615 28122 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.044425 27931 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:21.092020 27931 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.092818 28122 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:21.096424 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.097043 26053 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.017212 26045 rpcz_store.cc:276] 0709 12:24:19.150471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.024746 (+874275us) service_pool.cc:224] Handling call
0709 12:24:20.025097 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.025465 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.025503 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.844161 (+818658us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b2c000014c8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:20.844750 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.946932 (+102182us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:20.953068 (+  6136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.955588 (+  2520us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.960544 (+  4956us) raft_consensus.cc:1685] finished
0709 12:24:20.960578 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.996135 (+ 35557us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10940,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":54032256,"thread_start_us":88048,"threads_started":1,"wal-append.queue_time_us":97589}
I20260709 12:24:21.080811 28067 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:21.079967 28263 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.082065 28151 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.097694 28193 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.099608 28151 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:21.100138 28067 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:21.032401 27878 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.101097 27878 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.101389 26094 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:21.101265 28067 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.102550 28067 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:21.103312 26094 leader_election.cc:400] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
I20260709 12:24:21.103646 26094 leader_election.cc:403] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.106217 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.109841 28145 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
I20260709 12:24:21.109966 26045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:21.110709 27748 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:21.110778 26045 raft_consensus.cc:606] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.111385 26045 raft_consensus.cc:614] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:21.114403 27931 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.115052 28165 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:21.115471 28165 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.116489 26053 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:21.119537 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.126482 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.133248 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.133985 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.134135 26053 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:21.139020 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.141546 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.143266 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.143523 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.143914 26053 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
W20260709 12:24:21.125798 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.145040 28269 raft_consensus.cc:2764] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:24:21.150838 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 60993.7us
I20260709 12:24:21.100064 28193 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:21.152210 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:21.146111 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:21.146901 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.154270 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.155584 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.153083 28267 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:21.158224 28269 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.158766 28269 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.158658 28151 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:21.159257 28269 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.125823 27773 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:21.166600 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.132937 27748 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.164958 28265 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:24:21.136327 28001 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.176563 27931 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.180043 26045 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:21.185781 27773 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.186976 28001 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.187129 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.187747 28001 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:21.188409 26045 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:21.189213 28001 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:21.189551 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.191393 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.142833 27685 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:21.196921 27685 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.198356 27685 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:21.183008 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.184578 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:21.202775 26199 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:21.132582 28165 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:21.204301 28270 consensus_queue.cc:1175] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 4 status { last_received { term: 4 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 4 index: 7 } }
I20260709 12:24:21.204129 28165 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.192178 26045 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.152407 28268 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 12:24:21.235288 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52367.6us
I20260709 12:24:21.164707 28264 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.236394 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.141294 28145 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:24:21.241361 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 88153.6us
I20260709 12:24:21.243569 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:21.234956 28274 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:21.181262 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.234650 28266 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.173538 28254 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:24:21.245134 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:21.242134 27931 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:21.264770 26045 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:21.266307 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.267843 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.268859 26045 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
W20260709 12:24:21.269520 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.270959 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 448) took 1471 ms. Trace:
I20260709 12:24:21.271224 28268 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:21.271764 28268 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:21.271381 26783 rpcz_store.cc:276] 0709 12:24:19.799577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.046761 (+247184us) service_pool.cc:224] Handling call
0709 12:24:20.047130 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.047517 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.047554 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.048087 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.083333 (+1035246us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b18001760e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:21.244412 (+161079us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:21.248564 (+  4152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.254942 (+  6378us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.270631 (+ 15689us) raft_consensus.cc:1685] finished
0709 12:24:21.270659 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.270883 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2016,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9159,"spinlock_wait_cycles":11211520,"thread_start_us":150066,"threads_started":1,"wal-append.queue_time_us":1220893}
I20260709 12:24:21.272836 28268 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:21.273399 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.270383 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.275595 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.257604 27773 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.277047 27748 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.277690 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.258623 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:21.248222 28261 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.279214 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:21.284659 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 102908us
I20260709 12:24:21.292496 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:21.284896 26199 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:21.244725 28264 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:21.286387 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.288998 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.297492 26202 raft_consensus.cc:2410] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:24:21.298450 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.303316 28264 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.310349 28165 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:21.310844 26757 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 2.178s (SENT) [suppressed 37 similar messages]
I20260709 12:24:21.311082 28165 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.311632 28165 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.314795 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.316020 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.316737 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.313745 28278 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:21.321252 28278 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.286183 28261 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:21.316062 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.317404 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.318068 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.330892 26052 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:21.290438 28151 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.331264 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:21.331750 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.289673 27773 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.332885 26200 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:21.338269 26049 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:21.338724 26049 pending_rounds.cc:85] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 5
I20260709 12:24:21.338985 26049 pending_rounds.cc:107] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:21.339223 26049 raft_consensus.cc:2889] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.339356 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.345825 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.346527 26199 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:21.315092 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.322597 28278 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.365974 27748 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.370636 27748 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.374394 27748 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:21.375092 26197 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:21.377497 26201 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:21.334434 28261 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.334645 28151 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:21.378031 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.376262 28278 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.359179 28165 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:21.380702 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64127.2us
I20260709 12:24:21.388355 28151 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.393654 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.404915 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.344967 27773 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.407588 28282 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.407806 28270 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.378680 27748 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:21.411567 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.412423 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 168935us
I20260709 12:24:21.415884 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.417279 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.418026 26789 raft_consensus.cc:2410] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:21.400913 28275 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:21.418473 26787 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:21.419067 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:21.419055 28269 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.422431 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.423710 26791 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:21.433499 28241 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.067s	user 0.016s	sys 0.003s
I20260709 12:24:21.435269 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1461) took 1565 ms. Trace:
I20260709 12:24:21.435659 26198 rpcz_store.cc:276] 0709 12:24:19.869421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.181066 (+311645us) service_pool.cc:224] Handling call
0709 12:24:20.181379 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.181716 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.181743 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.186919 (+  5176us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.294793 (+107874us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.295790 (+   997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.295877 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.432723 (+1136846us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:21.434944 (+  2221us) raft_consensus.cc:1685] finished
0709 12:24:21.434967 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.435203 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":29307264,"thread_start_us":78497,"threads_started":1,"wal-append.queue_time_us":103409}
I20260709 12:24:21.437879 26784 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:21.420655 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.430542 28275 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.419456 28136 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:21.421360 27748 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.442440 28275 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.444205 27748 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:21.442616 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 641) took 1572 ms (client timeout 1583 ms). Trace:
I20260709 12:24:21.449489 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.449805 28175 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.475278 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.433517 27773 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:21.544727 26194 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:21.482570 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:21.555446 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.690290 28278 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:21.565805 28087 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:21.481151 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.597074 28288 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:21.690902 28278 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:21.542901 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1034) took 2440 ms (client timeout 1525 ms). Trace:
I20260709 12:24:21.709292 28278 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.709041 28164 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:21.709321 26195 rpcz_store.cc:269] 0709 12:24:19.100993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.125990 (+ 24997us) service_pool.cc:224] Handling call
0709 12:24:21.540975 (+2414985us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:21.541177 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248372096}
I20260709 12:24:21.710314 28164 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.713315 25952 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.715234 25952 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.707758 28269 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:21.717463 26046 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:21.667800 28261 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:21.471122 26786 rpcz_store.cc:269] 0709 12:24:19.870551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.920014 (+ 49463us) service_pool.cc:224] Handling call
0709 12:24:21.442427 (+1522413us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:21.442554 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656363264}
I20260709 12:24:21.540048 28085 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:21.535447 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1061) took 2034 ms (client timeout 1901 ms). Trace:
W20260709 12:24:21.719434 26047 rpcz_store.cc:269] 0709 12:24:19.500996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.958494 (+457498us) service_pool.cc:224] Handling call
0709 12:24:21.535157 (+1576663us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b5400172854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:21.535341 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2227332480}
I20260709 12:24:21.668375 28165 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.536710 26051 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:21.720448 28261 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:21.722051 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2305) took 2571 ms. Trace:
I20260709 12:24:21.722383 26051 rpcz_store.cc:276] 0709 12:24:19.150797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.065599 (+914802us) service_pool.cc:224] Handling call
0709 12:24:21.536427 (+1470828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.720661 (+184234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.721878 (+  1217us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400172850. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:21.721997 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3371269888}
I20260709 12:24:21.722553 28278 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.430853 28266 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.470903 27931 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:21.651913 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.724133 27931 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:21.723798 28087 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.725193 27931 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.726468 28087 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.591380 28267 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:21.729472 28267 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:21.730279 28267 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.731037 28267 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.685086 28075 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:21.732592 28278 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.732648 28075 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.477015 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.733953 28075 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.734596 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2253) took 3096 ms. Trace:
I20260709 12:24:21.735258 28075 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.734948 26122 rpcz_store.cc:276] 0709 12:24:18.637863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.256393 (+1618530us) service_pool.cc:224] Handling call
0709 12:24:20.923695 (+667302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.924324 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.734400 (+810076us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540036abd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:21.734539 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444838272}
I20260709 12:24:21.687064 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:21.523460 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77736us
W20260709 12:24:21.738416 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.697264 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 214799us
I20260709 12:24:21.697971 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.740502 27953 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.740989 27953 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.741930 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.671703 28270 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.743459 27953 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.699231 28263 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.698953 28151 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.701646 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.745074 26201 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:21.745827 27878 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.701887 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.702617 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.746363 27878 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.746939 26199 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:21.528416 27748 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.700531 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.748584 27878 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:21.534327 27798 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:21.749156 27798 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:21.741096 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.745074 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:21.750124 26048 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.751230 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.752709 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:21.750150 27798 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.753772 27798 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.755151 27748 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.746357 28242 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:21.756807 28242 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.757700 28242 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.758116 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1473) took 1837 ms. Trace:
I20260709 12:24:21.758968 28242 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.758544 26194 rpcz_store.cc:276] 0709 12:24:19.920604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.205693 (+285089us) service_pool.cc:224] Handling call
0709 12:24:21.542029 (+1336336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.689457 (+147428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.757879 (+ 68422us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540039a150. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:21.758050 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3070754560}
I20260709 12:24:21.760372 27878 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:21.760828 27878 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.761212 27878 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.756012 27748 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:21.768316 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.769573 28270 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.770205 26787 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:21.770087 28270 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.772027 27748 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:21.773782 27748 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.775302 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.774960 25952 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 3 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.775916 26791 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:21.776640 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:21.777338 26785 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:21.777856 25952 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 4, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.780838 26054 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:21.781996 26783 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:21.782696 28291 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.778589 26789 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:21.783208 28291 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.767208 26786 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:21.785560 28291 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:21.786123 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.786450 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:21.787384 26195 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.787529 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.786738 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.788687 26051 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.790927 26048 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:21.792649 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.792865 26049 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:21.795352 28252 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.202s	user 0.000s	sys 0.027s
I20260709 12:24:21.796657 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:21.797178 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.797621 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.799817 28151 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:21.788113 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:21.801756 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:21.802235 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.797173 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1048) took 2203 ms. Trace:
I20260709 12:24:21.809237 26196 rpcz_store.cc:276] 0709 12:24:19.593912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.063214 (+469302us) service_pool.cc:224] Handling call
0709 12:24:20.346233 (+283019us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:20.346394 (+   161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.346588 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.346605 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.350864 (+  4259us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.468312 (+117448us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:20.468370 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.468470 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.794753 (+1326283us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:21.796952 (+  2199us) raft_consensus.cc:1685] finished
0709 12:24:21.796972 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.797115 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13702272,"thread_start_us":50492,"threads_started":1,"wal-append.queue_time_us":122464}
I20260709 12:24:21.810885 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.779546 25952 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 3 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.811092 26195 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:21.811757 26048 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:21.810964 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.807121 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.813092 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.814461 25952 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:24:21.818430 26051 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:21.821904 26195 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:21.822511 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:21.823163 26018 connection.cc:256] RPC call timeout handler was delayed by -1.9139302240005236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 12:24:21.826949 26018 connection.cc:256] RPC call timeout handler was delayed by -1.7708302240002922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
W20260709 12:24:21.827309 26018 connection.cc:256] RPC call timeout handler was delayed by -1.6606303800003843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2722s for a response
W20260709 12:24:21.827796 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1533122000000731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
I20260709 12:24:21.827296 25952 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.821503 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:21.828624 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.829823 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:21.830907 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:21.832310 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.835711 26054 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:21.838994 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.841293 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.841313 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:21.835338 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:21.845161 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:21.850251 26195 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:21.849334 27953 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.852144 28165 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.852406 28008 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.823153 28164 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.827574 28151 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.827040 28263 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:21.826835 28266 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:21.834713 28268 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:21.808379 28278 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:24:21.841912 28196 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:21.877230 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.878541 26195 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:21.874714 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.848683 28085 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.852929 28165 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.909309 26196 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:21.857157 28087 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.853307 28282 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:21.937809 26019 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.962s (SENT)
I20260709 12:24:21.944188 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.938969 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:21.933806 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.832801 28291 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:21.828562 28084 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:24:21.958254 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:21.954263 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:21.915500 26194 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:21.937844 28165 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.856714 28075 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:21.983022 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.984057 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:21.988183 26054 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:21.861352 28254 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 12:24:21.859938 28193 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:24:21.989914 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:21.880334 28263 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.938432 28196 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.992787 26791 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:21.968899 28119 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.981707 27953 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:21.980291 28270 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:21.919320 28266 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.935807 27878 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:21.982129 28085 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.996874 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:22.014616 27878 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.014829 28266 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:21.997323 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.946703 28282 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.016134 28282 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.990674 28075 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:21.997922 28291 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.021010 28291 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:21.998704 26168 connection.cc:256] RPC call timeout handler was delayed by -1.8675799940001525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
I20260709 12:24:22.003468 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:22.027220 26054 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:22.005178 28278 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.997483 28196 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.011317 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:21.992146 28151 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:22.039225 26785 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:22.039716 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.041019 26784 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:22.041997 28119 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:22.028404 26168 connection.cc:256] RPC call timeout handler was delayed by -1.8622245120000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 12:24:22.029241 26047 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:22.029014 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 183147us
I20260709 12:24:22.031072 26791 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.982326 28008 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:24:22.052946 26122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:24:22.053738 26122 raft_consensus.cc:606] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:22.054116 26122 raft_consensus.cc:614] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:22.051146 28136 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:24:22.057657 28084 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.058179 28084 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.060340 26783 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:22.060731 26784 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEARNER]: Advancing to term 4
W20260709 12:24:22.053013 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.067994 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.069660 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.069947 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.070595 26126 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:22.070735 26120 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:22.030699 28278 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.074641 26791 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:22.074786 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.036057 28151 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:22.083305 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.086962 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:22.087724 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.090565 26127 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:22.093375 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.094152 26127 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:22.096647 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.097218 26127 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.097777 26120 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:22.098599 28275 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:22.100198 28275 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:22.100579 28275 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:22.054888 28270 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:22.105383 26203 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:22.070713 28136 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.109887 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.114637 26127 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.081955 28278 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.112664 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.121886 26122 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:22.109864 28267 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.122901 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.123087 28267 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.124136 28075 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.104949 27931 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.123574 26783 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:22.127272 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.120468 28136 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.133589 28267 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:22.114961 28127 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:24:22.125084 28084 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.121289 28266 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:24:22.140295 28151 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.109357 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 417) took 1789 ms. Trace:
I20260709 12:24:22.120198 28270 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.234027 26128 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:22.164256 26789 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:22.101547 28275 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.184640 26127 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.177093 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:22.196933 26121 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:22.238201 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61208.3us
I20260709 12:24:22.197970 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.239591 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.239634 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.241088 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.205200 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.209053 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:22.161461 28263 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.157904 26784 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:22.244458 26124 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.270437 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.271507 26124 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:22.253456 26123 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:22.254282 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 347299us
I20260709 12:24:22.243609 26199 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:22.267370 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:22.274485 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.275528 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.253077 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.276626 26789 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:22.267666 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.278942 26127 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:22.268460 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:22.279807 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.275693 28291 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.160697 28008 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.092s
W20260709 12:24:22.281719 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.281987 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.179558 28075 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:22.280138 26121 raft_consensus.cc:2376] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5. Re-sending same reply.
I20260709 12:24:22.236595 28164 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:22.156167 28165 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.268102 28168 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.267192 28151 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.162508 28278 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.157541 28196 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.299145 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.307621 26121 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:22.280576 28152 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.170941 27931 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.278218 28087 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.266711 28269 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:22.339810 26122 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.340060 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.297360 28275 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.292827 28291 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.295645 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:22.309365 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 336040us
I20260709 12:24:22.296502 28164 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:22.349299 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.349583 27931 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.350121 27931 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.350425 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.349797 28164 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.350641 27931 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.351004 26124 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.346967 27953 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:22.351881 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.358865 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.359623 26124 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.352497 27953 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.353145 28087 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:22.340152 28269 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.334982 28254 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.354305 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.356822 26129 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.348592 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.366767 26128 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.367142 26127 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:22.367997 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.352604 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:22.361974 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 271377us
I20260709 12:24:22.369084 26123 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:22.369513 26121 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:22.296299 28075 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.363806 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.210433 26782 rpcz_store.cc:276] 0709 12:24:20.316587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.836529 (+519942us) service_pool.cc:224] Handling call
0709 12:24:20.840996 (+  4467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.841844 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.081827 (+1239983us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b5400355a50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:22.106373 (+ 24546us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9452416}
I20260709 12:24:22.372637 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.374020 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:22.367686 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:22.380921 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.381757 26124 raft_consensus.cc:2376] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6. Re-sending same reply.
I20260709 12:24:22.386188 26128 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:22.308038 28168 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.369733 28269 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.372520 28152 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:24:22.397874 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.399066 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.404548 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.405968 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.406510 26128 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:22.407248 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.407721 26128 raft_consensus.cc:2376] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4. Re-sending same reply.
I20260709 12:24:22.408517 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.381697 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.416121 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.362602 28291 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.419471 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.424276 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.426409 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.428259 26123 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:22.376263 28254 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.423849 26783 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:22.431061 28136 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.429145 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.431797 26128 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:22.437896 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 406304us
I20260709 12:24:22.474766 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.476053 26123 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:22.484754 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.486968 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.489264 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:22.494078 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 457113us
I20260709 12:24:22.439674 28264 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:22.506362 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.507382 26789 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:22.500211 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.496176 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.433272 27798 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:24:22.535660 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:22.535264 26123 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:22.536365 26048 raft_consensus.cc:606] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.538853 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:22.460999 28270 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:22.539678 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 456958us
I20260709 12:24:22.540740 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:22.548381 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 199817us
W20260709 12:24:22.560400 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52311.3us
I20260709 12:24:22.490128 27931 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.562489 28318 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:22.562842 28306 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:22.562996 28264 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:22.563726 28168 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:22.563495 28193 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:22.554863 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.560068 28152 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.503269 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.565372 28075 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.544464 26782 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6.
I20260709 12:24:22.497174 28298 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.572060 26786 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:22.592907 26127 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:22.587168 28291 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.587595 27953 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:22.590865 28312 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:22.598703 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.621269 26125 raft_consensus.cc:2368] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3.
I20260709 12:24:22.544272 28196 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.546996 28165 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.561368 28278 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.585345 28318 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:22.593418 28254 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.613544 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.618496 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.625293 28298 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.625733 26791 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:22.617403 28324 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:22.627084 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.639320 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.632205 26121 raft_consensus.cc:2376] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6. Re-sending same reply.
W20260709 12:24:22.643060 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77902.8us
I20260709 12:24:22.603564 28008 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:24:22.615413 28261 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:22.643328 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.635895 28318 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:22.639195 28324 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:22.621420 28136 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:24:22.656440 28324 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.651499 26121 raft_consensus.cc:2376] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5. Re-sending same reply.
W20260709 12:24:22.663501 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.638533 26783 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:22.667522 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.608263 28307 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:24:22.668421 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:22.697877 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116274us
W20260709 12:24:22.651255 28282 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.629s	user 0.003s	sys 0.008s
W20260709 12:24:22.665299 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.703433 26788 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.671273 28303 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:22.646809 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.676843 28264 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.680081 28298 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.681766 28324 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.686251 28318 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.715574 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.650789 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:22.693646 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.694912 26128 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:22.695196 26124 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:22.614135 28152 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.698719 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:22.644366 26784 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:22.625876 28312 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:24:22.722600 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:22.722862 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.710297 26126 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:22.723449 26788 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.724352 26126 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:22.725796 28326 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.725872 28075 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:24:22.726410 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:22.727000 28328 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:22.720253 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.724597 26125 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:22.706805 28266 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:22.712666 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.732436 26124 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:24:22.732353 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81635us
I20260709 12:24:22.665872 28261 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:22.745217 28317 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:24:22.705395 28269 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.711438 28168 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:22.727241 26784 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:22.748006 28269 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.727465 28328 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:22.748932 28328 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.749089 28318 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:22.751314 28152 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.721451 26782 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:22.753038 28136 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:22.753460 28136 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.753800 28136 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.755100 28324 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.755199 26788 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:22.756037 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:22.715342 28291 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:24:22.757826 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:22.759625 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:22.760396 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 201098us
W20260709 12:24:22.762686 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 200794us
I20260709 12:24:22.767540 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:22.762689 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:22.706039 28282 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.763634 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.799129 26126 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:22.768610 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:22.777444 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.768678 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:22.727008 28075 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:22.798036 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.704922 28327 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:24:22.800385 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.768154 28324 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:22.806259 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43647.6us
I20260709 12:24:22.807228 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:22.766592 27931 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:22.812139 26120 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 12:24:22.795720 28152 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:22.768365 28278 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.768837 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:22.817631 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.817943 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.818392 26125 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:22.797610 28151 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:22.767251 28263 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:24:22.800295 28318 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:22.825722 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.012s	sys 0.012s
W20260709 12:24:22.826133 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.012s	sys 0.012s
W20260709 12:24:22.826447 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 872356us
I20260709 12:24:22.827374 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:22.829382 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.830871 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.831498 26122 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:22.803378 26786 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:22.816171 26126 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:22.807173 28312 raft_consensus.cc:2764] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:24:22.832720 28297 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.761s	user 0.000s	sys 0.004s
I20260709 12:24:22.822966 27931 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:22.822484 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.852727 27798 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:22.811472 28310 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:24:22.811120 28196 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:22.872179 28261 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:22.872470 28196 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.872969 28165 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:22.873199 28303 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:22.875167 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:22.841462 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:22.815750 28152 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.813850 28084 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:22.804590 28075 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:22.817754 28264 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:22.825632 28318 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.821640 28151 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.836256 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:22.859313 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.842082 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1089) took 1517 ms. Trace:
I20260709 12:24:22.807983 28337 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 12:24:22.864053 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57986.2us
I20260709 12:24:22.870957 27798 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.822342 28278 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:22.810575 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.853657 27953 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:22.855163 28305 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:22.891309 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1098) took 1123 ms. Trace:
I20260709 12:24:22.893725 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.910965 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:22.911247 26788 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:22.911804 26203 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:22.898926 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:22.912186 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:22.900202 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:22.872918 28196 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.927690 28164 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.928851 28282 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.898753 28084 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:22.930186 28084 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.898146 28151 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.932199 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.892416 28261 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:22.935485 28261 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:22.895506 28335 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:24:22.881436 28136 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.897746 26045 rpcz_store.cc:276] 0709 12:24:21.316074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.350940 (+ 34866us) service_pool.cc:224] Handling call
0709 12:24:21.351485 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.360756 (+  9271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:21.368551 (+  7795us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.802154 (+433603us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b18003d3d80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:22.063127 (+260973us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b18003d3d80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:24:22.468591 (+405464us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:22.562269 (+ 93678us) log.cc:844] Serialized 26 byte log entry
0709 12:24:22.562505 (+   236us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:22.784827 (+222322us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.805030 (+ 20203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.811998 (+  6968us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.812844 (+   846us) raft_consensus.cc:1685] finished
0709 12:24:22.816640 (+  3796us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.824012 (+  7372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35746,"prepare.queue_time_us":318335,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":2250,"raft.queue_time_us":18810,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":7741,"spinlock_wait_cycles":28381312,"thread_start_us":530215,"threads_started":2,"wal-append.queue_time_us":1203945}
I20260709 12:24:22.938895 28136 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:22.881707 28152 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.903256 28312 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:22.907444 26199 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.935995 28261 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.909687 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:22.895824 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 59688.3us
I20260709 12:24:22.946450 26200 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.947045 26020 leader_election.cc:400] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 2 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3.
I20260709 12:24:22.910614 26050 rpcz_store.cc:276] 0709 12:24:21.767859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.773228 (+  5369us) service_pool.cc:224] Handling call
0709 12:24:21.773553 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.773896 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.773925 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.774390 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.874741 (+100351us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:21.913147 (+ 38406us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.914412 (+  1265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.914497 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.737398 (+822901us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:22.876851 (+139453us) raft_consensus.cc:1685] finished
0709 12:24:22.891084 (+ 14233us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.891246 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":962,"prepare.queue_time_us":33811,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":18108,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":15967,"spinlock_wait_cycles":12736768,"thread_start_us":99921,"threads_started":1,"wal-append.queue_time_us":222568}
I20260709 12:24:22.909324 28168 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:22.949117 28310 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:22.948840 28168 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.949651 28310 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:22.951475 28168 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:22.918176 28339 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.953543 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:22.918764 28338 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:22.920181 28165 raft_consensus.cc:1064] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:24:22.956341 28165 raft_consensus.cc:1068] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:22.902534 27931 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:22.921949 27798 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:22.872888 28270 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 12:24:22.975700 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72811.6us
I20260709 12:24:22.922950 28305 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:22.925122 26018 leader_election.cc:400] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6.
I20260709 12:24:22.985437 26018 leader_election.cc:403] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:22.986553 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71667.7us
I20260709 12:24:22.912611 26202 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:22.895051 28318 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.894207 28242 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.960054 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:22.922359 28278 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:23.013820 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 97300.5us
W20260709 12:24:23.013967 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 65786.1us
I20260709 12:24:23.015842 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.977929 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:22.988586 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 154320us
W20260709 12:24:23.025753 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47952.6us
I20260709 12:24:23.027496 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:22.999591 26200 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
W20260709 12:24:23.004997 26757 leader_election.cc:336] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 2.076s (SENT)
I20260709 12:24:23.028791 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.032593 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:23.033728 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:23.035379 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62031.5us
I20260709 12:24:23.039381 26094 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 2 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3.
I20260709 12:24:23.002396 28312 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:23.049117 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:22.986105 26194 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:23.019279 28310 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.030704 28325 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.028942 28339 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.047133 28326 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.059145 28349 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:23.053603 28269 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.059597 27931 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.972879 28328 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.026979 28338 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:24:23.053025 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.052172 28168 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.004204 28261 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.026734 27953 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:24:23.074697 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.077549 26202 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:23.078238 26791 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:23.056000 28266 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.055634 28263 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:23.080796 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104183us
I20260709 12:24:23.096769 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.099705 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.099913 26202 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:23.029172 28307 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:24:23.056622 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:23.083403 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:23.104794 26203 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:23.031601 28193 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:23.107340 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:23.108312 26196 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:23.084786 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 69035.5us
I20260709 12:24:23.025607 28242 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:23.089479 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62859.9us
I20260709 12:24:23.089633 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.072197 28310 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:23.118666 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.091388 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:23.120342 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:23.093503 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.019905 28075 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.097188 28278 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.095422 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.082548 28339 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:23.186185 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.140347 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2270) took 1042 ms. Trace:
W20260709 12:24:23.151579 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.169770 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.190129 26120 rpcz_store.cc:276] 0709 12:24:22.103677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.107604 (+  3927us) service_pool.cc:224] Handling call
0709 12:24:22.108047 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.812582 (+704535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.135887 (+323305us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:23.136207 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1618604288}
I20260709 12:24:23.091389 28168 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.113354 28261 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.113878 28316 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:23.114167 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.199043 26201 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:23.199332 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85260us
I20260709 12:24:23.115806 28242 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:23.117412 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:23.122927 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.070237 28152 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.110864 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:23.101312 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.201669 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.186133 28344 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:23.118615 26782 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:23.212944 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 111668us
I20260709 12:24:23.214951 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:23.207440 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87234.8us
I20260709 12:24:23.203779 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.217000 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.209708 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.130911 28267 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.152046 28298 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.216835 26202 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:23.132089 28270 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.194399 28168 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:23.226230 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 115507us
I20260709 12:24:23.228386 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.198367 28316 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:23.225973 28087 raft_consensus.cc:1064] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:23.195230 28291 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.219530 28164 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.218294 26198 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:23.128705 26785 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.251335 26200 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:23.105068 28266 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:23.255995 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.147693 28282 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.272702 28282 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:23.273860 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 224853us
I20260709 12:24:23.274709 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.275265 28282 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.275552 27931 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.084784 28305 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.222100 28264 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:23.224875 28344 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:23.211217 28075 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:23.263496 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46635.4us
I20260709 12:24:23.293973 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.257138 28338 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:23.220500 28268 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:24:23.265888 28303 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.225270 28349 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.266124 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:23.219285 28350 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.326964 26201 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.241705 28261 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.281100 28264 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.341616 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.342499 26200 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:23.275976 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.246781 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:23.285477 26123 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:23.148491 28027 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:23.250430 28242 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:24:23.269620 28336 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.315969 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:23.250541 28298 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.240779 28275 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.253823 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:23.364413 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 110698us
I20260709 12:24:23.252472 28196 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.254871 28270 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.342612 26125 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:23.355219 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 79346.6us
I20260709 12:24:23.373615 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.340757 28303 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:23.357715 26120 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.358106 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:23.325824 28324 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.382335 28193 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:23.383582 28324 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:23.383975 28193 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.384408 28324 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:23.384577 28193 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.386139 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.385676 28324 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.387006 28193 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:23.387665 28193 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.389092 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.388353 28193 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:23.378243 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 118948us
I20260709 12:24:23.366622 26201 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:23.368762 26203 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:23.343906 28337 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.345112 28264 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:23.277320 28267 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:23.378393 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.354096 28027 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:23.377789 26791 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:23.382423 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.365311 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.397374 28328 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.400395 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.361922 28355 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.368100 28307 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.403518 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.407583 26203 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:24:23.407639 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.473711 26203 pending_rounds.cc:85] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 6
I20260709 12:24:23.474150 26203 pending_rounds.cc:107] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:24:23.474488 26203 raft_consensus.cc:2889] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:23.360728 26194 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
W20260709 12:24:23.415870 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50648.1us
I20260709 12:24:23.476135 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:23.387770 28346 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.343572 28261 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.360406 28357 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.462777 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.391700 28269 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.327869 28338 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.398550 28335 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.398476 27931 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.343961 28316 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.487228 26791 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:23.344640 28350 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.303119 28339 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.398036 28164 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:24:23.492277 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:23.493870 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:23.518038 26123 raft_consensus.cc:2376] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3. Re-sending same reply.
W20260709 12:24:23.494451 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91112.5us
I20260709 12:24:23.495532 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:23.483027 26120 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:23.499593 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:23.405102 28275 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:24:23.462759 27748 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:24:23.520648 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:23.520789 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:23.521387 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:23.476950 28355 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.488152 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.470768 28268 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:24:23.582266 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:23.533597 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:23.655340 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 121886us
W20260709 12:24:23.649160 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 241286us
I20260709 12:24:23.657934 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.532421 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:23.659356 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 127045us
I20260709 12:24:23.660090 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.504187 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:23.580016 28151 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.000s
I20260709 12:24:23.665621 26120 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:23.621238 28316 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.666803 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:23.666047 26196 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 12:24:23.689127 26196 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:23.510469 28326 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:23.645215 28242 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.653050 28355 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.579924 28370 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:23.515295 28127 consensus_queue.cc:1175] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:23.508205 27931 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.665727 27798 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:23.681499 28269 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:23.657616 26787 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:23.489576 28307 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:23.704363 28084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.030s	sys 0.053s
I20260709 12:24:23.693785 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 472) took 1399 ms. Trace:
I20260709 12:24:23.681833 28328 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:24:23.588829 28027 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.714134 28378 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:23.703176 28165 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:23.692584 28151 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.692804 28326 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.752092 26789 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:23.706326 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:23.717084 28242 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.710028 28275 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:23.703943 28377 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.748035 26784 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 12:24:23.751294 28084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.030s	sys 0.054s
I20260709 12:24:23.755978 28378 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:23.706822 28266 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.732932 28370 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.756460 28362 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:23.767380 28270 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.778332 28027 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.723100 28196 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:23.800434 28375 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:23.797017 28362 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:23.801494 28369 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:24:23.801658 28362 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.802237 28378 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.802959 28362 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:23.803099 28359 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
I20260709 12:24:23.803409 28378 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:23.778154 28275 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:23.781178 28365 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:23.762511 28298 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.782251 28193 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.687120 28303 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:23.771226 28326 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.799571 26128 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:23.801091 28375 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.716810 27748 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:23.792842 28368 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 12:24:23.768883 28357 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:23.783713 28377 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.834509 28365 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:23.836561 28363 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.845024 28151 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:23.846268 28370 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:23.850047 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 449695us
I20260709 12:24:23.766372 28366 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:24:23.804536 28266 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:23.767499 28165 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:23.849718 28361 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
W20260709 12:24:23.866456 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 199752us
I20260709 12:24:23.755034 26786 rpcz_store.cc:276] 0709 12:24:22.288055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.654884 (+366829us) service_pool.cc:224] Handling call
0709 12:24:22.655265 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.862050 (+206785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.644133 (+782083us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:23.686981 (+ 42848us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246065408}
W20260709 12:24:23.869091 26095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.003s	sys 0.018s
W20260709 12:24:23.901662 26095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.003s	sys 0.018s
I20260709 12:24:23.895704 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.865398 28382 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:23.877765 28266 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:23.906284 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 234024us
I20260709 12:24:23.880206 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.909703 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.868682 28357 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:23.868566 28269 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:23.944226 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.947568 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.950321 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:23.865525 28355 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:23.962019 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82126.9us
I20260709 12:24:23.963307 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:23.963359 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.964308 26048 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:23.908695 28350 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.966604 26048 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:23.968621 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:23.914695 28382 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:23.973456 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.905181 28338 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:23.947666 28151 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.915027 27748 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:23.971946 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.970283 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.972929 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.951833 28357 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:23.970708 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:23.953554 28328 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:23.961643 28136 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:23.962745 26785 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.001s
I20260709 12:24:23.964964 28164 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.899725 28275 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:23.956679 28310 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.012s	sys 0.015s
I20260709 12:24:23.982682 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.983081 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:23.989126 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.971376 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.990569 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:23.967099 28291 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:23.965363 28368 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:24:23.976035 26051 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:24.035359 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.981453 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:23.980407 27748 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:23.976775 28254 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.893666 28370 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.002796 26047 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:23.989707 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:24.039304 26053 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:24.013852 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:24.012580 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:24.040658 26120 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:24.005211 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:23.959513 28264 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:23.992110 28136 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.062000 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.062636 28136 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.062922 26194 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:24.031690 28266 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.894400 28363 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.066006 28363 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:24.032020 28307 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:24.036435 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:23.997067 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:24.067297 26054 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:24.067351 28350 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:23.984998 28328 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.997566 26785 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:24.068173 28328 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.068501 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 480) took 1043 ms. Trace:
I20260709 12:24:24.068876 26785 rpcz_store.cc:276] 0709 12:24:23.025485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.028829 (+  3344us) service_pool.cc:224] Handling call
0709 12:24:24.068436 (+1039607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:24.069818 28264 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:24.070015 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:24.061093 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:24.070636 28165 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:24.068028 28350 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.038727 28377 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:24.070959 28382 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:23.997224 28318 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.972s	user 0.010s	sys 0.002s
I20260709 12:24:24.072851 28350 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:24.037967 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:24.020486 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:24.073781 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:24.074034 26046 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:24.074236 28298 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:24.075924 28355 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:24.044962 28376 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:24.044852 28385 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.045514 28366 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:24.045640 27931 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:24.046995 28375 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:24.047983 26045 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:24.048071 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:23.999660 28282 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:24.051465 28312 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.993s	user 0.010s	sys 0.000s
W20260709 12:24:24.053381 28075 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.748s	user 0.003s	sys 0.008s
I20260709 12:24:24.037585 28164 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.056221 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:24.038228 28168 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.774s	user 0.003s	sys 0.009s
I20260709 12:24:24.057514 26195 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:24:24.058218 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.148286 26195 pending_rounds.cc:85] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 5
I20260709 12:24:24.148363 26201 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:24.148839 26195 pending_rounds.cc:107] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:24.149276 26195 raft_consensus.cc:2889] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:24.038933 28254 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.061038 28370 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:23.999589 28151 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:24:24.069643 28325 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:24.072559 28318 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:24.077230 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.154712 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.078349 26052 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:24.035845 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:24.156668 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:24.079680 26788 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:24.042022 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 2.
I20260709 12:24:24.062536 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:24.021433 28371 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.030900 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:24.076434 28196 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:24.168470 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 201005us
I20260709 12:24:24.138388 28312 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6.
I20260709 12:24:24.170166 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:24.170718 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.171020 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.172581 26196 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:24.173849 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.176913 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:24.147333 28164 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.167853 28196 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:24.179417 28196 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.179603 26197 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:24.180950 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.151719 28254 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.184098 28380 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:24.153560 28168 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.085497 28269 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.188213 26194 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:24.122391 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.035389 28310 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.013s	sys 0.019s
W20260709 12:24:24.126940 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.113049 26198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48518
I20260709 12:24:24.151583 28378 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.222129 26198 raft_consensus.cc:606] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Received request to transfer leadership
I20260709 12:24:24.086220 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:24.051550 28326 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.222733 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.133888 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:24.224331 26120 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:24.139086 26123 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:24.132493 28282 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.129218 28366 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:24.240440 28366 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:24.242151 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.159312 26791 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:24.124034 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46885.2us
I20260709 12:24:24.243116 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:24.244822 28350 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:24.246227 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:24.247464 28084 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:24.248529 28084 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:24.249071 28136 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.249109 28084 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:24.250936 28084 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:23.998272 28365 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.175655 26786 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:24.229535 28386 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:24.234155 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:24.234447 28316 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.001s
I20260709 12:24:24.234388 28357 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.235387 28391 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.236709 28395 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:24:24.221834 28380 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:24.221625 28310 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:24.169831 28370 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:24.260543 28282 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.331758 28370 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:24.331504 28380 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.331534 28310 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.332748 26785 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:24.325793 28350 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:24.340719 26784 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:24.303759 28378 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.421253 28312 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.446926 28350 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:24.169600 28371 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.350445 28328 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.333206 28391 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.601604 28370 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.553793 28350 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.544776 28312 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.690596 28391 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:24.710083 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 623895us
W20260709 12:24:24.725227 28196 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.000s
W20260709 12:24:24.731011 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.803943 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.804414 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.735029 28254 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.000s
I20260709 12:24:24.702740 28312 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:24.737826 27798 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.991s	user 0.010s	sys 0.002s
W20260709 12:24:24.765309 28385 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.678s	user 0.005s	sys 0.005s
W20260709 12:24:24.769119 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 738256us
I20260709 12:24:24.769933 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.770920 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:24.771643 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 537522us
I20260709 12:24:24.776297 26791 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:24.911270 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:24.779897 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1596) took 1285 ms. Trace:
I20260709 12:24:24.780375 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.780885 28164 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:24.791262 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.916455 28385 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:24.916934 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.914826 26124 rpcz_store.cc:276] 0709 12:24:23.494036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.494709 (+   673us) service_pool.cc:224] Handling call
0709 12:24:23.494996 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.967651 (+472655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.967681 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.969640 (+  1959us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.147869 (+178229us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:24.147905 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.147971 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.779630 (+631659us) raft_consensus.cc:1685] finished
0709 12:24:24.779656 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.779811 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14502,"raft.queue_time_us":152908,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":12786,"thread_start_us":612247,"threads_started":2,"wal-append.queue_time_us":791856}
I20260709 12:24:24.918828 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:24.779897 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 533703us
I20260709 12:24:24.920704 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:24.922716 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.923705 26196 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:24.910905 26198 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:24.925355 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:24.926093 26198 raft_consensus.cc:2376] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6. Re-sending same reply.
I20260709 12:24:24.927259 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.929139 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.930698 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.931456 26196 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:24.932561 26198 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:24.933658 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.935362 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.938467 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 447) took 1025 ms. Trace:
I20260709 12:24:24.938920 26791 rpcz_store.cc:276] 0709 12:24:23.913384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.937645 (+1024261us) service_pool.cc:224] Handling call
0709 12:24:24.937948 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.938144 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.938167 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.938210 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:24.938243 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.938294 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.938411 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:24.940568 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:24.943408 26791 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:24.947651 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.950703 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.951501 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:24.955288 26791 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:24.960328 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:24.911827 28168 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:24.917011 27798 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.932384 28312 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.941650 28282 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 12:24:24.972509 26196 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:24.937470 28405 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.943837 28407 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.974493 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:24.845827 28196 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.975153 26791 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:24.975220 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62540.4us
W20260709 12:24:24.973738 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 425483us
I20260709 12:24:25.000635 26791 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:25.001724 26196 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:25.005947 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.990370 26019 connection.cc:256] RPC call timeout handler was delayed by -1.5269855559999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 12:24:24.909194 28254 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:24.943985 28391 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:24:25.008276 26019 connection.cc:256] RPC call timeout handler was delayed by -1.3777673069998855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:24:25.008809 26019 connection.cc:256] RPC call timeout handler was delayed by -1.1147855559997879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 12:24:25.008603 28391 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:25.009133 26019 connection.cc:256] RPC call timeout handler was delayed by -1.1053999999999178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260709 12:24:25.010438 28254 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:24.974998 28380 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:24.964296 28385 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:24:25.018242 26791 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:24.949198 28386 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:24:25.011215 28391 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:25.022673 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 101968us
W20260709 12:24:25.022589 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 110864us
I20260709 12:24:24.997133 28407 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.028512 26120 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:25.033281 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.033496 26126 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.037639 26791 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:25.038659 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.039237 26791 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:25.019812 28380 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.046968 26195 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:25.014387 28164 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:25.016142 27798 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:24.995199 28312 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:24.996471 28282 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.039997 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.035737 28254 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:25.060510 26126 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
W20260709 12:24:25.061331 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 119792us
I20260709 12:24:25.028992 28416 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:24.999323 28405 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.042068 28407 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.044066 28380 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.043504 28370 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:25.093773 28269 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:25.053143 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 479) took 2180 ms. Trace:
I20260709 12:24:25.049611 28415 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:24:25.101773 28278 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.077239 27798 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.072217 28254 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:25.085229 28168 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:25.135360 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 127468us
I20260709 12:24:25.073491 28164 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.108632 28416 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.123672 28415 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:25.105345 28370 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.108319 28269 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.168545 28196 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.169016 28411 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:24:25.167127 28423 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.164575 28415 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:25.169844 28412 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:24:25.122400 28278 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:25.157316 28419 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.108928 26782 rpcz_store.cc:276] 0709 12:24:22.869869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.881197 (+ 11328us) service_pool.cc:224] Handling call
0709 12:24:22.913504 (+ 32307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.219200 (+305696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.034656 (+1815456us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400565754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:25.050205 (+ 15549us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417766784}
I20260709 12:24:25.166854 28386 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:25.132014 28168 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.208488 28282 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.188144 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51886.3us
I20260709 12:24:25.153787 28416 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.193708 28164 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.193238 28386 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.200139 28278 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.185082 28423 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.201423 28370 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.203017 28269 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:25.205422 28413 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:24:25.205674 28312 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.206300 28405 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:25.207374 28350 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 12:24:25.166325 28254 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.204069 27798 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.254851 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1606) took 1246 ms. Trace:
I20260709 12:24:25.255473 26120 rpcz_store.cc:276] 0709 12:24:24.008090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.785250 (+777160us) service_pool.cc:224] Handling call
0709 12:24:24.785637 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.035963 (+250326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.254496 (+218533us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:25.254763 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557965568}
I20260709 12:24:25.191730 28419 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.258153 28423 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.258978 28419 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.259080 28373 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.544s	user 0.007s	sys 0.026s
I20260709 12:24:25.239462 28417 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:25.260108 28417 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.261349 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1092) took 1987 ms. Trace:
I20260709 12:24:25.261988 28326 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.262497 28414 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:24:25.263223 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1594) took 1769 ms. Trace:
I20260709 12:24:25.261889 26200 rpcz_store.cc:276] 0709 12:24:23.273762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.345651 (+ 71889us) service_pool.cc:224] Handling call
0709 12:24:23.345966 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:23.358955 (+ 12989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.358983 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:23.364103 (+  5120us) log.cc:844] Serialized 52 byte log entry
0709 12:24:23.483418 (+119315us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:23.483486 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.483615 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.258323 (+1774708us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:25.260982 (+  2659us) raft_consensus.cc:1685] finished
0709 12:24:25.261010 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.261285 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8416,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":9155,"spinlock_wait_cycles":37812096,"thread_start_us":78191,"threads_started":1,"wal-append.queue_time_us":268170}
I20260709 12:24:25.263568 26128 rpcz_store.cc:276] 0709 12:24:23.493525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.494508 (+   983us) service_pool.cc:224] Handling call
0709 12:24:23.494975 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.849084 (+354109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.262963 (+1413879us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54001f90d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:25.263155 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699839232}
I20260709 12:24:25.267012 26200 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:25.267304 28386 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.220037 28385 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:24:25.268848 26200 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:25.270139 28416 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.270545 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 444) took 1358 ms. Trace:
I20260709 12:24:25.247617 28350 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.270957 26788 rpcz_store.cc:276] 0709 12:24:23.911872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.158454 (+246582us) service_pool.cc:224] Handling call
0709 12:24:24.158913 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:24.339706 (+180793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.339743 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:24.779374 (+439631us) log.cc:844] Serialized 52 byte log entry
0709 12:24:24.957224 (+177850us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:25.060127 (+102903us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.060396 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.060474 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.261939 (+201465us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:25.263991 (+  2052us) raft_consensus.cc:1685] finished
0709 12:24:25.270328 (+  6337us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.270484 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1503,"prepare.queue_time_us":145780,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":6918,"raft.queue_time_us":81696,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":18898,"spinlock_wait_cycles":6937984,"thread_start_us":419927,"threads_started":3,"wal-append.queue_time_us":248443}
I20260709 12:24:25.273021 28328 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.273662 28350 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.231158 28415 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.279449 28327 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:25.279937 28327 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:25.281318 28327 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.282888 28327 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.229370 28282 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:24:25.285444 28415 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.286921 28168 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.296057 28419 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:25.296686 28419 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:25.298054 28419 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.302217 28382 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.305665 28380 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:25.306649 28432 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
I20260709 12:24:25.307509 26199 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:25.328879 28407 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.336061 28380 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.338158 28318 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:25.338428 28364 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.546s	user 0.041s	sys 0.033s
I20260709 12:24:25.347877 28417 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.387931 26019 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.575s (SENT)
I20260709 12:24:25.389276 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:25.172094 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 172245us
I20260709 12:24:25.396028 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:25.398309 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.398979 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.405544 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:25.415205 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2304) took 2298 ms. Trace:
I20260709 12:24:25.415838 26121 rpcz_store.cc:276] 0709 12:24:23.116383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.117156 (+   773us) service_pool.cc:224] Handling call
0709 12:24:23.117514 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.117902 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.117934 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.130892 (+ 12958us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.306259 (+175367us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:23.306318 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.306425 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.337348 (+2030923us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:25.413915 (+ 76567us) raft_consensus.cc:1685] finished
0709 12:24:25.413986 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.414907 (+   921us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2586,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":65554,"spinlock_wait_cycles":9752448,"thread_start_us":75834,"threads_started":1,"wal-append.queue_time_us":557982}
I20260709 12:24:25.420692 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:25.422170 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:25.423348 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:25.424286 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.429965 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 243) took 1455 ms. Trace:
W20260709 12:24:25.437799 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.440543 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 230621us
I20260709 12:24:25.446125 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1592) took 1954 ms. Trace:
I20260709 12:24:25.447623 28437 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:25.449105 28437 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.453313 28440 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:24:25.456827 28416 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.458005 28416 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:25.458498 28416 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.458832 28416 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.461889 28434 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.462999 28443 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.463866 28443 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.458040 28407 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.470276 28417 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:25.477263 28417 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:25.477787 28417 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.431357 26052 rpcz_store.cc:276] 0709 12:24:23.974181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.982417 (+  8236us) service_pool.cc:224] Handling call
0709 12:24:25.429375 (+1446958us) spinlock_profiling.cc:243] Waited 76.2 ms on lock 0x7b54001b3fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:25.429662 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175325056}
I20260709 12:24:25.479549 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.468914 28377 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:25.480418 26019 leader_election.cc:336] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.753s (SENT)
I20260709 12:24:25.481158 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:25.480460 28377 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.482434 28377 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:25.467964 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.446700 26125 rpcz_store.cc:276] 0709 12:24:23.491670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.491850 (+   180us) service_pool.cc:224] Handling call
0709 12:24:23.492299 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.505548 (+ 13249us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:23.513932 (+  8384us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.998889 (+1484957us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b180045afc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:24:25.237373 (+238484us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.238764 (+  1391us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.238967 (+   203us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:25.239016 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.239119 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.306345 (+ 67226us) spinlock_profiling.cc:243] Waited 9.18 ms on lock 0x7b240028f1a8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:24:25.445380 (+139035us) raft_consensus.cc:1685] finished
0709 12:24:25.445434 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.445871 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8778,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":22104,"raft.queue_time_us":31439,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":72019,"spinlock_wait_cycles":27607680,"thread_start_us":232415,"threads_started":2,"wal-append.queue_time_us":1680412}
I20260709 12:24:25.473835 26197 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:25.494226 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:25.495162 26197 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:25.496583 28440 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.497356 26200 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.500798 28415 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.501876 28437 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.507478 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.497437 28440 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.511561 28267 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:25.469662 28380 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:25.512682 28380 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.513244 28380 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.512152 28267 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:25.520362 28267 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:25.522065 27748 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.528520 27748 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.529599 28337 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:25.530113 28337 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.494278 28443 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:25.534165 28440 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:25.541460 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61122.6us
W20260709 12:24:25.542668 26019 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.827s (SENT)
I20260709 12:24:25.543278 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:25.546451 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:25.530716 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.525197 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.552767 26787 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:25.553534 28378 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.554317 28415 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:25.554471 28402 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.592s	user 0.000s	sys 0.004s
I20260709 12:24:25.544708 28371 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.560631 28419 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.564357 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:25.565138 26128 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.565352 26127 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.565505 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.566385 26124 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
W20260709 12:24:25.566576 26199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1489) took 1392 ms (client timeout 1828 ms). Trace:
W20260709 12:24:25.567098 26199 rpcz_store.cc:269] 0709 12:24:24.174064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.176599 (+  2535us) service_pool.cc:224] Handling call
0709 12:24:25.566204 (+1389605us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400396554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:25.566381 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597933952}
I20260709 12:24:25.568598 28382 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:25.569370 28382 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.569916 26120 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:25.568647 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.577461 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1111) took 1603 ms. Trace:
W20260709 12:24:25.578397 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 154114us
I20260709 12:24:25.577592 28305 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:25.583606 28305 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.591310 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.591789 28305 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.583492 28405 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.593062 26199 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:25.593290 28405 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.594619 26199 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:25.595664 26200 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:25.595829 28405 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.596413 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 385171us
I20260709 12:24:25.609647 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.610591 26122 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:25.613075 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.615331 26197 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.617877 28440 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.618600 28440 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.619107 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 684) took 2217 ms. Trace:
I20260709 12:24:25.617906 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.578254 26049 rpcz_store.cc:276] 0709 12:24:23.973471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.973630 (+   159us) service_pool.cc:224] Handling call
0709 12:24:23.989832 (+ 16202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.990227 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.990248 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.990704 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.769122 (+778418us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:24.941425 (+172303us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.998146 (+ 56721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.998251 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.553319 (+555068us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:25.572909 (+ 19590us) raft_consensus.cc:1685] finished
0709 12:24:25.574447 (+  1538us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b1800386140. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:25.576917 (+  2470us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.577187 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1778,"prepare.queue_time_us":234461,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":7213,"raft.queue_time_us":106406,"raft.run_cpu_time_us":487,"raft.run_wall_time_us":23282,"spinlock_wait_cycles":31911040,"thread_start_us":368267,"threads_started":3,"wal-append.queue_time_us":818946}
W20260709 12:24:25.625432 26789 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 677) took 2243 ms (client timeout 1905 ms). Trace:
I20260709 12:24:25.626332 28389 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.625623 28440 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.627045 28389 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.630340 27931 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:25.626088 26789 rpcz_store.cc:269] 0709 12:24:23.381873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.382114 (+   241us) service_pool.cc:224] Handling call
0709 12:24:25.624923 (+2242809us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54003c5854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:25.625161 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774484224}
I20260709 12:24:25.634166 27931 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.636744 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:25.639349 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 96807.8us
I20260709 12:24:25.639976 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:25.638901 28152 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:25.641027 26124 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:25.640225 26019 leader_election.cc:336] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.828s (SENT)
I20260709 12:24:25.638983 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 487) took 1565 ms. Trace:
I20260709 12:24:25.638847 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.650686 26785 rpcz_store.cc:276] 0709 12:24:24.058759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.069735 (+ 10976us) service_pool.cc:224] Handling call
0709 12:24:25.623869 (+1554134us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54005d20d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:25.624165 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278112000}
I20260709 12:24:25.631979 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:25.619515 26784 rpcz_store.cc:276] 0709 12:24:23.401253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.862196 (+460943us) service_pool.cc:224] Handling call
0709 12:24:23.912654 (+ 50458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.495633 (+582979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.618596 (+1122963us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540057a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:25.618923 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906543360}
I20260709 12:24:25.638792 28268 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:25.661921 28310 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:25.662778 28268 consensus_queue.cc:1243] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:25.641217 28389 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:25.664644 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:25.640671 28261 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:25.640694 28136 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.670596 26122 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:25.670919 28382 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.672566 28415 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:25.673107 28415 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:25.674441 28415 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.672677 27931 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.655507 28440 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.685460 28298 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:25.685982 28298 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:25.679636 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:25.677718 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80448.8us
W20260709 12:24:25.698171 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.689358 28298 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.676175 28415 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.700498 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.702988 26124 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:24:25.704445 26124 pending_rounds.cc:85] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 5
I20260709 12:24:25.664462 28417 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:25.729372 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61021.7us
I20260709 12:24:25.704852 26124 pending_rounds.cc:107] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:25.735188 26124 raft_consensus.cc:2889] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:25.735800 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.734401 28416 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.740478 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1091) took 2473 ms. Trace:
I20260709 12:24:25.741181 28416 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.741452 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:25.742238 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.741920 28440 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.743942 28303 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:25.744768 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.745137 28450 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.744740 28303 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.747164 26784 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:25.747571 28349 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:25.752002 26122 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.744822 26121 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:25.760715 28275 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.762471 28312 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.763304 28312 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:25.763855 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:25.764564 26050 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.764595 28327 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:25.765867 28312 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.766546 28119 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.767520 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:25.767373 28119 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.768294 26052 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.768496 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.768757 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.769214 26128 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:25.769477 26045 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:25.769987 28119 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.771023 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:25.771716 26049 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.771713 28417 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.772894 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.765477 28327 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.766652 27953 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.773406 28354 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:25.782869 28354 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:25.746078 28324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.886s	user 0.010s	sys 0.032s
W20260709 12:24:25.790525 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 150349us
I20260709 12:24:25.747213 28270 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:25.791992 26127 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
W20260709 12:24:25.797253 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.800002 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.801072 28305 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.753540 28303 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:25.802441 26019 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.887s (SENT)
I20260709 12:24:25.803571 26198 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:25.803485 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.804979 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:25.805454 28451 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:25.745868 28450 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:25.789819 28324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.006s	user 0.011s	sys 0.033s
I20260709 12:24:25.809530 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:25.815215 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75556.8us
I20260709 12:24:25.740972 26202 rpcz_store.cc:276] 0709 12:24:23.241079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.255006 (+ 13927us) service_pool.cc:224] Handling call
0709 12:24:23.255497 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.255770 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:25.498570 (+2242800us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b180015d540. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b77d9d3c7 00007f5b743af9a7 00007f5b743bf0bf
0709 12:24:25.524693 (+ 26123us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.655213 (+130520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.664830 (+  9617us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:25.677229 (+ 12399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.700124 (+ 22895us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.714484 (+ 14360us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4832,"prepare.queue_time_us":56708,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":15380,"raft.queue_time_us":16279,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":19629,"spinlock_wait_cycles":50064256,"thread_start_us":184623,"threads_started":2,"wal-append.queue_time_us":173757}
I20260709 12:24:25.819670 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.821702 26046 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:25.821815 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.822575 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:25.822681 28305 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.822454 27953 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:25.824008 26019 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.946s (SENT)
I20260709 12:24:25.824506 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:25.825980 26019 leader_election.cc:336] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.012s (SENT)
W20260709 12:24:25.827280 26019 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.062s (SENT)
I20260709 12:24:25.827719 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:25.828624 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:25.829520 28312 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:25.828722 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:25.830704 28327 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.831084 26202 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:25.831712 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.831995 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.832166 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:25.832485 26051 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:25.832628 26202 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.833063 26199 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:25.833760 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1487) took 1660 ms. Trace:
I20260709 12:24:25.834158 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.834136 26203 rpcz_store.cc:276] 0709 12:24:24.173458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.999200 (+825742us) service_pool.cc:224] Handling call
0709 12:24:25.026252 (+ 27052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.439806 (+413554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.439890 (+    84us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.441423 (+  1533us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.484712 (+ 43289us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:25.584368 (+ 99656us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.597659 (+ 13291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.631382 (+ 33723us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.833398 (+202016us) raft_consensus.cc:1685] finished
0709 12:24:25.833428 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.833576 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81898,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":214447,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":5137,"thread_start_us":455384,"threads_started":2,"wal-append.queue_time_us":179742}
I20260709 12:24:25.835665 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:25.836633 28416 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:25.840863 28001 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.843067 27953 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.843631 28075 raft_consensus.cc:1064] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:25.844606 28386 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.843765 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:25.821009 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.846714 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.794555 28270 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.858587 26050 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
W20260709 12:24:25.860327 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.801240 28380 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.864823 28371 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:25.865661 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.869302 26051 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:25.866416 26203 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:25.851387 28440 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.865833 28075 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:25.869627 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.867760 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.869246 28371 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:25.854661 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.861003 28270 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.867069 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.887213 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.886030 28261 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:25.888501 28261 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.889889 26125 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:25.899792 26052 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:25.903007 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:25.872658 28001 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.906476 28261 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.907407 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:25.909227 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:25.909921 26789 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.912623 28312 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:25.914299 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.915692 26787 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:25.919775 26201 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.929189 26199 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.897323 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.921669 28440 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.939764 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.940117 28087 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.915731 28357 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:25.932204 28450 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:25.961680 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.965513 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:25.966089 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.967166 28001 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:25.967350 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:25.968864 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:25.969372 26785 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 3: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:25.969511 26053 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 2.
I20260709 12:24:25.970490 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.898330 28354 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:25.978892 26789 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:25.984967 28354 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.986996 26194 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:25.987777 26194 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:25.964838 28440 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:26.010300 28420 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:26.010941 28420 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:26.013342 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.015561 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:25.944252 28298 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:26.018018 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1475) took 1848 ms (client timeout 2062 ms). Trace:
I20260709 12:24:26.020195 26196 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:26.013053 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:26.017036 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1100) took 1802 ms. Trace:
I20260709 12:24:25.990695 26194 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:25.973259 28371 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:26.025887 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:26.043772 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:27.237360 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 251) took 1475 ms. Trace:
I20260709 12:24:27.237978 26049 rpcz_store.cc:276] 0709 12:24:25.762339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.770747 (+  8408us) service_pool.cc:224] Handling call
0709 12:24:27.237274 (+1466527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.134708 28312 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:27.239486 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.239413 28312 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:26.529511 26201 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.001s
W20260709 12:24:27.240281 26785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.005s	sys 0.014s
I20260709 12:24:26.134130 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:26.520196 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:27.241698 26195 rpcz_store.cc:269] 0709 12:24:24.169456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.169901 (+   445us) service_pool.cc:224] Handling call
0709 12:24:26.017689 (+1847788us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b5400378554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:26.017856 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906416512}
W20260709 12:24:26.672458 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 94476.4us
W20260709 12:24:26.344202 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.496455 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.253068 28298 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:26.529886 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:26.977460 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1603) took 2972 ms (client timeout 1623 ms). Trace:
W20260709 12:24:27.300061 26791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.010s	sys 0.005s
W20260709 12:24:27.215653 28469 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.512s	user 0.000s	sys 0.001s
W20260709 12:24:27.299997 26123 rpcz_store.cc:269] 0709 12:24:24.004744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.004893 (+   149us) service_pool.cc:224] Handling call
0709 12:24:26.963294 (+2958401us) spinlock_profiling.cc:243] Waited 97.3 ms on lock 0x7b54001d24d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:26.977211 (+ 13917us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223820160}
I20260709 12:24:26.865775 26129 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:26.458793 28336 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.291669 26785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.006s	sys 0.014s
I20260709 12:24:26.049129 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:27.256042 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.000s	sys 0.012s
W20260709 12:24:27.306100 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.001s	sys 0.012s
W20260709 12:24:26.601673 28270 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.002s
I20260709 12:24:27.256579 26047 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:26.836630 28357 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.421s
I20260709 12:24:26.024600 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:27.283811 28486 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:27.012617 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 351) took 1471 ms. Trace:
W20260709 12:24:27.250164 26050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.011s	sys 0.008s
I20260709 12:24:26.690541 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:27.312749 26050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.011s	sys 0.008s
I20260709 12:24:27.312961 26052 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:27.313619 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2341) took 1274 ms. Trace:
I20260709 12:24:27.313750 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1114) took 1432 ms. Trace:
I20260709 12:24:27.313995 26052 rpcz_store.cc:276] 0709 12:24:26.039417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.690216 (+650799us) service_pool.cc:224] Handling call
0709 12:24:27.313520 (+623304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.314224 26050 rpcz_store.cc:276] 0709 12:24:25.881806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.881967 (+   161us) service_pool.cc:224] Handling call
0709 12:24:25.917977 (+ 36010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.947082 (+ 29105us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:26.255609 (+308527us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.313331 (+1057722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.313389 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:27.313445 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.313534 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.313679 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8434,"prepare.queue_time_us":41481,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":142,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"thread_start_us":1066720,"threads_started":2,"wal-append.queue_time_us":810581}
I20260709 12:24:27.316009 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.288962 26201 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
W20260709 12:24:26.631762 26199 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.004s
I20260709 12:24:27.316694 26199 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:27.317190 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 357) took 1591 ms. Trace:
I20260709 12:24:27.317274 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.317577 26199 rpcz_store.cc:276] 0709 12:24:25.725697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.862933 (+137236us) service_pool.cc:224] Handling call
0709 12:24:27.317134 (+1454201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.318557 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:26.025979 26202 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
W20260709 12:24:27.319293 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 359) took 1441 ms (client timeout 1527 ms). Trace:
W20260709 12:24:27.319389 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1109) took 1640 ms (client timeout 1864 ms). Trace:
W20260709 12:24:26.300319 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.257828 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 1.26555e+06us
W20260709 12:24:26.563640 26120 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.003s
W20260709 12:24:26.864780 26126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2303) took 3748 ms (client timeout 1618 ms). Trace:
I20260709 12:24:27.319952 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:26.088289 28377 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.002s
W20260709 12:24:27.297117 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.321190 26126 rpcz_store.cc:269] 0709 12:24:23.116027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.117157 (+  1130us) service_pool.cc:224] Handling call
0709 12:24:26.864275 (+3747118us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b5400186fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:26.864521 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4214710016}
I20260709 12:24:27.296425 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 696) took 1228 ms. Trace:
I20260709 12:24:27.257524 26786 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:27.208809 28466 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:24:26.067167 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:26.543864 28275 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.783s	user 0.002s	sys 0.007s
I20260709 12:24:27.284546 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:27.300635 26791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.010s	sys 0.005s
I20260709 12:24:27.253456 26198 rpcz_store.cc:276] 0709 12:24:24.215078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.933344 (+718266us) service_pool.cc:224] Handling call
0709 12:24:26.016800 (+1083456us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400302d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:26.016971 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956371328}
W20260709 12:24:26.087203 28337 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.005s
W20260709 12:24:26.596516 28339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.006s	sys 0.026s
I20260709 12:24:27.301885 26049 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:27.302376 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1608) took 1732 ms. Trace:
I20260709 12:24:27.331921 26125 rpcz_store.cc:276] 0709 12:24:25.562918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.568334 (+  5416us) service_pool.cc:224] Handling call
0709 12:24:27.294911 (+1726577us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54001c4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:27.295031 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718892800}
W20260709 12:24:26.967847 28453 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.117s	user 0.000s	sys 0.023s
W20260709 12:24:27.264925 28087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.010s	sys 0.001s
I20260709 12:24:27.355883 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 690) took 1473 ms. Trace:
W20260709 12:24:27.356269 28087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.011s	sys 0.002s
I20260709 12:24:27.330801 28336 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.356469 26786 rpcz_store.cc:276] 0709 12:24:25.882688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.885941 (+  3253us) service_pool.cc:224] Handling call
0709 12:24:26.015367 (+129426us) spinlock_profiling.cc:243] Waited 886 us on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:27.355787 (+1340420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2038016}
I20260709 12:24:27.358011 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:27.358939 28336 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.359699 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:26.678318 28440 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:26.262480 28380 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.311743 28270 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.363664 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:27.233579 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 497) took 1336 ms. Trace:
W20260709 12:24:27.257527 26790 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.001s	sys 0.005s
W20260709 12:24:27.368028 26790 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.002s	sys 0.005s
I20260709 12:24:26.554759 28386 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.310602 26200 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:27.303861 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:27.315078 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.312556 26203 rpcz_store.cc:276] 0709 12:24:25.506729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.874302 (+367573us) service_pool.cc:224] Handling call
0709 12:24:25.920623 (+ 46321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.962898 (+ 42275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.962945 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.041949 (+ 79004us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.510283 (+468334us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:26.510345 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.510446 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.977922 (+467476us) raft_consensus.cc:1685] finished
0709 12:24:26.977950 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.978157 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8904,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5545,"thread_start_us":320458,"threads_started":1,"wal-append.queue_time_us":785390}
I20260709 12:24:26.329823 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.318565 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:27.309523 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1118) took 1348 ms. Trace:
I20260709 12:24:27.320730 26120 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:24:27.321619 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.204898 28468 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.483s	user 0.000s	sys 0.000s
I20260709 12:24:27.369287 26047 rpcz_store.cc:276] 0709 12:24:25.961222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.012578 (+ 51356us) service_pool.cc:224] Handling call
0709 12:24:27.309453 (+1296875us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:27.211019 28354 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.225s	user 0.012s	sys 0.001s
I20260709 12:24:26.666280 28379 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.319749 26201 rpcz_store.cc:269] 0709 12:24:25.878049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.906886 (+ 28837us) service_pool.cc:224] Handling call
0709 12:24:27.319231 (+1412345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.324046 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2340) took 1296 ms. Trace:
I20260709 12:24:27.324680 26050 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:27.303391 26783 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.327623 26789 rpcz_store.cc:276] 0709 12:24:26.067857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.093131 (+ 25274us) service_pool.cc:224] Handling call
0709 12:24:27.296366 (+1203235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.326357 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:27.325615 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 352) took 1810 ms (client timeout 1887 ms). Trace:
W20260709 12:24:27.335035 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50601.7us
W20260709 12:24:27.377102 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2342) took 1215 ms (client timeout 1618 ms). Trace:
I20260709 12:24:27.309504 26194 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.282897 28478 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:24:27.360750 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2337) took 1426 ms (client timeout 1757 ms). Trace:
W20260709 12:24:27.361249 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.407387 28467 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.858s	user 0.001s	sys 0.010s
W20260709 12:24:27.407719 26169 connection.cc:256] RPC call timeout handler was delayed by -1.6446744330000911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 12:24:27.365650 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:27.408908 28267 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.409188 26198 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:27.410395 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:27.411044 26198 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:27.360198 26053 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:27.412525 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1119) took 1428 ms. Trace:
I20260709 12:24:27.412633 28336 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:27.412866 26053 rpcz_store.cc:276] 0709 12:24:25.984661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.996761 (+ 12100us) service_pool.cc:224] Handling call
0709 12:24:27.412452 (+1415691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.413358 28336 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:27.413883 28478 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.413983 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.414463 26201 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:27.414875 26053 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:27.335752 28339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.533s	user 0.019s	sys 0.054s
W20260709 12:24:27.360954 26788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.007s	sys 0.010s
W20260709 12:24:27.319876 26202 rpcz_store.cc:269] 0709 12:24:25.686475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.831767 (+145292us) service_pool.cc:224] Handling call
0709 12:24:27.319334 (+1487567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.371224 26122 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.373690 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.332319 28371 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:26.022660 28420 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.331593 28377 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.415107 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1106) took 1778 ms. Trace:
I20260709 12:24:27.361176 28254 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.329414 28486 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:27.415599 26048 rpcz_store.cc:269] 0709 12:24:25.934264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.937429 (+  3165us) service_pool.cc:224] Handling call
0709 12:24:27.360668 (+1423239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.416883 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.416595 26054 raft_consensus.cc:2368] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6.
W20260709 12:24:27.367659 26787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.011s	sys 0.048s
W20260709 12:24:27.452833 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 153242us
W20260709 12:24:27.406077 26051 rpcz_store.cc:269] 0709 12:24:26.162164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.329434 (+167270us) service_pool.cc:224] Handling call
0709 12:24:27.377040 (+1047606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.373121 28440 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.421895 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:27.420727 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.372485 28270 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:27.449352 26788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.007s	sys 0.015s
I20260709 12:24:27.354696 28337 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.460340 28479 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.460567 28165 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:27.460587 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.461194 28165 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:27.461376 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.462131 26048 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:27.462193 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.462354 28165 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.463708 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2343) took 1221 ms. Trace:
I20260709 12:24:27.463928 28336 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:27.453950 28478 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:27.464609 28336 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.464883 28478 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.465168 28336 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:27.416663 28454 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.614s	user 0.007s	sys 0.029s
W20260709 12:24:27.405035 26197 rpcz_store.cc:269] 0709 12:24:25.510484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.530293 (+ 19809us) service_pool.cc:224] Handling call
0709 12:24:27.320776 (+1790483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.468014 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 691) took 1574 ms. Trace:
I20260709 12:24:27.372709 26786 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:27.419122 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 495) took 1666 ms. Trace:
I20260709 12:24:27.459714 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.375002 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 693) took 1428 ms. Trace:
I20260709 12:24:27.463897 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2344) took 1039 ms. Trace:
I20260709 12:24:27.459120 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:27.458339 28466 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:27.466789 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.456972 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:27.456501 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:27.464089 26048 rpcz_store.cc:276] 0709 12:24:26.256662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.462765 (+1206103us) service_pool.cc:224] Handling call
0709 12:24:27.463028 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.463389 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.463418 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.463457 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:27.463500 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.463553 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.463661 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:27.469590 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.475035 26053 rpcz_store.cc:276] 0709 12:24:26.429638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.463160 (+1033522us) service_pool.cc:224] Handling call
0709 12:24:27.463427 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.463593 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.463615 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.463676 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:27.463710 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.463764 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.463853 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:27.477505 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.475984 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:27.478012 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.478420 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.449173 26201 rpcz_store.cc:276] 0709 12:24:25.636201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.413812 (+1777611us) service_pool.cc:224] Handling call
0709 12:24:27.414085 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.414857 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.415046 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:27.456871 26787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.021s	sys 0.062s
W20260709 12:24:27.456179 28455 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.654s	user 0.009s	sys 0.023s
I20260709 12:24:27.377200 28438 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.472712 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:27.476715 26789 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.479179 28316 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:27.492197 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.469566 26788 rpcz_store.cc:276] 0709 12:24:25.893204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.937698 (+ 44494us) service_pool.cc:224] Handling call
0709 12:24:25.959982 (+ 22284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.968483 (+  8501us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:25.968782 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.458842 (+1490060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.458887 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:27.458931 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.458985 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.467965 (+  8980us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110766,"prepare.queue_time_us":3805,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":51,"raft.queue_time_us":111603,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":1378620,"threads_started":2,"wal-append.queue_time_us":679413}
I20260709 12:24:27.406711 26045 rpcz_store.cc:276] 0709 12:24:26.027365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.028485 (+  1120us) service_pool.cc:224] Handling call
0709 12:24:27.323965 (+1295480us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.462384 28337 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.483296 26197 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:27.457464 28440 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:27.484704 26052 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:27.486333 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.475630 26785 rpcz_store.cc:276] 0709 12:24:25.944905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.971424 (+ 26519us) service_pool.cc:224] Handling call
0709 12:24:25.971807 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:26.461014 (+489207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.461072 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:26.559083 (+ 98011us) log.cc:844] Serialized 52 byte log entry
0709 12:24:27.321032 (+761949us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:27.329337 (+  8305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.335662 (+  6325us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.354619 (+ 18957us) raft_consensus.cc:1685] finished
0709 12:24:27.361468 (+  6849us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.372832 (+ 11364us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":542770,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":278,"thread_start_us":606989,"threads_started":1,"wal-append.queue_time_us":598406}
W20260709 12:24:27.490370 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2336) took 1603 ms (client timeout 1727 ms). Trace:
I20260709 12:24:27.490625 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:27.490820 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:27.474839 28339 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:27.500650 26054 rpcz_store.cc:269] 0709 12:24:25.887412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.902454 (+ 15042us) service_pool.cc:224] Handling call
0709 12:24:27.490138 (+1587684us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:27.490298 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454726016}
I20260709 12:24:27.504720 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.484584 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:27.498176 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 361) took 1029 ms. Trace:
I20260709 12:24:27.418839 28386 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.508041 26197 rpcz_store.cc:276] 0709 12:24:26.468756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.473972 (+1005216us) service_pool.cc:224] Handling call
0709 12:24:27.483181 (+  9209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.497913 (+ 14732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.498117 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.417680 28379 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.460958 28479 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.503245 28440 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.501844 28357 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.523708 26052 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:27.508940 28312 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.501370 28275 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:24:27.503649 26786 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:27.528187 28357 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.534478 28482 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.534767 28481 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:27.535161 28485 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:27.533699 26199 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:27.495903 28438 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.549099 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.550339 26203 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:24:27.560536 26203 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 4
I20260709 12:24:27.560853 26203 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:24:27.561092 26203 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:27.561137 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 698) took 1347 ms. Trace:
I20260709 12:24:27.561595 26788 rpcz_store.cc:276] 0709 12:24:26.213351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.510224 (+1296873us) service_pool.cc:224] Handling call
0709 12:24:27.520713 (+ 10489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.560758 (+ 40045us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:27.560787 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.560827 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:27.560880 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.560939 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.561061 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:27.544196 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53679.2us
I20260709 12:24:27.519515 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 692) took 1586 ms. Trace:
I20260709 12:24:27.504004 28316 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.549162 28498 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:27.563323 26047 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:27.563432 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 364) took 1087 ms. Trace:
I20260709 12:24:27.571753 26199 rpcz_store.cc:276] 0709 12:24:26.469764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.486041 (+1016277us) service_pool.cc:224] Handling call
0709 12:24:27.495742 (+  9701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.556716 (+ 60974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.556913 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.557380 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 455) took 1033 ms. Trace:
I20260709 12:24:27.611668 26786 rpcz_store.cc:276] 0709 12:24:26.521763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.490436 (+968673us) service_pool.cc:224] Handling call
0709 12:24:27.555562 (+ 65126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.613138 28479 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.564179 26787 rpcz_store.cc:276] 0709 12:24:25.930724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.932264 (+  1540us) service_pool.cc:224] Handling call
0709 12:24:25.939244 (+  6980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.939670 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:25.940095 (+   425us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.482585 (+1542490us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.492947 (+ 10362us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:27.499680 (+  6733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.507111 (+  7431us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.517167 (+ 10056us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":186706,"prepare.queue_time_us":17919,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":203955,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":1223920,"threads_started":2,"wal-append.queue_time_us":1022495}
W20260709 12:24:27.613808 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.614290 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.614753 28490 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.615131 26052 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:27.615262 28377 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.615304 28490 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.615985 28377 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.616525 28377 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.556900 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.617515 26045 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:27.617868 28490 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.618391 28377 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.618871 26125 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:27.619376 26195 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:24:27.554466 28357 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.545589 28312 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:27.570813 26047 pending_rounds.cc:85] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 1
I20260709 12:24:27.554678 28420 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.623714 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:27.624481 26047 pending_rounds.cc:107] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:27.626482 26047 raft_consensus.cc:2889] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:27.627729 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 219243us
I20260709 12:24:27.556558 28485 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:27.561187 28478 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:27.631174 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:27.632090 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.629510 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:27.528187 28339 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.618955 26045 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:27.635552 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.636318 26127 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.620307 28488 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.636839 26126 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:27.614769 26787 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:27.636781 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.138s (SENT) [suppressed 42 similar messages]
I20260709 12:24:27.637264 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.616529 28484 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.640820 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.648340 26788 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:27.648980 26128 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.652489 26789 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:27.658630 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:27.659217 26125 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.635365 28485 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.367766 26782 rpcz_store.cc:276] 0709 12:24:25.896778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.904204 (+  7426us) service_pool.cc:224] Handling call
0709 12:24:25.904639 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.928495 (+ 23856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.928550 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.937311 (+  8761us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.352637 (+415326us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:26.378753 (+ 26116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.437944 (+ 59191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.233148 (+795204us) raft_consensus.cc:1685] finished
0709 12:24:27.233171 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.233375 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19125,"raft.queue_time_us":21881,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8432,"thread_start_us":315381,"threads_started":1,"wal-append.queue_time_us":1255286}
I20260709 12:24:27.541383 26201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48518
I20260709 12:24:27.669883 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.688418 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 498) took 1769 ms. Trace:
I20260709 12:24:27.673243 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 700) took 1192 ms. Trace:
I20260709 12:24:27.689428 26788 rpcz_store.cc:276] 0709 12:24:26.464336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.563012 (+1098676us) service_pool.cc:224] Handling call
0709 12:24:27.622311 (+ 59299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.656602 (+ 34291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.656921 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.674603 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.689002 26791 rpcz_store.cc:276] 0709 12:24:25.902858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.905062 (+  2204us) service_pool.cc:224] Handling call
0709 12:24:25.905368 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.905748 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:26.342416 (+436668us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.565715 (+1223299us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.671898 (+106183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.671958 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:27.672002 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.672061 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.672182 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30006,"prepare.queue_time_us":25761,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":92,"raft.queue_time_us":83624,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":1456167,"threads_started":4,"wal-append.queue_time_us":800624}
I20260709 12:24:27.637988 28354 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:27.679394 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.697172 26127 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.697288 26126 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.655982 28492 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:27.684054 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50547.5us
I20260709 12:24:27.687069 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:27.699395 26051 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.669678 28497 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.691190 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:27.648725 28479 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.634737 28312 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.621953 28482 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.693197 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2345) took 1142 ms. Trace:
I20260709 12:24:27.655682 28316 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.682390 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.684471 28165 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:27.704780 28495 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.704842 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.716212 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.681645 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 699) took 1298 ms. Trace:
I20260709 12:24:27.706493 26049 rpcz_store.cc:276] 0709 12:24:26.555360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.474523 (+919163us) service_pool.cc:224] Handling call
0709 12:24:27.482975 (+  8452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.496617 (+ 13642us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:27.507475 (+ 10858us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.691766 (+184291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.691851 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:27.691922 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.693007 (+  1085us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.693134 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14739,"prepare.queue_time_us":6236,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4891,"thread_start_us":126851,"threads_started":1}
I20260709 12:24:27.695631 28267 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.696513 28354 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.726972 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.718611 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.728538 26126 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:27.729848 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.730804 26126 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.719796 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:27.473778 26784 rpcz_store.cc:276] 0709 12:24:25.750358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.751184 (+   826us) service_pool.cc:224] Handling call
0709 12:24:25.751624 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.752095 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:25.860894 (+108799us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.330464 (+469570us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.630161 (+299697us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.660412 (+ 30251us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:26.722179 (+ 61767us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.238368 (+516189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.238489 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.392477 (+153988us) raft_consensus.cc:1685] finished
0709 12:24:27.392530 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.417038 (+ 24508us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":675506,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2786,"raft.queue_time_us":49397,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":294,"thread_start_us":504987,"threads_started":2,"wal-append.queue_time_us":746523}
I20260709 12:24:27.733397 26124 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:27.734669 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:27.736852 28504 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:27.737442 28504 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:27.738471 28504 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.703487 28497 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.740465 28497 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.744254 28386 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.701326 28492 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.745559 28492 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.745870 28165 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:27.746369 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1112) took 1964 ms. Trace:
I20260709 12:24:27.746992 28165 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.748044 28275 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:27.748711 28275 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.750911 28275 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.740370 28496 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:27.659941 28488 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.747224 28267 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.720988 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 685) took 4311 ms. Trace:
I20260709 12:24:27.746774 26202 rpcz_store.cc:276] 0709 12:24:25.774387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.453390 (+1679003us) service_pool.cc:224] Handling call
0709 12:24:27.482250 (+ 28860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.513755 (+ 31505us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:27.675455 (+161700us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.721464 (+ 46009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.721537 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:27.721598 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.722221 (+   623us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.722355 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":208313,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":178648,"threads_started":2}
I20260709 12:24:27.672760 28484 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.718539 28316 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.763481 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1103) took 2285 ms. Trace:
I20260709 12:24:27.719334 28336 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:27.765048 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45795.6us
I20260709 12:24:27.766139 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.767056 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.768481 28377 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.768929 28377 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.768679 28506 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:27.769315 28377 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.724417 28495 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.771519 28495 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.772516 28490 raft_consensus.cc:2764] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:27.773295 28490 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:27.773715 28490 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.774075 28490 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.775605 28491 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:27.711458 28479 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:27.722536 28481 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:27.776916 28491 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.777318 28491 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:27.777657 28481 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.778390 28479 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:27.778255 28491 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.779008 26201 raft_consensus.cc:606] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Received request to transfer leadership
I20260709 12:24:27.779403 26201 raft_consensus.cc:614] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:27.780457 28500 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.781121 28500 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:27.764081 28484 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.784310 28375 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.786262 28312 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.772083 28495 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.787442 28495 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.789227 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:27.789810 28495 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.786933 28312 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:27.790525 28312 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:27.791889 28312 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.793318 28312 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.794926 28339 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.795701 28075 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.796307 28075 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:27.797360 28075 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.799974 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1598) took 4304 ms. Trace:
W20260709 12:24:27.801229 26020 connection.cc:256] RPC call timeout handler was delayed by -2.23348205100001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 12:24:27.801837 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 354) took 2210 ms. Trace:
I20260709 12:24:27.768023 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:27.786285 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48518 (request call id 85) took 3011 ms. Trace:
I20260709 12:24:27.802711 26201 rpcz_store.cc:276] 0709 12:24:24.774495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.500051 (+2725556us) service_pool.cc:224] Handling call
0709 12:24:27.786054 (+286003us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:27.786167 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208901888}
I20260709 12:24:27.800421 26129 rpcz_store.cc:276] 0709 12:24:23.495924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.583175 (+ 87251us) service_pool.cc:224] Handling call
0709 12:24:26.865139 (+3281964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.322377 (+457238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.799719 (+477342us) spinlock_profiling.cc:243] Waited 3.28 s on lock 0x7b5400186fd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:27.799882 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7546487040}
I20260709 12:24:27.783640 28316 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.805078 26125 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:27.763927 26200 rpcz_store.cc:276] 0709 12:24:25.477693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.630424 (+152731us) service_pool.cc:224] Handling call
0709 12:24:27.310353 (+1679929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.368716 (+ 58363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.763274 (+394558us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400379450. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:27.763416 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3862512000}
I20260709 12:24:27.820734 26051 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:27.809528 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.830607 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:27.802245 26198 rpcz_store.cc:276] 0709 12:24:25.591186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.411806 (+1820620us) service_pool.cc:224] Handling call
0709 12:24:27.412173 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.502223 (+ 90050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.502272 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.513511 (+ 11239us) log.cc:844] Serialized 26 byte log entry
0709 12:24:27.723357 (+209846us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:27.723410 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.723499 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.801611 (+ 78112us) raft_consensus.cc:1685] finished
0709 12:24:27.801632 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.801777 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78208,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":188607,"threads_started":2,"wal-append.queue_time_us":245901}
W20260709 12:24:27.833794 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65851.8us
I20260709 12:24:27.753774 26790 rpcz_store.cc:276] 0709 12:24:23.401635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.088975 (+687340us) service_pool.cc:224] Handling call
0709 12:24:24.113759 (+ 24784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.113965 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:24.114281 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.556958 (+2442677us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b180043db00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:24:27.704982 (+1148024us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.705138 (+   156us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:27.705315 (+   177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.705532 (+   217us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.712585 (+  7053us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3788,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3606,"raft.queue_time_us":41260,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":1633,"spinlock_wait_cycles":21885056,"thread_start_us":1124342,"threads_started":2,"wal-append.queue_time_us":3170365}
W20260709 12:24:27.850104 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 211245us
I20260709 12:24:27.771735 28497 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.851962 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.834015 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.764098 28267 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:27.784354 28087 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:24:27.783656 28500 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:27.856079 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.861763 28087 consensus_queue.cc:1243] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:27.833081 28479 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:27.863234 28087 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:27.815514 28275 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:27.863691 28479 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:27.864414 28510 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:27.864910 28275 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:27.866029 28479 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:27.869715 28316 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:27.870167 26168 connection.cc:256] RPC call timeout handler was delayed by -1.3365121519996137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 12:24:27.870357 28316 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.870608 26168 connection.cc:256] RPC call timeout handler was delayed by -1.3005121519995555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 12:24:27.871016 26168 connection.cc:256] RPC call timeout handler was delayed by -1.2591121519994886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
I20260709 12:24:27.870893 28316 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:27.871474 26168 connection.cc:256] RPC call timeout handler was delayed by -1.2384121519999098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 12:24:27.872277 26121 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:27.871855 26168 connection.cc:256] RPC call timeout handler was delayed by -1.2114121519998662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:24:27.873135 26168 connection.cc:256] RPC call timeout handler was delayed by -1.1790121519998138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:24:27.873639 26168 connection.cc:256] RPC call timeout handler was delayed by -1.155612151999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 12:24:27.872807 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.874050 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0836121519996595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:24:27.875366 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0782121519996508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
I20260709 12:24:27.875504 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:27.875872 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0503121519996057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
I20260709 12:24:27.718062 26785 rpcz_store.cc:276] 0709 12:24:26.374467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.517327 (+1142860us) service_pool.cc:224] Handling call
0709 12:24:27.561282 (+ 43955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.627397 (+ 66115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.629779 (+  2382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.635033 (+  5254us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:27.636645 (+  1612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.652364 (+ 15719us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.673129 (+ 20765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.863695 28087 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:27.815397 28496 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.883625 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:27.899271 26127 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.898252 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.869794 28497 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:27.876965 28336 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.874497 28270 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:27.904421 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 197260us
I20260709 12:24:27.903383 28440 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.877220 28498 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:27.909890 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:27.880097 28494 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.871801 28377 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:27.911152 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 159401us
I20260709 12:24:27.913805 26126 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.915165 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:27.916445 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 367421us
I20260709 12:24:27.928704 26128 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:27.929587 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:27.937767 26194 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.001s
I20260709 12:24:27.938575 26194 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:27.939267 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 353) took 2398 ms. Trace:
I20260709 12:24:27.939605 26194 rpcz_store.cc:276] 0709 12:24:25.540799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.985975 (+445176us) service_pool.cc:224] Handling call
0709 12:24:25.986677 (+   702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.938944 (+1952267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.939197 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.940795 28380 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.940544 28087 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.941884 28087 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.918171 28494 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.934514 28298 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.941398 28380 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:27.947762 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50762.4us
I20260709 12:24:27.948130 26129 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:27.947463 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:27.917212 28270 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:27.969341 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:27.978948 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47183.5us
I20260709 12:24:27.982227 28495 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:27.947052 28497 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:24:27.983316 28495 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.948168 28267 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.983951 28495 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:27.984174 28267 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.985075 28495 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.986358 28495 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.945112 28298 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:27.983357 28497 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:27.926025 28336 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:27.944175 28496 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:27.976215 28481 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:27.995362 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 160747us
I20260709 12:24:27.993216 28497 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:28.010262 28478 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:27.986364 28375 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:27.997318 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:28.014201 25952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 188) took 15556 ms (client timeout 15000 ms). Trace:
I20260709 12:24:28.025803 28496 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:28.000391 28515 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:28.026764 28496 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:27.995031 28516 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:28.002156 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:28.015355 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:27.997627 27798 raft_consensus.cc:2764] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:28.009431 28339 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:28.051604 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:28.000957 28336 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.022452 28375 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.001267 28481 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.027408 28496 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.062958 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:28.080356 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.094719 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84294.4us
I20260709 12:24:28.053260 28316 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:28.048982 28516 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:28.122503 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73772.3us
I20260709 12:24:28.035018 28479 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:24:28.062711 28481 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.996384 28511 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 12:24:28.060683 28509 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:27.998965 28270 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.025830 28497 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:28.156476 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.099907 28485 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.159063 28514 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:24:28.061982 28339 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.159862 26124 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:28.179785 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:28.180693 26788 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:28.175889 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.135381 28478 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:28.251001 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 248963us
W20260709 12:24:28.228072 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.181511 28316 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:28.250730 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 246458us
W20260709 12:24:28.258054 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.162106 28474 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.878s	user 0.007s	sys 0.007s
I20260709 12:24:28.262404 28440 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:28.168733 28357 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.535s	user 0.004s	sys 0.007s
I20260709 12:24:28.262655 28497 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.173319 28492 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.173326 28506 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:28.263365 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:28.282662 26789 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:28.171795 26784 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEARNER]: Advancing to term 6
W20260709 12:24:28.263994 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.258222 26094 connection.cc:256] RPC call timeout handler was delayed by -2.416088727999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
I20260709 12:24:28.131719 28516 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.138252 28509 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:28.263837 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:28.301787 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:28.021946 25952 rpcz_store.cc:269] 0709 12:24:12.432133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.432298 (+   165us) service_pool.cc:224] Handling call
0709 12:24:27.988719 (+15556421us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:22.399335 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:22.582697 (+183362us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:22.584740 (+  2043us) write_op.cc:435] Acquired schema lock
0709 12:24:22.594457 (+  9717us) tablet.cc:662] Decoding operations
0709 12:24:22.725956 (+131499us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:22.726028 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:22.726063 (+    35us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:24:22.726658 (+   595us) tablet.cc:701] Row locks acquired
0709 12:24:22.726680 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:24:22.726843 (+   163us) write_op.cc:270] Start()
0709 12:24:22.726964 (+   121us) write_op.cc:276] Timestamp: P: 1783599862726797 usec, L: 0
0709 12:24:22.726991 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:24:22.746548 (+ 19557us) log.cc:844] Serialized 2898 byte log entry
0709 12:24:23.055627 (+309079us) op_driver.cc:464] REPLICATION: finished
0709 12:24:23.300081 (+244454us) write_op.cc:301] APPLY: starting
0709 12:24:23.300195 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:23.357280 (+ 57085us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:23.373341 (+ 16061us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:23.762707 (+389366us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:23.791951 (+ 29244us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:24:23.792009 (+    58us) write_op.cc:312] APPLY: finished
0709 12:24:23.792366 (+   357us) log.cc:844] Serialized 59 byte log entry
0709 12:24:23.792560 (+   194us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:23.804992 (+ 12432us) write_op.cc:454] Released schema lock
0709 12:24:23.860720 (+ 55728us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:23.903406 (+ 42686us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:23.909247 (+  5841us) write_op.cc:454] Released schema lock
0709 12:24:25.270855 (+1361608us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a232a 00007f5b74327f14 00007f5b74327ec3 00007f5b74327e38 00007f5b74327df3 00007f5b743270dc 00007f5b74350e16 00007f5b74338cbb 00007f5b7432339d 00007f5b74326d75
Metrics: {"mutex_wait_us":939495,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":235455,"apply.run_cpu_time_us":10415,"apply.run_wall_time_us":581261,"num_ops":7,"prepare.queue_time_us":458941,"prepare.run_cpu_time_us":21737,"prepare.run_wall_time_us":433994,"raft.queue_time_us":81852,"raft.run_cpu_time_us":45310,"raft.run_wall_time_us":2261752,"replication_time_us":310684,"spinlock_wait_cycles":41233664,"thread_start_us":562432,"threads_started":4,"wal-append.queue_time_us":134654,"wal-append.run_cpu_time_us":7883,"wal-append.run_wall_time_us":2015528}]]}
I20260709 12:24:28.281759 28357 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.286399 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.288365 28354 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.292423 28506 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:28.292553 28316 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:28.296743 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 171535us
I20260709 12:24:28.035256 28498 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.160308 28514 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.143s
W20260709 12:24:28.308580 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.329257 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1114) took 2484 ms. Trace:
I20260709 12:24:28.372669 26203 rpcz_store.cc:276] 0709 12:24:25.844470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.466936 (+1622466us) service_pool.cc:224] Handling call
0709 12:24:27.477002 (+ 10066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.566267 (+ 89265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.328903 (+762636us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:28.329167 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147119360}
I20260709 12:24:28.373512 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1097) took 4159 ms. Trace:
I20260709 12:24:28.274390 28377 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:28.284811 26783 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.867s	user 0.008s	sys 0.004s
I20260709 12:24:28.374065 26196 rpcz_store.cc:276] 0709 12:24:24.214135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.017164 (+803029us) service_pool.cc:224] Handling call
0709 12:24:26.019726 (+1002562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.258959 (+1239233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.373263 (+1114304us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400378550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:28.373439 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304358144}
I20260709 12:24:28.383703 28537 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:28.384378 26783 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:24:28.384874 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 697) took 2222 ms (client timeout 2389 ms). Trace:
W20260709 12:24:28.385339 26783 rpcz_store.cc:269] 0709 12:24:26.162130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.519852 (+357722us) service_pool.cc:224] Handling call
0709 12:24:28.384805 (+1864953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:28.384917 28537 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.351404 28492 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.386205 28537 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:28.386656 28537 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:28.351873 28490 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
I20260709 12:24:28.388525 28485 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.388617 28490 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.320926 28509 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.356685 28336 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:28.405982 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.414184 26786 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:28.385756 28514 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:28.415459 28514 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:28.393158 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1117) took 2455 ms. Trace:
I20260709 12:24:28.418833 26046 rpcz_store.cc:276] 0709 12:24:25.941318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.948223 (+  6905us) service_pool.cc:224] Handling call
0709 12:24:25.967671 (+ 19448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.968114 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:26.012600 (+ 44486us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.779963 (+1767363us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b18003cacc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:28.351812 (+571849us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b18003cacc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b
0709 12:24:28.353585 (+  1773us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b18003cacc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:24:28.392657 (+ 39072us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:28.392728 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:28.392807 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.392900 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.393050 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38486,"prepare.queue_time_us":52141,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":31664,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":219,"spinlock_wait_cycles":104814464,"thread_start_us":410909,"threads_started":1,"wal-append.queue_time_us":712187}
W20260709 12:24:28.422106 26000 auto_rebalancer.cc:893] Removing replace marker failed: Replica a981c9a7972c4e99a0903924c64fbb5c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:24:28.422588 26000 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:24:28.422871 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:24:28.428680 28506 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.392275 28481 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:28.433327 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 408240us
W20260709 12:24:28.434654 26019 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.602s (SENT)
W20260709 12:24:28.434696 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 133230us
I20260709 12:24:28.435290 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:28.432092 28525 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 12:24:28.434875 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 171113us
I20260709 12:24:28.439535 26123 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:28.440876 28375 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.442065 28481 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.443282 28375 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.446673 26785 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:28.447494 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:28.448537 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:28.448863 28357 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:28.449600 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 163306us
I20260709 12:24:28.450492 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:28.450860 28377 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:28.451309 28377 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.451676 28377 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.416430 28514 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.452714 28316 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:28.433610 28529 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:24:28.455664 26128 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:28.456672 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:28.457545 28485 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.457952 28485 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:28.458302 28485 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.459600 28496 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.453310 28514 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.451824 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.468860 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:24:28.475322 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.475360 26094 connection.cc:256] RPC call timeout handler was delayed by -2.2279887280001276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 12:24:28.476008 26094 connection.cc:256] RPC call timeout handler was delayed by -2.1901887280000665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 12:24:28.476670 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7284887280002295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24550000000000002s for a response
I20260709 12:24:28.477214 26020 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:28.477797 26020 leader_election.cc:403] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.483147 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.471817 28336 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:28.490338 28357 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:28.472277 28535 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.494071 28535 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.457424 28516 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.498386 28516 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.474073 28495 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:28.500763 26094 connection.cc:256] RPC call timeout handler was delayed by -1.29918872799999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2932s for a response
W20260709 12:24:28.501180 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1470799389999229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260709 12:24:28.501490 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1362799390003602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 12:24:28.501785 26094 connection.cc:256] RPC call timeout handler was delayed by -1.064288728000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31930000000000003s for a response
W20260709 12:24:28.502074 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0445284140000695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 12:24:28.503310 28519 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:28.508111 28519 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.510318 28519 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:28.511263 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.511920 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:28.512976 26788 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
I20260709 12:24:28.514348 26093 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
I20260709 12:24:28.514623 26093 leader_election.cc:403] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.515235 28519 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.473688 28420 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:28.487941 28479 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:28.521045 28482 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.521699 28509 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.523934 28504 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:28.524410 28504 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:28.499653 28336 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:28.525048 28336 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.525430 28504 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.526845 28504 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.529840 28486 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:28.533439 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 95205.1us
W20260709 12:24:28.535175 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 100540us
I20260709 12:24:28.531570 28537 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.489781 28490 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.500144 28357 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.543028 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61902.5us
W20260709 12:24:28.547269 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64221.5us
I20260709 12:24:28.562940 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.534636 28530 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.542299 28357 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.571768 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.572727 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.573589 26053 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:28.576107 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.578118 26047 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:28.577071 26053 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:28.580368 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.580520 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.581164 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:28.584410 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:28.585055 26050 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.585196 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:28.586812 28379 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.588536 28537 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.561842 28479 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.584566 26784 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:24:28.561828 28496 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.592059 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:28.592418 26120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 311) took 2971 ms (client timeout 1887 ms). Trace:
W20260709 12:24:28.592836 26120 rpcz_store.cc:269] 0709 12:24:25.620555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.620729 (+   174us) service_pool.cc:224] Handling call
0709 12:24:28.592249 (+2971520us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:28.592350 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683419392}
I20260709 12:24:28.593348 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.594465 28485 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.596810 28519 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
W20260709 12:24:28.599857 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51655.7us
I20260709 12:24:28.562307 26785 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:28.600404 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 321) took 2855 ms (client timeout 3359 ms). Trace:
I20260709 12:24:28.538375 28377 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.560227 28533 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.603286 26121 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:28.565188 28535 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:28.590570 26784 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 4
W20260709 12:24:28.606490 26019 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.211s (SENT)
I20260709 12:24:28.558445 28509 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.603807 26126 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:28.564762 28316 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:28.604800 26122 rpcz_store.cc:269] 0709 12:24:25.745195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.821473 (+ 76278us) service_pool.cc:224] Handling call
0709 12:24:28.600185 (+2778712us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001f5ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:28.600345 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3550677248}
I20260709 12:24:28.584435 28438 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:28.607692 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49609.7us
W20260709 12:24:28.610900 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.614593 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:28.617594 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.623153 26123 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:28.594527 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.625030 26053 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
W20260709 12:24:28.625798 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 363) took 2146 ms (client timeout 1500 ms). Trace:
I20260709 12:24:28.626240 26053 pending_rounds.cc:85] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 3
I20260709 12:24:28.626580 26053 pending_rounds.cc:107] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:24:28.626325 26195 rpcz_store.cc:269] 0709 12:24:26.469448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.307571 (+838123us) service_pool.cc:224] Handling call
0709 12:24:28.609161 (+1301590us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:28.615352 (+  6191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489343744}
I20260709 12:24:28.626942 26053 raft_consensus.cc:2889] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.637168 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:28.613080 28533 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.609256 26784 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:24:28.640470 26784 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.644641 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.648531 26046 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4. Re-sending same reply.
I20260709 12:24:28.652761 27798 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:28.653276 27798 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:28.653680 28494 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.656216 28380 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.655929 28377 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.656981 28380 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:28.659344 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:28.661144 28377 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:28.652848 28485 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.662281 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:28.665199 26784 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:28.666023 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59540.8us
W20260709 12:24:28.666640 26019 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.212s (SENT)
I20260709 12:24:28.667948 28537 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.668665 26049 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:28.669503 28484 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:28.672049 28533 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:28.673204 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.674242 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:28.674901 26783 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.675581 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:28.664642 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48013.7us
I20260709 12:24:28.663923 28491 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.677186 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:28.678833 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.682945 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:28.683022 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:28.684943 26052 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:28.689272 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:28.685668 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71819.7us
I20260709 12:24:28.691980 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.699714 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:28.701460 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.659286 28530 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:28.684710 28504 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:28.655905 28357 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.692423 28386 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.848536 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:28.849216 26053 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.701678 28494 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:28.856339 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 156734us
I20260709 12:24:28.857874 26790 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:28.858356 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 164840us
I20260709 12:24:28.816864 28504 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:28.862129 28504 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.847004 28498 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:28.864925 28498 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:28.839707 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:28.865983 28498 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.865155 26054 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:24:28.868368 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.871335 26784 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:28.872339 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.873732 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.874138 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.727798 28312 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:28.868368 28357 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:28.876533 28312 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:28.878188 28298 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.877199 28504 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.879527 28312 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.882705 28312 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.879227 28479 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:28.885277 28267 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.888075 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:28.888633 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1623) took 1085 ms. Trace:
I20260709 12:24:28.889066 28275 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:28.889019 26123 rpcz_store.cc:276] 0709 12:24:27.803431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.440467 (+637036us) service_pool.cc:224] Handling call
0709 12:24:28.440759 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.623811 (+183052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.888378 (+264567us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000b95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:28.888567 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418826496}
I20260709 12:24:28.889751 28275 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:28.890916 28275 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:28.896839 27798 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.900058 26049 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:28.883404 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:28.904888 26168 leader_election.cc:400] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6.
I20260709 12:24:28.905542 26168 leader_election.cc:403] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.911933 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.915544 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.906687 28504 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:28.916291 26054 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:28.926017 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:28.929059 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:28.929487 26053 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:28.933357 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.934561 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:28.937469 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.938634 28478 raft_consensus.cc:2764] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:28.939648 28496 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6.
I20260709 12:24:28.939500 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:28.940474 28496 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.940965 28496 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:28.941217 28479 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.924925 28530 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.945182 28386 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:28.945429 28339 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:28.941308 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 454) took 2419 ms. Trace:
I20260709 12:24:28.946354 26786 rpcz_store.cc:276] 0709 12:24:26.521444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.613016 (+1091572us) service_pool.cc:224] Handling call
0709 12:24:28.413911 (+800895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.414761 (+   850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.941095 (+526334us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54004f4550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:28.941249 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794259840}
I20260709 12:24:28.949167 28504 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:28.906929 28551 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:28.966193 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1617) took 1211 ms. Trace:
I20260709 12:24:28.967000 26121 rpcz_store.cc:276] 0709 12:24:27.755681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.984031 (+228350us) service_pool.cc:224] Handling call
0709 12:24:28.600887 (+616856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.612984 (+ 12097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.965984 (+353000us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54001f5ed0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:28.966133 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417636608}
W20260709 12:24:28.969713 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.966132 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104222us
I20260709 12:24:28.923866 28533 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:28.974279 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:28.977092 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58822.4us
I20260709 12:24:28.980768 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:28.983444 26196 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:24:28.983996 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:28.984578 26195 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.962024 28386 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:28.963011 28380 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:29.004721 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.010514 26129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1620) took 1207 ms (client timeout 1585 ms). Trace:
I20260709 12:24:29.010412 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:29.011165 26129 rpcz_store.cc:269] 0709 12:24:27.802624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.815299 (+ 12675us) service_pool.cc:224] Handling call
0709 12:24:29.010247 (+1194948us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:29.010434 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186947840}
I20260709 12:24:28.982239 28500 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:28.981998 28533 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:29.015494 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78227.4us
I20260709 12:24:28.990226 28497 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.962284 28479 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.011441 26196 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:29.017349 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:28.958271 28530 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:29.024012 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:28.995535 28478 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:29.027599 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.981916 28087 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:24:29.041512 26203 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:29.028749 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:29.029726 28486 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.995859 28537 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:24:29.047330 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.005764 28339 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:24:29.051761 26196 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:29.008664 28336 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:29.055330 28535 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:29.061829 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.083828 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 102549us
I20260709 12:24:29.091703 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:29.031347 28497 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.055548 26194 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:29.034946 28500 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:29.117075 28496 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:29.123123 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.096215 28339 consensus_queue.cc:1243] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:29.123788 26203 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.127810 28494 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:29.130560 28479 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:29.133060 28552 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 12:24:29.134501 28553 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:24:29.135533 28546 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.231s	user 0.000s	sys 0.002s
I20260709 12:24:29.138392 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.082840 28386 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:29.138468 28497 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:29.140509 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.141144 26200 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:29.142500 26202 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:29.108949 26194 pending_rounds.cc:85] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 5
I20260709 12:24:29.143217 26194 pending_rounds.cc:107] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:29.143539 26194 raft_consensus.cc:2889] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.143744 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.144403 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:29.149252 28533 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:29.151506 26783 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:29.152271 28375 raft_consensus.cc:2764] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:24:29.152691 28547 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:24:29.118155 26201 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:29.160427 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:29.178659 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:29.174183 26054 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:29.191529 26196 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:29.192414 26198 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:29.194828 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 166199us
W20260709 12:24:29.195655 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 112285us
I20260709 12:24:29.197758 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:29.203594 28298 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:29.207492 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:29.207665 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.208559 26203 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:29.213737 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.214358 26203 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:29.216374 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:29.217044 26051 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:29.217075 28087 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:29.221612 26203 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:29.189911 28494 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.231925 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.233274 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.234885 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:29.235854 26051 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.204152 28380 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:24:29.237542 28336 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:29.238471 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 509) took 1067 ms. Trace:
I20260709 12:24:29.238905 26787 rpcz_store.cc:276] 0709 12:24:28.171346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.174145 (+  2799us) service_pool.cc:224] Handling call
0709 12:24:29.237658 (+1063513us) spinlock_profiling.cc:243] Waited 74.1 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:29.237876 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.238078 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:29.238110 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:29.238165 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:29.238207 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.238272 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.238408 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170488192}
I20260709 12:24:29.241426 28500 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:29.242015 26194 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.246852 28500 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:29.247418 28500 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.233961 28530 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:29.250221 28500 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:29.223278 28087 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:29.253856 28087 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.255167 28087 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.282781 26051 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:29.285513 26194 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:29.286412 28488 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:29.223625 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:29.289508 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.296774 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.297582 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.313416 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.317449 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:29.318886 26053 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:29.319458 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.318238 26049 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.318186 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.325666 26124 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:29.497470 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 174554us
W20260709 12:24:29.498613 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.505340 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.506501 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.507304 26198 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.515672 26049 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
W20260709 12:24:29.517313 26756 connection.cc:256] RPC call timeout handler was delayed by -1.3836170250001487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
I20260709 12:24:29.518477 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:29.519340 26051 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:29.521482 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 297902us
I20260709 12:24:29.517295 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.522946 28380 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:29.523727 28380 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:29.524220 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.525403 26125 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:29.526276 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.526547 28380 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:29.526877 26125 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:29.527872 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:29.529320 26816 heartbeater.cc:646] Failed to heartbeat to 127.24.71.190:42395 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.24.71.190:42395 timed out after 15.000s (SENT)
I20260709 12:24:29.537264 26125 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:29.537460 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:29.529820 26194 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:29.531505 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.538878 26120 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:29.532106 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.539932 26202 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.540275 26122 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:29.532712 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.533291 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:29.534117 26046 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:29.517870 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.538498 26050 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:29.497011 28275 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.545604 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.547845 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.541311 26127 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.557279 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.558300 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.561177 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.561467 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.562296 26127 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:29.562505 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.563225 26786 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:29.564224 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.575152 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.575911 26122 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.576555 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.573661 26198 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
W20260709 12:24:29.577584 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.580592 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:29.581738 26785 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:29.584714 26128 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.590530 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.593904 26120 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:29.594864 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.595911 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:29.597576 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 507) took 1427 ms (client timeout 1777 ms). Trace:
I20260709 12:24:29.599539 26127 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
W20260709 12:24:29.601558 26790 rpcz_store.cc:269] 0709 12:24:28.170667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.173881 (+  3214us) service_pool.cc:224] Handling call
0709 12:24:28.585061 (+411180us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:29.597389 (+1012328us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:29.597519 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628816128}
I20260709 12:24:29.602324 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.603332 26127 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.603183 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.597635 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:29.604938 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.606034 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.607867 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.609009 26120 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:29.611129 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.611799 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.039s	sys 0.084s
W20260709 12:24:29.615303 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.039s	sys 0.084s
I20260709 12:24:29.614050 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:29.616158 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 524492us
I20260709 12:24:29.618618 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:29.623518 26201 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:29.632520 28557 raft_consensus.cc:2764] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:29.634538 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.635370 26122 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.636399 26125 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:29.637409 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.637995 26129 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.638669 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.651031 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.599759 26785 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:29.652451 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:29.651702 26121 raft_consensus.cc:2376] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8. Re-sending same reply.
I20260709 12:24:29.660442 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 326) took 1930 ms. Trace:
I20260709 12:24:29.661144 26127 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.660955 26126 rpcz_store.cc:276] 0709 12:24:27.729647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.944596 (+214949us) service_pool.cc:224] Handling call
0709 12:24:28.603636 (+659040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.609157 (+  5521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.660108 (+1050951us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54001d1fd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:29.660367 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514920832}
I20260709 12:24:29.636075 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.679507 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.676600 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.678534 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:29.667054 28386 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.014s	sys 0.089s
I20260709 12:24:29.681563 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.684130 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:29.689149 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.690810 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.691591 26198 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:29.657704 28485 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:29.692098 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.695333 26202 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:29.699124 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.700590 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.701406 26126 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:29.701262 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.703246 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.701884 26201 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.703799 26198 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.697914 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.702231 26787 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:29.711514 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:29.690062 26783 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:29.712783 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.714154 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.715287 28498 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:29.718370 28087 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.721781 28087 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.718670 28312 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.723354 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:29.724994 26124 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:29.724016 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.723956 26790 raft_consensus.cc:2376] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7. Re-sending same reply.
I20260709 12:24:29.705987 28298 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.729743 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.732475 26124 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.732592 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.733381 26126 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:29.735606 28087 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:29.774571 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:29.726442 26194 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:29.722777 28312 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.781059 26790 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:29.781270 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.783563 26124 raft_consensus.cc:2376] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6. Re-sending same reply.
I20260709 12:24:29.782388 28087 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.782734 26127 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:29.784426 28087 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:29.786895 28087 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:29.788683 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.791993 26201 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
W20260709 12:24:29.793514 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 239453us
I20260709 12:24:29.795326 26198 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:29.773247 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:29.797199 26203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.041s	sys 0.090s
W20260709 12:24:29.797726 26203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.041s	sys 0.090s
I20260709 12:24:29.796777 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.799108 26198 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:29.800724 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.808522 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.809401 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.809464 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:29.740659 28298 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:29.810837 26199 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:29.811903 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.812055 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.812537 26199 raft_consensus.cc:2376] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5. Re-sending same reply.
I20260709 12:24:29.813480 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:29.818853 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.820611 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.821455 26198 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:29.807134 28486 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.033s	sys 0.000s
I20260709 12:24:29.824362 26201 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:29.824955 26199 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:29.830883 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:29.831625 26790 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:29.834911 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:29.835634 26124 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:29.828545 28486 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.034s	sys 0.000s
I20260709 12:24:29.838434 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.810473 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:29.842582 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.844130 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.844950 26196 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:29.845664 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:29.846486 26124 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:29.849556 28380 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:29.851404 28380 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:29.861927 27798 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.868752 28379 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:29.869402 28379 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:29.871062 28379 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.010002 28379 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:30.207991 28567 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:24:29.877274 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.010285 26787 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:30.010651 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.220055 26127 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:30.154795 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.038705 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 216771us
I20260709 12:24:30.242081 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.242918 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:29.961716 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.883509 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 172081us
W20260709 12:24:30.080428 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.014464 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:30.222187 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 212615us
W20260709 12:24:30.222250 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 124126us
W20260709 12:24:29.877257 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 116493us
I20260709 12:24:29.874769 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:29.895396 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.233605 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.237598 26201 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:30.213879 26125 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:30.216049 26121 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:30.244007 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.246579 26196 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:30.254637 26790 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:30.015121 28566 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
W20260709 12:24:30.247495 26757 leader_election.cc:336] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.887s (SENT)
I20260709 12:24:30.237598 26128 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:30.248216 26169 leader_election.cc:336] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.618s (SENT)
W20260709 12:24:30.248656 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.262998 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.246982 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:30.249272 26019 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 4.062s (SENT)
W20260709 12:24:30.260026 26129 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.610s	user 0.004s	sys 0.002s
I20260709 12:24:30.265527 26129 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:30.266390 26126 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:30.123095 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.262149 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:30.253455 28568 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
W20260709 12:24:30.278913 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 464108us
I20260709 12:24:30.267669 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.212397 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.279759 26201 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:30.280900 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.223323 28386 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:30.244988 28312 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:30.239760 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.273372 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.286235 26201 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:30.288407 26196 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:30.289342 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.290056 26196 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:30.290310 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.267627 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:29.882658 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.256767 28377 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:30.256951 28485 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:30.264915 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:30.254410 28087 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:30.284226 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.277565 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.272886 28357 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:30.291487 26197 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:30.295329 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.289074 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:30.303555 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55366.7us
I20260709 12:24:30.284166 28298 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:30.305289 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.305508 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.308820 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:30.312127 26199 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:30.312669 26790 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:30.285717 28577 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:30.314795 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.315420 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
W20260709 12:24:30.316998 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.289568 28386 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:30.318214 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.319321 26791 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:30.308826 28486 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:30.309252 28357 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.295694 28312 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:24:30.307331 28377 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:30.317058 28577 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:30.322242 28575 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.130s	user 0.000s	sys 0.002s
W20260709 12:24:30.322003 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72759.6us
I20260709 12:24:30.328595 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 372) took 1100 ms. Trace:
I20260709 12:24:30.322711 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.329218 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.308277 28562 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:30.330456 26195 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:30.342550 26019 leader_election.cc:336] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.561s (SENT)
I20260709 12:24:30.318792 28506 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:30.343156 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.341938 26199 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.346180 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:30.341284 26203 rpcz_store.cc:276] 0709 12:24:29.228163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.228327 (+   164us) service_pool.cc:224] Handling call
0709 12:24:29.228695 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:29.234539 (+  5844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:29.234567 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:29.240460 (+  5893us) log.cc:844] Serialized 26 byte log entry
0709 12:24:29.798219 (+557759us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:29.798276 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.798365 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:30.328345 (+529980us) raft_consensus.cc:1685] finished
0709 12:24:30.328361 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.328518 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2116,"thread_start_us":557342,"threads_started":1,"wal-append.queue_time_us":635504}
I20260709 12:24:30.322108 28496 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.352577 26789 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:30.353127 26784 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:30.318393 28087 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.356338 28551 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:30.359184 28551 consensus_queue.cc:1243] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:30.357708 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.360531 26196 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:30.358413 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.322402 28486 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:30.323771 28357 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.342787 28577 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.356020 28377 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:30.371876 28578 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:24:30.361840 26195 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:30.368957 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.346601 28506 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:30.370123 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.365114 28486 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.430212 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.366392 28562 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:30.445164 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.447489 26203 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:30.447961 26200 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:30.449218 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.450104 26196 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:30.451995 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.455199 26199 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:30.456717 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.457499 26199 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.456723 26195 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:30.417578 26786 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:30.463416 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 217370us
I20260709 12:24:30.359997 28496 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.473608 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.474972 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.476128 26200 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:30.474959 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.476924 26203 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:30.477203 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.477969 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:30.467012 28581 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:24:30.446617 28562 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:30.488797 26196 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:30.495157 28312 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:30.496263 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.496207 28312 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:30.497947 28312 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:30.498672 26196 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:30.500942 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 197234us
I20260709 12:24:30.492530 26199 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:30.502206 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:30.502952 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.503973 26199 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
W20260709 12:24:30.504634 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 217538us
I20260709 12:24:30.505399 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.505764 26199 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:30.499152 28312 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:30.512503 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 169972us
W20260709 12:24:30.513511 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.514528 26019 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.585s (SENT)
W20260709 12:24:30.515177 26019 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.635s (SENT)
I20260709 12:24:30.502638 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.515938 26203 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:30.515763 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.514528 26196 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:30.517560 26199 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:24:30.525842 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48705.2us
I20260709 12:24:30.533049 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.533735 26203 raft_consensus.cc:2376] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5. Re-sending same reply.
I20260709 12:24:30.533649 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.535008 26020 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
W20260709 12:24:30.536055 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 214691us
I20260709 12:24:30.536284 26020 leader_election.cc:403] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:30.521314 28485 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:30.558965 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.560122 26200 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:30.561807 26203 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:30.562821 28580 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:30.565116 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.566882 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.568686 28570 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:30.569331 28087 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:30.569999 28087 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:30.570374 28570 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:30.570909 28087 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:30.571177 28570 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:30.572676 28580 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:30.574908 28580 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.579640 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.539180 28596 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.580394 26199 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:30.580145 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.568054 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:30.530228 28594 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:30.581939 26196 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:30.600673 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 98610.1us
W20260709 12:24:30.604794 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 67229.6us
I20260709 12:24:30.606365 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:30.606786 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 99656.2us
I20260709 12:24:30.544932 28275 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.552702 28312 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:30.546756 28557 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.580772 28596 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.609683 28375 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:30.611126 28357 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.611958 28591 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:30.579746 28486 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.601140 28496 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:30.603232 28571 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:30.581598 28583 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:30.612941 28597 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:24:30.613431 28595 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:30.613812 28485 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.485713 28506 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.614241 28494 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:30.614336 28386 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:30.607486 27798 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.005s
I20260709 12:24:30.615021 28379 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:30.619083 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 342073us
W20260709 12:24:30.620096 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 82269.6us
I20260709 12:24:30.620568 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 340) took 1028 ms. Trace:
I20260709 12:24:30.624981 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.678186 26201 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
I20260709 12:24:30.624985 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:30.625833 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:30.678539 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.651328 28583 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.629994 28275 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.638200 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.642475 28357 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:30.642926 28087 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:30.643867 28562 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.637981 28557 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.645316 28600 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:30.632635 28586 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:30.661931 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.677796 26122 rpcz_store.cc:276] 0709 12:24:29.591767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.658433 (+ 66666us) service_pool.cc:224] Handling call
0709 12:24:29.674832 (+ 16399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.675027 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:30.034135 (+359108us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.606327 (+572192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.606385 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:30.606439 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.620417 (+ 13978us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.620523 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33533,"prepare.queue_time_us":615382,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":88887,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":10852,"spinlock_wait_cycles":39040,"thread_start_us":886111,"threads_started":3,"wal-append.queue_time_us":466455}
I20260709 12:24:30.651360 28375 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:30.732669 28312 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:30.667659 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.671586 28506 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.673238 28486 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:30.743386 28557 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:30.744733 28562 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:30.679348 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53632.2us
I20260709 12:24:30.667095 28494 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:30.677045 28379 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.747568 28494 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:30.629995 28580 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:30.749153 28379 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:30.682169 28596 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:30.682453 26194 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:30.749657 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.750479 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 468) took 1040 ms. Trace:
I20260709 12:24:30.750797 26788 rpcz_store.cc:276] 0709 12:24:29.710485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.749370 (+1038885us) service_pool.cc:224] Handling call
0709 12:24:30.750418 (+  1048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:30.684808 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.689213 28595 consensus_queue.cc:1243] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:30.752545 26201 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:24:30.699896 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.685345 28380 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:30.753269 26200 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:30.701057 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.703248 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:30.665742 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.754128 26199 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:30.734987 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.754966 26203 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:30.740396 28375 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:30.741012 28312 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:30.701126 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80037.7us
I20260709 12:24:30.738178 28275 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:30.789722 28580 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:30.791867 28591 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:30.763624 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.764672 28557 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.765394 28551 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:30.765785 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.768693 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.811904 26201 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:30.812245 26790 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:30.752162 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.753428 28479 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.735837 28298 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:30.771059 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.814248 26203 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:30.815586 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.772064 28375 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.776358 28486 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:30.776540 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:30.778784 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77816.1us
I20260709 12:24:30.781332 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:30.786413 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.786470 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:30.788192 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:30.736423 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 130120us
I20260709 12:24:30.693207 28583 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:30.792377 28377 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:30.805855 28591 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.785748 28586 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:30.792898 28595 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:30.793467 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.793503 28594 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.761058 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.868388 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.801702 26198 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:24:30.804435 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 202960us
W20260709 12:24:30.869201 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.806756 26785 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:30.807406 28597 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:30.788853 28312 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.875519 26122 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:30.760528 28380 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:30.876205 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:30.876957 26124 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:30.771427 28494 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.879977 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:30.856302 26194 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 3.
I20260709 12:24:30.856077 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:30.886534 26201 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 3: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:30.868333 28606 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:30.890679 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.873927 28379 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:24:30.891638 26049 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:30.892781 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:30.874383 28375 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:30.894045 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:30.894626 26121 raft_consensus.cc:2376] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6. Re-sending same reply.
I20260709 12:24:30.817828 28479 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.858224 26200 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:30.869086 26128 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:30.805738 26126 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:30.869475 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1508) took 1166 ms. Trace:
I20260709 12:24:30.869941 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:30.871255 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.899721 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.877717 28380 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.812300 28557 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.868765 26196 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:30.872960 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.873214 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.817818 28551 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:30.878119 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.878439 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:30.906611 26124 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.907055 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:30.816735 26203 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:30.869102 28377 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:30.860119 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.910801 26129 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:30.871163 28596 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:30.889355 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.912746 26051 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:30.914441 26051 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 12:24:30.888206 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.888615 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:30.861251 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74834.9us
I20260709 12:24:30.917002 26120 raft_consensus.cc:2376] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6. Re-sending same reply.
I20260709 12:24:30.917390 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:30.918708 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:30.868146 28298 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:24:30.898568 26198 rpcz_store.cc:276] 0709 12:24:29.703272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.839763 (+136491us) service_pool.cc:224] Handling call
0709 12:24:30.763417 (+923654us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:30.869417 (+106000us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56725376}
I20260709 12:24:30.871902 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:30.900605 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.901939 26054 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:30.901607 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 111334us
I20260709 12:24:30.903287 26050 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:30.870355 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.904356 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.867224 28486 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.902040 26052 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:30.924285 26202 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:30.902102 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.910979 26053 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:30.901098 28606 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:30.919559 26122 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:30.919100 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:30.912204 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.949898 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:30.951803 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:30.952253 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:30.958680 28486 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:30.959790 26045 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:30.872879 28594 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:30.964534 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:30.972303 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.974104 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.947373 28377 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:30.958681 26046 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:30.922858 26048 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:24:30.950564 26049 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:30.968978 28580 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:30.968727 28596 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:30.975291 28597 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:30.976958 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 106702us
I20260709 12:24:30.986761 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:30.993192 26053 raft_consensus.cc:2410] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:31.031708 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:30.969594 28586 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:30.993029 28377 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:31.045789 28377 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:31.047325 28583 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.035763 28607 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:24:31.049146 28607 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:31.049515 28607 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:31.050326 28607 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.051571 28607 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:30.993477 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 123591us
I20260709 12:24:31.055546 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:30.997766 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:31.057406 26045 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.002635 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.005102 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.005467 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:30.980314 26124 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
W20260709 12:24:31.061743 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56378.3us
W20260709 12:24:31.061944 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 199102us
I20260709 12:24:31.034876 28494 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:31.063889 27798 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:31.064364 27798 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:31.064479 28599 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:31.064900 27798 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.065019 28599 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:31.066285 26052 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:30.982374 28571 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:31.042078 28577 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.623s	user 0.008s	sys 0.001s
I20260709 12:24:31.063138 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:31.013216 26167 leader_election.cc:336] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.138s (SENT)
I20260709 12:24:31.084522 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:30.982283 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:31.051587 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1506) took 1362 ms. Trace:
W20260709 12:24:31.085417 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 103282us
I20260709 12:24:31.085292 26202 rpcz_store.cc:276] 0709 12:24:29.689494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.696203 (+  6709us) service_pool.cc:224] Handling call
0709 12:24:29.857077 (+160874us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:30.765415 (+908338us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280e9c 00007f5b71280a5b 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:31.051384 (+285969us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:31.051526 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311490816}
I20260709 12:24:31.009915 28479 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:31.000681 28612 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:31.059171 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 3.
I20260709 12:24:31.011016 26048 pending_rounds.cc:85] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 4
W20260709 12:24:31.014539 26018 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.171s (SENT)
I20260709 12:24:31.088166 26048 pending_rounds.cc:107] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:24:31.086210 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:31.088492 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74016.7us
I20260709 12:24:31.088611 26048 raft_consensus.cc:2889] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:30.984990 28379 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:31.059157 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72528us
W20260709 12:24:30.995268 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 123429us
I20260709 12:24:31.016851 28357 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:31.019210 28551 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:31.004276 28613 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.020627 28596 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:31.024825 28597 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:31.212177 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 120440us
I20260709 12:24:31.000706 28611 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.079636 28087 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:31.004293 28589 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:30.997004 28609 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:31.012661 28580 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:31.076808 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45176.9us
I20260709 12:24:31.207899 28494 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:31.090297 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.001387 28569 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.036610 28557 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:31.165591 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78716.9us
W20260709 12:24:31.186338 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.165537 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 152331us
I20260709 12:24:31.213361 28588 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:31.214108 28597 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.004307 28610 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.083166 28571 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:31.213941 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 130570us
I20260709 12:24:31.209972 28479 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:24:31.092139 28357 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:31.088419 28616 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:30.983842 28275 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.208388 28485 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.222873 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:31.058234 28600 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.155s
I20260709 12:24:31.165233 28617 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.212493 28596 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.002980 28606 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.057302 28506 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:31.200555 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 128320us
I20260709 12:24:31.252719 28583 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.237229 26045 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:31.238430 26167 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.205s (SENT)
W20260709 12:24:31.300388 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61997.9us
I20260709 12:24:31.302024 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:31.236279 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.216429 28603 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.223574 28580 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.092031 28577 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.186060 28591 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.010682 28594 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.250730 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:31.318490 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.255854 26202 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:31.238965 28588 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.223773 28551 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 12:24:31.284761 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61957.1us
I20260709 12:24:31.165109 28562 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:31.287148 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:31.326570 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.326673 26196 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.290886 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:31.247326 28571 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:31.216105 28087 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.335412 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.293985 26199 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:31.293331 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.234963 28380 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.340948 25950 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 3 to 5, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.342144 28380 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:31.343106 26125 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:31.342799 25950 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.304505 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:31.245182 28557 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:24:31.344300 28087 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.207898 28599 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:31.332974 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.334280 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:31.348506 26128 raft_consensus.cc:2376] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4. Re-sending same reply.
I20260709 12:24:31.327702 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:31.337839 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.350543 26120 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:31.339773 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.344637 26200 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.323664 28588 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.321010 28479 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.345992 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:31.352773 25950 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 4 to 5, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.326327 28562 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:31.365971 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 129783us
W20260709 12:24:31.389288 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96021.7us
W20260709 12:24:31.384791 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 165409us
I20260709 12:24:31.375706 25950 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.330109 28379 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:31.345036 28506 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.388581 26196 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:31.390035 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:31.384020 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 133346us
I20260709 12:24:31.347900 28594 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:31.405465 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.398223 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 308008us
I20260709 12:24:31.399958 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:31.400514 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.400552 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:31.361877 28583 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:31.408439 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.421651 26019 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:31.392055 28599 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.392385 28562 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.410646 25950 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.405263 28506 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.394199 28626 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.398075 28613 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:31.435240 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:31.336086 28357 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.437359 26053 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:31.413261 28583 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:31.463889 26053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "689b9b7393b349afbf302f0f99710830"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:31.463635 26045 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:31.427152 28557 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.463946 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.423722 28643 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.474414 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.475163 26126 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:31.467536 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.468353 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:31.472404 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.477368 26123 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:31.477406 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:31.447976 25950 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 3, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:31.470261 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.479252 26128 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:31.242774 28569 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.443681 28626 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.480278 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.481329 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.461659 26200 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:31.482173 28569 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.482690 26128 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:31.234746 28601 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.267s
I20260709 12:24:31.483393 28626 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:31.473062 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.452308 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.484537 26127 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:31.470606 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.485153 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.485309 26050 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 3.
I20260709 12:24:31.485785 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:31.470261 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.486734 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:31.486925 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:31.469249 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.487607 25950 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.489251 26126 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:31.491417 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.491541 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:31.492084 26050 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 2.
I20260709 12:24:31.424562 28479 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:24:31.438380 28506 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.496842 26169 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:31.498965 26169 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:31.465910 28631 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.274394 28616 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.476717 26047 raft_consensus.cc:2376] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5. Re-sending same reply.
I20260709 12:24:31.440162 28632 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.483871 26128 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:31.499908 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.463440 28386 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.464874 28583 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:31.507170 28595 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.636s	user 0.008s	sys 0.002s
I20260709 12:24:31.495705 28638 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:31.516587 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:31.444633 28634 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:31.551810 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55582.5us
I20260709 12:24:31.474848 28617 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:31.557705 25950 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 4 to 5, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.496603 28643 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:31.562423 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.496034 28626 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:31.218433 28611 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.496826 28622 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:24:31.577746 26047 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 3.
I20260709 12:24:31.556762 28557 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:31.590821 26047 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:24:31.255075 28610 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:31.318390 28603 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:31.576674 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115808us
I20260709 12:24:31.555310 28486 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:31.579644 28375 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.003s
I20260709 12:24:31.652936 25950 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 3, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.561364 28601 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.575501 26046 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:31.588781 28577 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.563552 28595 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.645828 26047 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 2
I20260709 12:24:31.649688 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.696122 26047 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:31.696554 26047 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:31.696722 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47103.6us
W20260709 12:24:31.651669 28312 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.004s
I20260709 12:24:31.701701 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.648726 28551 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:31.648196 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 459792us
W20260709 12:24:31.581254 28623 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
W20260709 12:24:31.645197 28618 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:24:31.582887 28626 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:31.506036 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 97303us
I20260709 12:24:31.563830 28641 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.673175 28616 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.678061 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:31.706934 25950 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 4, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 4 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.742555 25950 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 4, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 4 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.595307 28386 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:31.747350 28386 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.748034 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:31.709128 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:31.751717 27798 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.686s	user 0.007s	sys 0.004s
I20260709 12:24:31.751959 28577 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.651181 28591 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:31.742684 28626 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.752578 28577 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:31.713404 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.775290 26047 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:31.717314 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 317427us
I20260709 12:24:31.656880 28601 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.775341 28485 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:31.781502 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.776127 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:31.740952 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.651743 28634 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5.
I20260709 12:24:31.753650 26094 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6.
W20260709 12:24:31.788520 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47632.9us
I20260709 12:24:31.712556 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.754552 25950 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 4 to 5, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.705258 28551 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.752504 28585 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 12:24:31.768121 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 449664us
I20260709 12:24:31.800458 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.561151 28632 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.702481 28638 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.752763 28591 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:24:31.657287 28486 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.551844 28631 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.734213 28562 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.791867 26790 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:31.791518 25950 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:31.793416 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.794035 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.742354 28641 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:31.798187 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.711268 28312 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.645789 28642 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.144s
I20260709 12:24:31.804500 28377 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.556097 28379 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.781888 28601 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.892695 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.752166 27798 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.585798 28613 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.785046 28617 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.825335 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:31.915032 26198 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:31.825057 25950 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 4, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 4 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.822006 28641 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.826288 28638 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.826892 28494 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:31.827739 28632 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:31.828064 28654 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:31.829363 26047 pending_rounds.cc:85] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 6
I20260709 12:24:31.805286 28557 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.933406 28557 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:31.934331 28557 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:31.935098 28580 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.610s	user 0.003s	sys 0.006s
W20260709 12:24:31.834895 28655 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:24:31.936477 28651 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:31.835678 28639 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:31.937680 28562 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:31.775310 28648 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:31.828171 26051 raft_consensus.cc:2376] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7. Re-sending same reply.
I20260709 12:24:31.836786 28625 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:31.939316 28603 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.837704 28486 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.939638 28625 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.910248 28377 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:31.941751 28087 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.000s
I20260709 12:24:31.941956 28625 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:31.838586 28631 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.809700 28635 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.942276 28087 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.942968 28635 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.782698 28485 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:31.945250 28635 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:31.839550 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45598.3us
I20260709 12:24:31.947331 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.947907 26129 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:31.940017 28607 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.888s	user 0.007s	sys 0.004s
I20260709 12:24:31.840365 28634 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:31.782467 28577 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:31.840982 28628 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:31.841742 28611 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.842942 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:31.970803 26123 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
W20260709 12:24:31.970943 28628 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:31.971760 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:31.972496 26123 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:31.798894 28595 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.917862 28494 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:31.844561 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:31.976758 26196 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:31.797348 26045 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:31.805462 28551 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:31.864705 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.942334 28377 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:31.983621 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:31.898824 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.908038 26195 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:31.914300 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:31.917223 26054 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:31.919970 25950 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 4, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 4 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:24:31.987213 26202 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.988792 27798 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:31.791129 28643 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.925324 26047 pending_rounds.cc:107] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:24:31.990041 26047 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:31.989928 28494 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:31.932706 28654 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:31.929315 28610 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:31.948753 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:31.999037 28656 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:31.999086 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50405.4us
I20260709 12:24:31.999742 28656 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.000203 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:31.949873 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:32.001286 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.001531 28599 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:31.935611 28580 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.002215 28599 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:31.960750 26049 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:32.004513 28580 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.003363 28599 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:31.963840 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 165733us
I20260709 12:24:32.006484 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:32.007606 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:31.988703 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:31.987877 25950 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 4, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:24:32.008570 26124 raft_consensus.cc:2368] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 3. Current term is 4.
I20260709 12:24:32.009779 28660 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.010320 28660 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.009449 25950 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 2 to 3, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 3 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:32.010748 28660 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:31.948158 26094 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.544s (SENT)
I20260709 12:24:31.993240 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:31.997323 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:32.012770 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64639.1us
I20260709 12:24:31.977969 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:32.013368 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:32.013913 26203 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:32.000386 28656 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.832774 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:32.018254 26050 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.978917 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:32.020704 26196 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.021970 28638 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.014536 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:32.024966 26198 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:31.977427 28571 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 12:24:32.028295 28562 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:32.028903 28562 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.029358 28562 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:32.032660 27798 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.027405 28571 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.035907 28571 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.021540 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.039319 28638 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:32.039845 28638 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.040750 28638 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:32.048424 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 149907us
I20260709 12:24:32.048692 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:32.051770 26197 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:32.052811 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55335.9us
I20260709 12:24:32.057958 26050 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:32.059625 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.991353 28494 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.973791 28380 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.066687 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52033.9us
I20260709 12:24:32.067823 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:32.067853 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:32.082465 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:32.052389 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.075479 26196 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:31.978482 28595 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.976907 28631 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.085614 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.099362 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:31.996870 28087 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.093533 26126 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:32.051697 28596 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.000s
I20260709 12:24:32.103180 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:32.128490 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:32.137269 26123 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.996822 28611 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.123602 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2378) took 1121 ms. Trace:
I20260709 12:24:32.082197 28610 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:32.146255 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:32.147250 26045 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:32.150863 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.087258 26203 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:32.134919 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:32.100899 28642 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:32.140038 26054 rpcz_store.cc:276] 0709 12:24:31.001777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.011036 (+  9259us) service_pool.cc:224] Handling call
0709 12:24:31.026518 (+ 15482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.987208 (+960690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.123222 (+136014us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:32.123504 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047984384}
I20260709 12:24:32.076679 28628 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.096951 28595 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.156446 26197 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:32.131899 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:32.157541 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.155391 28486 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:32.149242 28611 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.136556 28603 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.133827 28607 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.145635 28610 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:32.205940 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48515.5us
I20260709 12:24:32.223062 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:32.094488 28580 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:24:32.142052 28648 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.217550 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:32.235342 26045 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:32.171409 26126 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:32.218645 26050 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:32.100589 28380 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.207315 26202 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:32.169167 28622 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.145123 28596 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.144786 28591 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:24:32.163125 28656 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.098215 28634 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.121s
I20260709 12:24:32.165068 28642 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.331168 28642 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.334323 28562 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.136178 28631 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:32.336827 28597 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.083s	user 0.007s	sys 0.005s
W20260709 12:24:32.337216 28662 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:24:32.337401 28597 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.339015 28631 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:32.217516 28588 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.000s
I20260709 12:24:32.214030 28386 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:32.214920 28603 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.242749 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.342576 26048 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:32.217643 28607 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:32.237834 28610 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:32.230129 27798 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.214469 28486 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:32.346504 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103903us
I20260709 12:24:32.279915 28616 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:32.233249 28375 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:24:32.279196 28599 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.368381 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:32.398540 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 255096us
I20260709 12:24:32.384941 26050 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:32.345830 28603 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.387368 26197 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:32.362708 28607 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.365407 28622 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.217785 28628 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.361336 28631 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.341902 28386 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:32.432042 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.215301 28654 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.345539 28588 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.457758 28648 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:32.401790 28616 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:32.375519 27798 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.477527 28386 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:32.535715 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 483988us
I20260709 12:24:32.412636 28599 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.543609 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:32.540922 26200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.004s	sys 0.014s
W20260709 12:24:32.541538 28485 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.011s	sys 0.014s
I20260709 12:24:32.549014 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:32.475867 28613 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:32.589866 26200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.004s	sys 0.020s
W20260709 12:24:32.594717 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 485089us
I20260709 12:24:32.612130 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:32.479357 28643 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:24:32.568107 26125 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:32.486332 28631 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:32.617246 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 214636us
I20260709 12:24:32.567899 28562 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.543144 28597 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.614575 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:32.605141 28485 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.014s	sys 0.019s
W20260709 12:24:32.637701 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.647818 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:32.602686 28087 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.650413 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.618821 28649 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:32.538219 28648 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.620114 28611 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:32.629124 28557 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:32.660024 28562 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:32.652997 28613 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.666983 28596 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:32.686210 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 137321us
I20260709 12:24:32.651648 28674 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:32.690944 26123 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.000s
W20260709 12:24:32.693274 26018 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.930s (SENT)
I20260709 12:24:32.695714 26123 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:32.695969 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:32.669377 28380 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.676981 28597 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.676715 28087 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.743703 28087 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.721638 27798 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.745260 28609 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:32.745747 28609 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:32.682555 28611 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:32.708935 28583 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.134s	user 0.008s	sys 0.003s
W20260709 12:24:32.710470 28666 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
W20260709 12:24:32.711194 28672 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 12:24:32.713735 28678 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:24:32.714588 28670 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:24:32.684291 28664 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.338s	user 0.002s	sys 0.000s
W20260709 12:24:32.715267 28506 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.195s	user 0.005s	sys 0.007s
W20260709 12:24:32.715862 28667 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
W20260709 12:24:32.717674 28601 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.799s	user 0.007s	sys 0.002s
W20260709 12:24:32.719070 28594 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.305s	user 0.010s	sys 0.002s
I20260709 12:24:32.708154 28375 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:24:32.720528 28599 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:32.705931 28676 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:24:32.725580 28681 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 12:24:32.726727 28626 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.974s	user 0.010s	sys 0.002s
I20260709 12:24:32.703370 28656 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:32.727380 28357 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:32.729518 28638 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.000s
I20260709 12:24:32.817420 28638 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.819888 28638 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.730458 28689 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.722688 28613 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.733286 28631 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:32.704752 28660 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.000s
I20260709 12:24:32.735343 28648 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.736313 28588 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.740123 26126 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:32.741379 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:32.707746 28494 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.001s
I20260709 12:24:32.749128 26127 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:32.750363 26123 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:32.754415 28641 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.000s
W20260709 12:24:32.715332 28671 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 12:24:32.770931 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101157us
W20260709 12:24:32.781158 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 133458us
I20260709 12:24:32.722147 28616 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.721912 28557 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.779888 28551 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 12:24:32.754979 28674 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:32.800330 28594 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.799871 28601 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.849601 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2363) took 1006 ms. Trace:
I20260709 12:24:32.849391 28674 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.850030 26129 rpcz_store.cc:276] 0709 12:24:31.843160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.843367 (+   207us) service_pool.cc:224] Handling call
0709 12:24:31.843675 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.948389 (+104714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.849324 (+900935us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:32.849539 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239306880}
I20260709 12:24:32.850850 28601 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:32.806454 28377 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:24:32.851688 28377 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:24:32.852401 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:32.855384 28595 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.000s
I20260709 12:24:32.857106 26053 raft_consensus.cc:606] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.857553 26053 raft_consensus.cc:614] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:32.859239 28607 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.859654 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60878 (request call id 10) took 1419 ms. Trace:
I20260709 12:24:32.860049 26129 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:32.860260 28656 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.860070 26053 rpcz_store.cc:276] 0709 12:24:31.463032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.463763 (+   731us) service_pool.cc:224] Handling call
0709 12:24:32.859461 (+1395698us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:32.859590 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200513920}
I20260709 12:24:32.777680 28583 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.861943 28486 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.863330 28583 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.793720 28506 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.865294 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:32.865784 28588 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:32.866040 28506 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.837981 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.868410 28696 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:32.868847 28696 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:32.869371 28696 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:32.787492 26018 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.697s (SENT)
I20260709 12:24:32.870318 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:32.810106 28626 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.822167 28628 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.848294 28587 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.828481 28609 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.860913 28656 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.921402 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:32.871667 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:32.878466 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:32.924077 26129 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:32.882414 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:32.879040 28642 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:24:32.932811 26790 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:32.900676 28494 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.881578 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:32.855846 28595 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:32.901571 28607 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:32.871696 28551 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:32.885538 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:32.871307 28557 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.838624 28641 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.908098 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:32.909125 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.904580 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:32.903045 28626 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.909924 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:32.834749 28660 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:32.910975 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:32.829905 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 138038us
I20260709 12:24:32.831524 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1172) took 1001 ms. Trace:
W20260709 12:24:32.917517 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 130066us
I20260709 12:24:32.919778 26052 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:32.930943 26200 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:32.913690 28601 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.933605 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:33.506793 26123 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:33.514038 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.515025 26791 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:33.486212 26018 leader_election.cc:336] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.072s (SENT)
I20260709 12:24:33.488098 26195 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:32.939648 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.488814 26049 rpcz_store.cc:276] 0709 12:24:31.830549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.743119 (+912570us) service_pool.cc:224] Handling call
0709 12:24:32.743442 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.743675 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:32.744012 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.830961 (+ 86949us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.831154 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.831205 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:32.831258 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.831328 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.831463 (+   135us) inbound_call.cc:177] Queueing success response
0709 12:24:32.985739 (+154276us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b1800332860. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
Metrics: {"mutex_wait_us":135,"prepare.queue_time_us":5941,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":111,"raft.queue_time_us":680,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":9319,"spinlock_wait_cycles":319580800,"thread_start_us":84753,"threads_started":1,"wal-append.queue_time_us":178933}
W20260709 12:24:33.054503 26167 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.621s (SENT)
I20260709 12:24:32.962997 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1166) took 1134 ms. Trace:
I20260709 12:24:33.491851 26201 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:33.320569 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1148) took 1474 ms. Trace:
I20260709 12:24:33.494138 26053 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:33.494902 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.152578 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.516283 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.549351 26789 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:33.548317 26199 rpcz_store.cc:276] 0709 12:24:31.845891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.858112 (+ 12221us) service_pool.cc:224] Handling call
0709 12:24:31.907751 (+ 49639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.083648 (+175897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.083674 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.097459 (+ 13785us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.364591 (+267132us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:32.364769 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.313046 (+948277us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400399754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:33.313102 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.313164 (+    62us) raft_consensus.cc:1685] finished
0709 12:24:33.313186 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.320477 (+  7291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5065,"raft.queue_time_us":18460,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":37511,"spinlock_wait_cycles":391710464,"thread_start_us":199242,"threads_started":1,"wal-append.queue_time_us":336763}
I20260709 12:24:33.550856 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:33.551723 26049 raft_consensus.cc:2376] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6. Re-sending same reply.
I20260709 12:24:33.553238 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:33.553412 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:32.941324 28386 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.554919 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:33.467926 28610 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.092s	user 0.013s	sys 0.002s
W20260709 12:24:33.265671 28698 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
I20260709 12:24:33.554579 26049 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:33.503964 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:32.963390 28603 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.001s
W20260709 12:24:33.221619 26020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.890s (SENT) [suppressed 61 similar messages]
I20260709 12:24:33.230206 28562 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:32.982257 26787 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:32.940105 28700 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:33.543401 28377 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:33.502687 28485 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:33.556994 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.559712 26049 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:33.597231 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1177) took 1765 ms. Trace:
I20260709 12:24:33.597589 26049 rpcz_store.cc:276] 0709 12:24:31.831881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.559023 (+1727142us) service_pool.cc:224] Handling call
0709 12:24:33.559371 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.596957 (+ 37586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.597167 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:32.964372 28692 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:33.556214 28386 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:33.599824 28386 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:33.599973 26049 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:33.600659 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1178) took 1538 ms. Trace:
I20260709 12:24:33.600968 26049 rpcz_store.cc:276] 0709 12:24:32.062715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.598556 (+1535841us) service_pool.cc:224] Handling call
0709 12:24:33.599643 (+  1087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.600386 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.600602 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:33.602458 28386 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.556627 28610 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.604734 28485 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.606706 28610 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:33.607316 28610 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:33.571691 26198 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:33.573086 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 86912.8us
W20260709 12:24:33.611043 26018 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.857s (SENT)
I20260709 12:24:33.611486 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:33.573648 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:32.984863 28609 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:33.573853 26053 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:33.620975 26053 pending_rounds.cc:85] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 3
I20260709 12:24:33.621492 26053 pending_rounds.cc:107] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:33.622069 26053 raft_consensus.cc:2889] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:33.582226 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.623202 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1174) took 1792 ms. Trace:
I20260709 12:24:33.623764 26053 rpcz_store.cc:276] 0709 12:24:31.831073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.555797 (+1724724us) service_pool.cc:224] Handling call
0709 12:24:33.556092 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.622857 (+ 66765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.623126 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:33.574880 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:33.569720 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:33.625663 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:33.543016 28494 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.574978 26045 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:33.556555 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:33.587194 28677 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.583324 28562 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.632824 28677 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.588821 28557 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:33.578548 28603 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.590435 28375 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.637885 28375 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.636755 28557 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:33.639628 28557 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:33.640527 26049 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:33.571627 28626 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.640938 28557 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.641916 28626 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.643095 28557 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:33.557054 26047 rpcz_store.cc:276] 0709 12:24:31.828691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.828822 (+   131us) service_pool.cc:224] Handling call
0709 12:24:31.829162 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.990727 (+161565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.990762 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.991167 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.249222 (+258055us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:32.480172 (+230950us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.493052 (+ 12880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.493133 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.962731 (+469598us) raft_consensus.cc:1685] finished
0709 12:24:32.962753 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.962924 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":371255,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3063,"raft.queue_time_us":9685,"raft.run_cpu_time_us":5996,"raft.run_wall_time_us":71612,"thread_start_us":488114,"threads_started":2,"wal-append.queue_time_us":545053}
I20260709 12:24:33.644934 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.645887 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:33.628891 28700 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:33.569034 28675 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.648694 28675 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.590046 28377 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.618950 28610 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:33.653733 28610 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.655793 28675 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:33.625582 26123 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:33.658274 28377 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:33.646818 26047 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:33.660547 28626 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:33.662460 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:33.586764 28696 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.000s
I20260709 12:24:33.663762 26047 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:33.665126 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:33.665939 26047 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:33.666029 28656 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:33.580792 28695 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.008s	sys 0.002s
W20260709 12:24:33.674942 28695 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.008s	sys 0.003s
I20260709 12:24:33.670610 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:33.677145 26783 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:33.663908 28696 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.681437 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:33.682276 26018 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.971s (SENT)
I20260709 12:24:33.683205 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:33.683467 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.672763 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:33.688405 26018 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.031s (SENT)
I20260709 12:24:33.639972 28603 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:33.614028 28613 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:33.695780 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:33.608884 28485 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:33.689745 28626 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:33.635982 28677 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:32.949694 28642 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.699471 26053 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:33.706080 26123 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:33.594123 28601 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:33.713606 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.738595 26783 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:33.739696 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.741153 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:33.743808 26018 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.580s (SENT)
I20260709 12:24:33.744503 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:33.747094 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:33.748776 26053 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 7
W20260709 12:24:33.723876 28654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.012s	sys 0.026s
I20260709 12:24:33.733143 26054 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:24:33.725695 28617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.011s	sys 0.028s
I20260709 12:24:33.767349 26783 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:33.750001 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:33.776439 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.778069 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:33.757900 28654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.012s	sys 0.026s
I20260709 12:24:33.755767 28603 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:33.764802 26194 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:33.783340 26198 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:33.778772 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 724258us
I20260709 12:24:33.780165 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.781188 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:33.784853 26167 leader_election.cc:336] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.720s (SENT)
I20260709 12:24:33.785351 26203 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:33.784880 26202 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:24:33.785779 26167 leader_election.cc:336] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.782s (SENT)
I20260709 12:24:33.786374 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:33.773651 28617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.011s	sys 0.029s
I20260709 12:24:33.786857 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.716631 28610 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:33.731848 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:33.839145 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 107372us
I20260709 12:24:33.840190 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:33.841636 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:33.789590 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 567955us
I20260709 12:24:33.843993 28626 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:33.786908 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:33.844800 28626 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.845871 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:33.846451 26053 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:33.847316 28626 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:33.847541 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:33.759173 28703 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:33.848466 28703 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:33.790647 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.745987 28589 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:24:33.849442 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:33.849748 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.850405 26197 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:33.850533 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:33.855495 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:33.857812 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.864742 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:33.868144 28626 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:33.868908 28626 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:33.870072 28626 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.871701 26197 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:33.873562 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.874784 26197 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:33.813956 26047 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:33.814160 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.815466 26195 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:33.877286 26203 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:33.877689 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:33.877848 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.878321 26047 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:33.879058 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:33.879200 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:33.880159 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.881408 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.882162 26195 raft_consensus.cc:2368] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:33.882965 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:33.814814 26167 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.838s (SENT)
W20260709 12:24:33.884430 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92569.5us
I20260709 12:24:33.814464 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:33.885977 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71616.7us
I20260709 12:24:33.791574 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.813733 26194 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:33.897485 26195 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:33.897866 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:33.790256 28674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.011s	sys 0.018s
I20260709 12:24:33.859038 26202 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.899923 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.901819 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:33.900460 26047 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:33.905849 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:33.906610 26198 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:33.909359 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:33.912796 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 296140us
I20260709 12:24:33.920297 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:33.816063 28654 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.804366 28551 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:33.894165 28485 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:33.894665 28677 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:33.929752 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.050659 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 165526us
W20260709 12:24:34.056421 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 154632us
I20260709 12:24:33.885666 28710 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:33.946286 26202 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:33.902315 28617 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:33.965927 26203 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:33.930088 26053 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:33.909565 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1170) took 2073 ms. Trace:
I20260709 12:24:33.927301 26197 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:34.118937 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:34.090102 26047 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:34.051600 26167 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.010s (SENT)
I20260709 12:24:33.973704 28641 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:34.034078 28654 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:33.892086 28709 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.383s
W20260709 12:24:34.160439 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.254808 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:34.526433 26053 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:33.972179 28700 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:34.121408 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:34.555406 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:34.556514 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:34.557303 26047 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:34.558539 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1186) took 1711 ms. Trace:
I20260709 12:24:34.558848 26047 rpcz_store.cc:276] 0709 12:24:32.846856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.557800 (+1710944us) service_pool.cc:224] Handling call
0709 12:24:34.558074 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.558254 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.558278 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.558318 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:34.558349 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.558398 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.558490 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:34.262329 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 375428us
W20260709 12:24:34.533162 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 612939us
W20260709 12:24:34.534695 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.535254 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 483650us
W20260709 12:24:34.545800 26787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 554) took 2562 ms (client timeout 2166 ms). Trace:
I20260709 12:24:34.546491 28601 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:34.547174 26786 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:34.522598 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:34.548988 26124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1650) took 3214 ms (client timeout 2688 ms). Trace:
I20260709 12:24:34.574595 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:34.575336 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52847.1us
I20260709 12:24:34.576418 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:34.550251 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1169) took 2720 ms. Trace:
I20260709 12:24:34.552507 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:33.973368 28708 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:34.097637 28617 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:34.565434 26167 leader_election.cc:336] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.167s (SENT)
I20260709 12:24:34.570052 26053 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:34.574240 26124 rpcz_store.cc:269] 0709 12:24:31.335013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.335153 (+   140us) service_pool.cc:224] Handling call
0709 12:24:34.548703 (+3213550us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b540037c154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:34.548887 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546176768}
I20260709 12:24:34.577615 26045 rpcz_store.cc:276] 0709 12:24:31.829631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.280329 (+450698us) service_pool.cc:224] Handling call
0709 12:24:32.342338 (+ 62009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.630794 (+1288456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.549926 (+919132us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:34.550180 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2834208000}
W20260709 12:24:34.579527 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1155) took 3112 ms (client timeout 1544 ms). Trace:
W20260709 12:24:34.588456 26051 rpcz_store.cc:269] 0709 12:24:31.467460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.468987 (+  1527us) service_pool.cc:224] Handling call
0709 12:24:34.579305 (+3110318us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400029254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:34.579439 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782373632}
I20260709 12:24:34.590358 26051 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:34.478729 28647 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 2.563s	user 0.017s	sys 0.011s
I20260709 12:24:33.917399 28603 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:34.593852 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:34.554075 28312 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:34.580853 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1179) took 2517 ms. Trace:
I20260709 12:24:34.621073 26049 rpcz_store.cc:276] 0709 12:24:32.063086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.601688 (+1538602us) service_pool.cc:224] Handling call
0709 12:24:33.602013 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.641125 (+ 39112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.580573 (+939448us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b5400030054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:34.580781 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87893888}
I20260709 12:24:34.582819 26050 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:34.579566 26790 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:34.587844 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:34.116969 28494 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:34.593711 26053 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:34.592185 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:34.160348 28626 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.618484 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:34.438822 28707 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.554s	user 0.000s	sys 0.001s
W20260709 12:24:34.613227 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.622799 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 727) took 1750 ms (client timeout 1838 ms). Trace:
I20260709 12:24:34.577370 28613 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:24:34.553920 28715 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.568301 28660 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:34.701637 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82797.8us
I20260709 12:24:34.701969 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2384) took 1085 ms. Trace:
I20260709 12:24:34.702545 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:34.699803 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2393) took 1051 ms. Trace:
I20260709 12:24:34.703665 26049 rpcz_store.cc:276] 0709 12:24:33.648213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.679301 (+1031088us) service_pool.cc:224] Handling call
0709 12:24:34.699124 (+ 19823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.699364 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.699401 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.699459 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:34.699508 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.699579 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.699730 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:34.691605 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 121640us
I20260709 12:24:34.585831 28485 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:34.702518 26053 rpcz_store.cc:276] 0709 12:24:33.616306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.588470 (+972164us) service_pool.cc:224] Handling call
0709 12:24:34.593529 (+  5059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.701698 (+108169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.701908 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.585391 28601 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:34.585088 28700 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:34.595392 28617 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:34.707860 26790 rpcz_store.cc:269] 0709 12:24:32.872216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.552071 (+1679855us) service_pool.cc:224] Handling call
0709 12:24:34.622721 (+ 70650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.700078 26784 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Advancing to term 6
I20260709 12:24:34.703902 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:34.625000 28709 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:34.771291 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 205896us
I20260709 12:24:34.701224 28479 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:34.775560 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1653) took 3436 ms. Trace:
I20260709 12:24:34.691540 28721 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:34.617244 28641 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:34.626055 28551 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:34.780287 26167 connection.cc:256] RPC call timeout handler was delayed by -1.4641928640003243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
I20260709 12:24:34.663300 28312 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:34.784261 26121 rpcz_store.cc:276] 0709 12:24:31.338996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.339119 (+   123us) service_pool.cc:224] Handling call
0709 12:24:31.344983 (+  5864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.371159 (+ 26176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.371199 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.387110 (+ 15911us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.580584 (+193474us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:31.580644 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.580753 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.290719 (+2709966us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:34.770659 (+479940us) raft_consensus.cc:1685] finished
0709 12:24:34.770686 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.775463 (+  4777us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25760,"raft.queue_time_us":95910,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":21318,"spinlock_wait_cycles":5263744,"thread_start_us":175517,"threads_started":1,"wal-append.queue_time_us":418871}
I20260709 12:24:34.706809 28494 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:34.811054 26049 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:34.811450 26049 pending_rounds.cc:85] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 3
W20260709 12:24:34.567662 26787 rpcz_store.cc:269] 0709 12:24:31.982871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.988385 (+  5514us) service_pool.cc:224] Handling call
0709 12:24:34.545537 (+2557152us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b54000e2c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:34.545703 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2203899264}
I20260709 12:24:34.783053 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:34.713001 28715 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.776518 28660 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.781242 28609 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.101s	user 0.006s	sys 0.017s
W20260709 12:24:34.821772 26783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.003s	sys 0.011s
W20260709 12:24:34.823988 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 120194us
W20260709 12:24:34.807835 28713 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.520s	user 0.011s	sys 0.000s
I20260709 12:24:34.823753 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:34.824736 28613 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:24:34.778885 28736 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 12:24:34.808653 28557 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.165s	user 0.017s	sys 0.001s
I20260709 12:24:34.783447 28714 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:34.828475 26783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.011s
I20260709 12:24:34.828858 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:34.788547 28721 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:34.647595 28603 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.812467 28551 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.811681 26049 pending_rounds.cc:107] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260709 12:24:34.787490 28637 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:34.840569 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70975us
I20260709 12:24:34.779670 28730 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:24:34.808074 28709 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.710781 28626 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.809641 28479 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:34.823347 28312 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:34.850066 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:34.823215 26786 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:34.787693 28601 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:34.836460 28641 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:34.841182 28494 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:34.883668 28494 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:34.883939 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1163) took 2021 ms (client timeout 1789 ms). Trace:
I20260709 12:24:34.880606 28641 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:34.884438 26194 rpcz_store.cc:269] 0709 12:24:32.863024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.871363 (+  8339us) service_pool.cc:224] Handling call
0709 12:24:34.883755 (+2012392us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:34.883864 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933406592}
I20260709 12:24:34.878535 28725 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.885699 28725 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.853368 26049 raft_consensus.cc:2917] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260709 12:24:34.776187 28700 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.880443 28601 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.888201 28725 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:34.838807 28660 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:34.894635 28386 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:34.847123 28485 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.848935 28557 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.895174 28386 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:34.895648 28557 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:34.896998 28721 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.869082 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:34.842070 26195 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:34.853139 28551 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.884909 28641 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.888741 26049 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:34.890767 28700 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:34.891916 28601 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:34.851703 28714 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:34.892086 28494 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:34.877689 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.003s	sys 0.013s
W20260709 12:24:34.924217 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.003s	sys 0.013s
I20260709 12:24:34.848723 28613 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:34.853716 28637 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:34.902458 28677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.012s	sys 0.006s
I20260709 12:24:34.902616 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:34.862268 28609 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:34.906044 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:34.911350 28562 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.010s	sys 0.017s
I20260709 12:24:34.860426 28743 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.932157 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77864.7us
I20260709 12:24:34.921095 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1190) took 1052 ms. Trace:
I20260709 12:24:34.933343 26049 rpcz_store.cc:276] 0709 12:24:33.869335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.775968 (+906633us) service_pool.cc:224] Handling call
0709 12:24:34.810724 (+ 34756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.920718 (+109994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.920999 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.897814 28557 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:34.924582 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 867139us
I20260709 12:24:34.899695 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:34.900149 28654 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:34.935062 28743 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.922627 28721 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:34.928870 28677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.013s	sys 0.006s
W20260709 12:24:34.930872 28562 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.010s	sys 0.017s
I20260709 12:24:34.965415 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 564) took 1462 ms. Trace:
I20260709 12:24:34.927287 28613 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:34.974588 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 72083.8us
I20260709 12:24:34.840701 28607 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:24:34.981416 28705 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.932s	user 0.000s	sys 0.002s
I20260709 12:24:34.984789 28714 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:34.986482 28718 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:24:34.989228 28494 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.990971 28557 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.991698 28754 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:35.044900 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.143533 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.173743 26783 rpcz_store.cc:276] 0709 12:24:33.502851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.779695 (+276844us) service_pool.cc:224] Handling call
0709 12:24:33.783158 (+  3463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.783399 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:34.261975 (+478576us) log.cc:844] Serialized 14 byte log entry
0709 12:24:34.848767 (+586792us) log.cc:844] Serialized 14 byte log entry
0709 12:24:34.964936 (+116169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.965009 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:34.965064 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.965155 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.965335 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10482,"prepare.queue_time_us":1082919,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":63258,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":33283,"spinlock_wait_cycles":70090112,"thread_start_us":1072323,"threads_started":3}
I20260709 12:24:34.991853 28654 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:34.952839 28749 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:34.977669 28677 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:35.103698 28740 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:35.199962 28730 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.216732 28649 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:35.216938 28626 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:35.210705 28749 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.258352 28749 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:35.217377 28715 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:34.107575 26046 rpcz_store.cc:276] 0709 12:24:31.829959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.338721 (+508762us) service_pool.cc:224] Handling call
0709 12:24:32.339402 (+   681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.402663 (+ 63261us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:32.662303 (+259640us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.838917 (+176614us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.913520 (+ 74603us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.569101 (+655581us) log.cc:844] Serialized 564 byte log entry
0709 12:24:33.590832 (+ 21731us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:33.609246 (+ 18414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.618767 (+  9521us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.815104 (+196337us) raft_consensus.cc:1685] finished
0709 12:24:33.815134 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.902978 (+ 87844us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59872,"prepare.queue_time_us":223907,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":77067,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":8616,"spinlock_wait_cycles":4702464,"thread_start_us":263284,"threads_started":2,"wal-append.queue_time_us":278916}
I20260709 12:24:35.218098 28740 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:35.264490 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.202726 28557 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:35.301997 28740 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:35.303874 28716 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.087s	user 0.000s	sys 0.009s
I20260709 12:24:35.289644 28600 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:35.279594 28730 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:34.969347 28479 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:35.260977 28653 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.305868 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1570) took 1524 ms. Trace:
I20260709 12:24:35.310789 28312 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:35.261548 28603 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:35.212980 28654 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:35.285995 28649 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.292430 28754 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.292673 28562 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.318264 26198 rpcz_store.cc:276] 0709 12:24:33.781611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.907547 (+125936us) service_pool.cc:224] Handling call
0709 12:24:34.518927 (+611380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:34.519418 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.519459 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:34.520077 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.770181 (+250104us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:34.770227 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.770306 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.305652 (+535346us) raft_consensus.cc:1685] finished
0709 12:24:35.305670 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.305807 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8716,"raft.queue_time_us":6543,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":19860,"thread_start_us":212855,"threads_started":1,"wal-append.queue_time_us":680520}
I20260709 12:24:35.296372 28660 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:35.294169 28733 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:35.142277 28637 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:35.316816 28743 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:35.268077 28613 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:35.421885 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1163) took 3704 ms. Trace:
I20260709 12:24:35.325431 28653 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.392645 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2391) took 1745 ms. Trace:
I20260709 12:24:35.426954 26048 rpcz_store.cc:276] 0709 12:24:31.717290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.746060 (+ 28770us) service_pool.cc:224] Handling call
0709 12:24:31.775132 (+ 29072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.343067 (+567935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.421590 (+3078523us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b5400172d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:35.421821 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304337280}
I20260709 12:24:35.391584 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:35.430851 26783 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:35.433508 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1171) took 3603 ms. Trace:
I20260709 12:24:35.433998 26050 rpcz_store.cc:276] 0709 12:24:31.830264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.407346 (+577082us) service_pool.cc:224] Handling call
0709 12:24:34.582468 (+2175122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.622148 (+ 39680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.433291 (+811143us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b5400029250. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:35.433446 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5001525504}
I20260709 12:24:35.430738 26045 rpcz_store.cc:276] 0709 12:24:33.647341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.591970 (+944629us) service_pool.cc:224] Handling call
0709 12:24:34.627177 (+ 35207us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:34.709420 (+ 82243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.709452 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:34.710224 (+   772us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.928519 (+218295us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:34.929040 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.929144 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.392429 (+463285us) raft_consensus.cc:1685] finished
0709 12:24:35.392452 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.392586 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209125,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8836,"thread_start_us":226695,"threads_started":2,"wal-append.queue_time_us":406736}
I20260709 12:24:35.439567 26203 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:35.397143 26196 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:35.445360 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.446394 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:35.447499 26198 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:35.399958 26197 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.375168 28696 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:35.344391 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:35.421419 28709 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:35.421240 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:35.470394 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:35.471484 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:35.335659 28617 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.002s
I20260709 12:24:35.325245 28600 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:35.437582 28653 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.475620 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:35.477735 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:35.478087 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:35.506026 26045 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:35.506155 26048 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:35.334707 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:35.437070 28730 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:35.480211 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:35.509898 26202 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:35.511638 26202 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:35.452122 28637 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.455507 28613 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.494767 26787 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
W20260709 12:24:35.498343 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.060s	sys 0.093s
I20260709 12:24:35.460935 28562 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:35.507975 26784 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
I20260709 12:24:35.528482 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:35.497490 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:35.525630 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.061s	sys 0.093s
W20260709 12:24:35.535642 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 943485us
I20260709 12:24:35.486127 28617 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:35.542029 26049 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:35.452757 28557 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:35.433791 28733 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:35.531613 26783 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:35.532510 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:35.532629 28600 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:35.539486 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.547444 26049 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 12:24:35.335072 28677 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:35.372560 28754 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:35.550153 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.003s	sys 0.009s
W20260709 12:24:35.552621 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.003s	sys 0.009s
W20260709 12:24:35.552949 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 576938us
I20260709 12:24:35.553874 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:35.522099 26785 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:35.550475 28754 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:35.524487 28569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.005s	sys 0.006s
I20260709 12:24:35.536175 28654 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:35.569527 28641 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.645s	user 0.008s	sys 0.003s
W20260709 12:24:36.143745 26047 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.022s	sys 0.034s
I20260709 12:24:36.200548 28600 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:36.201462 28641 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.201977 28600 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.202782 28617 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:35.626672 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72846.6us
W20260709 12:24:36.148862 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.001s	sys 0.021s
W20260709 12:24:36.206172 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.001s	sys 0.021s
W20260709 12:24:36.206812 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 1.00139e+06us
W20260709 12:24:35.666584 28609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.003s	sys 0.023s
W20260709 12:24:36.208930 28609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.213s	user 0.003s	sys 0.024s
I20260709 12:24:36.209440 28609 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:36.211048 28641 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:35.668520 28616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.803s	user 0.016s	sys 0.016s
W20260709 12:24:36.216919 28616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.351s	user 0.017s	sys 0.016s
I20260709 12:24:36.218755 26782 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:35.604400 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.049s	sys 0.127s
W20260709 12:24:35.609392 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.000s	sys 0.013s
W20260709 12:24:36.224772 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.269s	user 0.049s	sys 0.127s
W20260709 12:24:36.225059 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.000s	sys 0.013s
W20260709 12:24:36.160723 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.172215 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:35.565040 26784 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:24:36.228228 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 1.32217e+06us
I20260709 12:24:36.230233 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:36.231115 26167 connection.cc:256] RPC call timeout handler was delayed by -1.6157437020001453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260709 12:24:35.544899 26786 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:35.556348 26787 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:36.234815 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:36.235836 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.173734 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:36.184573 28312 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:36.239346 26785 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.239552 28312 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.243417 28312 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:36.185156 28485 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.017s	sys 0.013s
W20260709 12:24:36.244530 28485 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.017s	sys 0.013s
I20260709 12:24:36.186257 26197 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:36.190414 28562 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:35.600574 28595 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:36.227705 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 565) took 1701 ms. Trace:
I20260709 12:24:36.211544 28609 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.199332 28605 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:36.199927 28601 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.272s	user 0.034s	sys 0.068s
I20260709 12:24:36.244097 28312 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:36.249361 26783 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:36.249696 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1655) took 4904 ms (client timeout 1890 ms). Trace:
W20260709 12:24:36.201099 26047 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.023s	sys 0.034s
W20260709 12:24:36.165376 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.144382 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.240s	user 0.005s	sys 0.009s
W20260709 12:24:36.251605 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.251691 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.347s	user 0.005s	sys 0.010s
I20260709 12:24:36.247838 28601 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.250872 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:36.154820 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:36.255198 28721 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.272s	user 0.019s	sys 0.010s
I20260709 12:24:36.258708 28733 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:35.584327 28759 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:24:36.260962 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55084.6us
W20260709 12:24:36.263378 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54498.3us
I20260709 12:24:36.263881 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1188) took 3256 ms. Trace:
W20260709 12:24:36.264027 28749 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.005s	user 0.015s	sys 0.011s
I20260709 12:24:36.264807 28749 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.269917 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:36.268172 28659 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:24:36.265087 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:36.272941 28764 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:36.273932 28677 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:36.274633 28677 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:36.275720 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:36.276652 28732 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:36.276072 28677 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.264379 26047 rpcz_store.cc:276] 0709 12:24:33.007651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.559773 (+1552122us) service_pool.cc:224] Handling call
0709 12:24:34.560030 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:34.770764 (+210734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.771099 (+   335us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:34.778909 (+  7810us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.966917 (+188008us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:36.250733 (+1283816us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.251265 (+   532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.251403 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.263356 (+ 11953us) raft_consensus.cc:1685] finished
0709 12:24:36.263382 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.263752 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":247636,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":2508,"thread_start_us":1403523,"threads_started":3,"wal-append.queue_time_us":620814}
I20260709 12:24:36.277249 28732 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:36.282294 28732 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.271883 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:36.267868 28749 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:36.245865 26784 rpcz_store.cc:276] 0709 12:24:34.526532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.554517 (+ 27985us) service_pool.cc:224] Handling call
0709 12:24:34.680049 (+125532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.226867 (+1546818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.227433 (+   566us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:36.297806 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65839.9us
I20260709 12:24:36.247157 28605 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.250200 28312 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:36.250249 26125 rpcz_store.cc:269] 0709 12:24:31.345638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.345764 (+   126us) service_pool.cc:224] Handling call
0709 12:24:36.249085 (+4903321us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:36.249416 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2694875392}
I20260709 12:24:36.364068 26202 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:36.290869 28659 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:36.235976 26786 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:36.406041 28613 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.878s	user 0.022s	sys 0.004s
I20260709 12:24:36.416615 28562 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.416774 28732 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.418763 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:36.362071 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:36.412505 28769 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.310213 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 353) took 4455 ms. Trace:
I20260709 12:24:36.316383 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.361377 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.311966 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:36.315296 28551 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.395s	user 0.007s	sys 0.018s
I20260709 12:24:36.438620 28769 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.441462 28298 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.441917 28769 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.442641 28769 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.443315 28769 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:36.443740 28607 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.127s	user 0.005s	sys 0.020s
I20260709 12:24:36.448037 28769 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:36.449524 26788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 726) took 3536 ms (client timeout 1782 ms). Trace:
I20260709 12:24:36.450306 28732 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.408216 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:36.452873 26124 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.454017 28642 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:36.456637 26783 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 12:24:36.439272 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 167680us
I20260709 12:24:36.419869 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 490) took 1026 ms. Trace:
I20260709 12:24:36.442303 28298 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:36.449975 26788 rpcz_store.cc:269] 0709 12:24:32.871993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.407859 (+3535866us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:36.408067 (+   208us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:36.465257 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:36.466195 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 722) took 3596 ms. Trace:
I20260709 12:24:36.466866 26782 rpcz_store.cc:276] 0709 12:24:32.869744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.878498 (+  8754us) service_pool.cc:224] Handling call
0709 12:24:32.899894 (+ 21396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.219579 (+3319685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.465692 (+246113us) spinlock_profiling.cc:243] Waited 3.32 s on lock 0x7b54004ba854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.466057 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7630798592}
I20260709 12:24:36.468029 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.397604 28605 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.310679 28771 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:36.474175 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 244321us
I20260709 12:24:36.470165 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 485) took 1078 ms. Trace:
I20260709 12:24:36.490152 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:36.431823 28601 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.475705 26783 rpcz_store.cc:276] 0709 12:24:35.391715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.250412 (+858697us) service_pool.cc:224] Handling call
0709 12:24:36.470071 (+219659us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.500801 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 393) took 1015 ms. Trace:
I20260709 12:24:36.492179 26198 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:36.461040 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 488) took 1036 ms. Trace:
W20260709 12:24:36.506539 28637 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.945s	user 0.006s	sys 0.022s
I20260709 12:24:36.499190 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1167) took 4670 ms. Trace:
I20260709 12:24:36.473871 28771 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.539671 26054 rpcz_store.cc:276] 0709 12:24:31.829068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.214278 (+385210us) service_pool.cc:224] Handling call
0709 12:24:32.233788 (+ 19510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.757257 (+1523469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.498437 (+2741180us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.498950 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3447292160}
I20260709 12:24:36.462790 26784 rpcz_store.cc:276] 0709 12:24:35.393109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.418348 (+1025239us) service_pool.cc:224] Handling call
0709 12:24:36.419783 (+  1435us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:36.500205 26052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1152) took 5033 ms (client timeout 2905 ms). Trace:
I20260709 12:24:36.503398 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:36.525930 26197 rpcz_store.cc:276] 0709 12:24:35.485629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.500193 (+ 14564us) service_pool.cc:224] Handling call
0709 12:24:35.545602 (+ 45409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.245154 (+699552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.500446 (+255292us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400216bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.500691 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470622080}
W20260709 12:24:36.497056 26167 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.186s (SENT)
W20260709 12:24:36.524904 28702 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 3.495s	user 0.003s	sys 0.032s
I20260709 12:24:36.543360 28771 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.542714 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:36.498728 28312 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.496824 28600 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:36.555698 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:36.542109 26052 rpcz_store.cc:269] 0709 12:24:31.466647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.468025 (+  1378us) service_pool.cc:224] Handling call
0709 12:24:36.499676 (+5031651us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:36.499915 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3204791552}
I20260709 12:24:36.564893 26201 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:36.568683 28653 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.994s	user 0.009s	sys 0.018s
I20260709 12:24:36.568550 28764 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:36.569432 28653 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.583046 28653 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.561009 26045 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:36.600656 26787 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:36.601951 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 486) took 1209 ms. Trace:
I20260709 12:24:36.602424 26787 rpcz_store.cc:276] 0709 12:24:35.391996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.530395 (+138399us) service_pool.cc:224] Handling call
0709 12:24:36.601833 (+1071438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.605568 28587 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:36.606341 28587 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.603878 26200 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:36.426987 26122 rpcz_store.cc:276] 0709 12:24:31.792832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.793021 (+   189us) service_pool.cc:224] Handling call
0709 12:24:31.817493 (+ 24472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.944339 (+126846us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:32.218486 (+274147us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.729337 (+510851us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b18003127c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:24:32.780391 (+ 51054us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.792879 (+ 12488us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.793101 (+   222us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:32.793167 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.932370 (+139203us) spinlock_profiling.cc:243] Waited 7.31 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:24:36.247522 (+3315152us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54001c3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.247667 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.247744 (+    77us) raft_consensus.cc:1685] finished
0709 12:24:36.247755 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.248071 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52601,"prepare.queue_time_us":466992,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":52451,"raft.run_cpu_time_us":4030,"raft.run_wall_time_us":142756,"spinlock_wait_cycles":360616448,"thread_start_us":453462,"threads_started":2,"wal-append.queue_time_us":424270,"wal-append.run_cpu_time_us":3013,"wal-append.run_wall_time_us":1177792}
I20260709 12:24:36.501787 28693 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 12:24:36.486339 28551 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.617753 28551 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.619061 26194 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:36.620730 26197 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:36.621156 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.501288 28562 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:24:36.620859 28375 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:36.623099 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:36.623747 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 487) took 1231 ms. Trace:
I20260709 12:24:36.624703 28298 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:36.586117 28764 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:36.536540 26786 rpcz_store.cc:276] 0709 12:24:35.392534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.233905 (+841371us) service_pool.cc:224] Handling call
0709 12:24:36.425014 (+191109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.626385 28764 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.630503 28764 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:36.627400 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 286) took 1082 ms. Trace:
W20260709 12:24:36.634418 26018 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 3.197s (SENT)
I20260709 12:24:36.635059 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:36.637288 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 395) took 1096 ms. Trace:
I20260709 12:24:36.637719 26202 rpcz_store.cc:276] 0709 12:24:35.540768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.163484 (+622716us) service_pool.cc:224] Handling call
0709 12:24:36.164499 (+  1015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.397582 (+233083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.637016 (+239434us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400396f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.637209 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458107264}
I20260709 12:24:36.644495 26125 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.647058 28737 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:36.647729 28737 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:36.632262 26045 rpcz_store.cc:276] 0709 12:24:35.529695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.529893 (+   198us) service_pool.cc:224] Handling call
0709 12:24:36.502987 (+973094us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:36.611598 (+108611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4298880}
I20260709 12:24:36.661125 28312 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:36.624071 26785 rpcz_store.cc:276] 0709 12:24:35.392287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.568188 (+175901us) service_pool.cc:224] Handling call
0709 12:24:36.623667 (+1055479us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.664161 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.665307 26122 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:24:36.666862 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.672694 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.673825 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.675287 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.643918 28587 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.678936 28747 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.679105 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 240849us
I20260709 12:24:36.680044 28693 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.680753 28551 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:36.688426 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 372478us
I20260709 12:24:36.686326 28551 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:36.672346 28737 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:36.800457 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.787211 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60791.2us
W20260709 12:24:36.829737 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.741027 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.681417 28562 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:36.721606 28587 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.827476 28770 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.833781 28693 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.834208 28749 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.799010 26125 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:36.854269 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:36.836591 28649 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.332s	user 0.017s	sys 0.038s
W20260709 12:24:36.856614 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 222394us
I20260709 12:24:36.798377 28601 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:36.797183 28312 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.842553 28562 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:36.879943 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:36.899760 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44474.9us
I20260709 12:24:36.899590 26122 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:36.716562 28747 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:36.906047 26049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.031s	sys 0.041s
W20260709 12:24:36.910575 26049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.031s	sys 0.041s
I20260709 12:24:36.880118 28605 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.919951 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:36.921290 28763 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:36.895787 28749 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.889343 28770 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.886222 28649 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:36.898444 28771 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.931133 26124 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.891479 28693 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.939316 28562 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:36.940279 28756 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.943071 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.945225 28766 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.946574 28551 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:36.950263 26020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.019s	sys 0.026s
I20260709 12:24:36.953915 28749 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.955015 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 386) took 1561 ms. Trace:
I20260709 12:24:36.954574 28312 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:36.955763 28737 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:36.960378 28763 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:36.961645 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:36.949693 28605 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:36.964201 28770 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:36.965072 28747 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:36.964280 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1574) took 3161 ms. Trace:
I20260709 12:24:36.968397 28708 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.975343 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.007s	sys 0.023s
W20260709 12:24:36.998839 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.007s	sys 0.023s
W20260709 12:24:36.999185 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 682743us
W20260709 12:24:36.978879 26020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.019s	sys 0.027s
I20260709 12:24:36.985280 26194 rpcz_store.cc:276] 0709 12:24:35.393796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.393951 (+   155us) service_pool.cc:224] Handling call
0709 12:24:35.394350 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.619601 (+1225251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.954622 (+335021us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.954919 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2815309056}
I20260709 12:24:36.993163 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:36.927517 28601 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:36.990001 26018 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.666s (SENT)
W20260709 12:24:37.007805 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 851150us
I20260709 12:24:36.996903 26195 rpcz_store.cc:276] 0709 12:24:33.803029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.911616 (+108587us) service_pool.cc:224] Handling call
0709 12:24:33.912036 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.898600 (+986564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.963727 (+2065127us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b5400301954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:36.964046 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2136632064}
I20260709 12:24:36.993840 28766 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.030953 28741 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:37.032876 28766 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:37.033951 28562 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:37.036821 28771 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:37.037761 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 103135us
W20260709 12:24:37.040344 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.013s	sys 0.032s
W20260709 12:24:37.040740 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.014s	sys 0.033s
W20260709 12:24:37.040999 26756 connection.cc:256] RPC call timeout handler was delayed by -2.5305786540002373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 12:24:37.041332 26756 connection.cc:256] RPC call timeout handler was delayed by -2.516178654000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
I20260709 12:24:37.041379 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:37.041697 26756 connection.cc:256] RPC call timeout handler was delayed by -2.3649786539999695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:24:37.042790 26019 connection.cc:256] RPC call timeout handler was delayed by -1.700231776000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2676s for a response
W20260709 12:24:37.043253 26019 connection.cc:256] RPC call timeout handler was delayed by -1.619305630999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:24:37.043560 26019 connection.cc:256] RPC call timeout handler was delayed by -1.5150918389999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2978s for a response
W20260709 12:24:37.043915 26019 connection.cc:256] RPC call timeout handler was delayed by -1.3347437789998366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 12:24:37.035138 26785 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:37.045686 26791 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEARNER]: Advancing to term 5
I20260709 12:24:37.047793 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:37.048715 26790 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:37.049925 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71386.7us
I20260709 12:24:37.052507 28708 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.053112 26129 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.054301 28747 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:37.054934 28708 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:37.056528 26125 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:37.057201 28601 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:37.043699 28756 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.055006 28747 consensus_queue.cc:1243] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:37.063421 26124 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
W20260709 12:24:37.064265 26046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.386s	user 0.020s	sys 0.065s
W20260709 12:24:37.064718 26046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.531s	user 0.020s	sys 0.066s
I20260709 12:24:37.065033 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:37.065621 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 285) took 1597 ms. Trace:
I20260709 12:24:37.065806 26050 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:37.066044 26046 rpcz_store.cc:276] 0709 12:24:35.468155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.470471 (+  2316us) service_pool.cc:224] Handling call
0709 12:24:35.497947 (+ 27476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:35.503491 (+  5544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.503522 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:35.504095 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.065155 (+1561060us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:37.065246 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.065343 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.065385 (+    42us) raft_consensus.cc:1685] finished
0709 12:24:37.065398 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.065568 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5967,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":3670,"thread_start_us":1531662,"threads_started":1,"wal-append.queue_time_us":1418937}
I20260709 12:24:37.068184 28649 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:36.994123 28749 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.099680 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.102058 26124 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:37.102548 26791 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:37.078788 28708 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.109619 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:37.109483 28708 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:37.111696 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:37.112128 28708 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:37.106773 28551 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:37.125626 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.145262 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:37.141323 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:37.138554 28659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.000s	sys 0.031s
I20260709 12:24:37.158342 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:37.158020 28677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.039s	sys 0.038s
W20260709 12:24:37.158167 28659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.000s	sys 0.032s
I20260709 12:24:37.142129 28708 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.169716 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:37.171041 26046 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:37.165337 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.171890 26194 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:37.148929 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.156827 28312 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:37.163070 28607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.014s	sys 0.027s
I20260709 12:24:37.172879 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.173981 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.170630 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.182801 26124 raft_consensus.cc:2368] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:37.194633 26201 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
W20260709 12:24:37.165441 28609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.024s	sys 0.025s
I20260709 12:24:37.179998 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:37.177244 26790 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:37.184988 28748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.013s	sys 0.023s
W20260709 12:24:37.170437 28677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.039s	sys 0.039s
I20260709 12:24:37.184904 28708 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:37.185956 28606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.023s	sys 0.006s
W20260709 12:24:37.186239 28724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.009s	sys 0.026s
W20260709 12:24:37.192404 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80877.8us
W20260709 12:24:37.195273 28607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.014s	sys 0.027s
I20260709 12:24:37.192824 26045 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:37.185284 26050 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
W20260709 12:24:37.199606 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47841.3us
I20260709 12:24:37.173252 28659 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:37.192459 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:37.216588 26054 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:37.217968 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:37.202908 28748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.013s	sys 0.024s
I20260709 12:24:37.233851 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.158566 28749 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.237787 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:37.204389 28609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.025s	sys 0.025s
I20260709 12:24:37.252338 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.236778 26195 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:37.252882 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:37.224872 28606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.023s	sys 0.007s
W20260709 12:24:37.233861 28724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.009s	sys 0.027s
I20260709 12:24:37.244817 26789 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
W20260709 12:24:37.245651 28764 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.007s
I20260709 12:24:37.258155 26201 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.258702 26125 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.237198 28312 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.263114 26046 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:37.225221 28693 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:37.276458 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:37.294469 26123 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:37.296301 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.259472 28747 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:37.293078 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:37.304364 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.305470 28749 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:37.305828 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:37.306619 26046 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:37.308145 26783 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.259464 28770 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.313136 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:37.289527 28710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.899s	user 0.027s	sys 0.038s
I20260709 12:24:37.264012 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:37.318156 28714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.012s	sys 0.046s
W20260709 12:24:37.318002 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56485.6us
I20260709 12:24:37.320861 28479 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.321995 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:37.330754 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:37.317070 28710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.091s	user 0.028s	sys 0.039s
I20260709 12:24:37.321002 26195 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:24:37.335322 26757 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.005s	sys 0.026s
W20260709 12:24:37.336359 26757 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.005s	sys 0.026s
W20260709 12:24:37.336663 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 502546us
W20260709 12:24:37.325302 28714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.012s	sys 0.047s
I20260709 12:24:37.337882 28380 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
W20260709 12:24:37.338830 28275 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.028s	sys 0.032s
I20260709 12:24:37.338660 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.280658 28312 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.339311 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:37.343833 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.345598 26124 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.342525 26203 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:37.321473 28747 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:37.347051 26201 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:37.357026 26125 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:37.343803 28275 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.028s	sys 0.032s
I20260709 12:24:37.330022 28756 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:37.397239 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.398095 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0539679850003267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:37.339439 28763 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:37.416600 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.359284 26050 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.420644 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:37.427356 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.429786 28714 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.431520 28609 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.341461 28770 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.436859 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.437359 28693 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.438020 26199 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:37.439128 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.440178 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.440949 26203 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:37.441126 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.441804 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.444460 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.444618 28708 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:37.444834 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.445623 28747 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.446435 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.356698 28749 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.342207 28710 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:37.420754 28479 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.480798 26121 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
W20260709 12:24:37.486732 26094 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.010s	sys 0.035s
I20260709 12:24:37.500394 26783 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:37.504469 26202 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:37.519474 26094 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.010s	sys 0.035s
W20260709 12:24:37.520282 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 858012us
I20260709 12:24:37.479103 28601 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:37.527069 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 162715us
I20260709 12:24:37.346720 28659 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.532680 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.527275 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.545676 26125 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:37.453109 28796 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:24:37.545221 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:37.550659 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.013s	sys 0.034s
I20260709 12:24:37.553709 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.522707 28708 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.426095 28785 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
W20260709 12:24:37.554774 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.014s	sys 0.034s
I20260709 12:24:37.532624 28275 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:37.533217 28693 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.578903 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:37.532526 28749 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.585379 26202 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.586958 26124 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:37.572681 28275 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.527773 28609 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:37.593025 26018 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.537s (SENT)
I20260709 12:24:37.593559 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:37.577880 28708 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:37.624995 28700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.016s	sys 0.029s
W20260709 12:24:37.625470 28700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.016s	sys 0.029s
I20260709 12:24:37.626013 28700 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:37.626217 28708 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:37.626639 28700 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:37.629288 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52746.2us
I20260709 12:24:37.574303 28601 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.592856 28275 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.632748 28796 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:37.640450 28788 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:24:37.640707 28796 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:37.641669 28794 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:24:37.642925 28791 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:24:37.580561 28714 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:37.646166 28714 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:37.653503 26199 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:37.656322 26046 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:37.652946 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:37.634408 28562 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.000s
I20260709 12:24:37.674768 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.662281 26784 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:37.662477 28770 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:37.663273 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.664059 28479 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:37.664798 26094 connection.cc:256] RPC call timeout handler was delayed by -2.80024038800002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 12:24:37.674674 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.624848 28710 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.680203 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.641772 28796 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.682057 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:37.692873 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60244.6us
I20260709 12:24:37.693698 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:37.696524 26094 connection.cc:256] RPC call timeout handler was delayed by -2.1888851310000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2785s for a response
W20260709 12:24:37.697091 26094 connection.cc:256] RPC call timeout handler was delayed by -2.0833066580003106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 12:24:37.697482 26094 connection.cc:256] RPC call timeout handler was delayed by -2.0491066580002553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:24:37.697856 26094 connection.cc:256] RPC call timeout handler was delayed by -1.9960066580001694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:24:37.698203 26094 connection.cc:256] RPC call timeout handler was delayed by -1.9771066580001389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
I20260709 12:24:37.681105 28749 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:37.682235 26120 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 12:24:37.700295 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.688947 28700 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:37.701370 28700 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:37.701610 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.701892 26197 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:37.702466 28700 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:37.702850 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.703047 28700 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.703532 28700 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:37.698557 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5288791869998022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 12:24:37.705340 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5288791869998022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 12:24:37.703560 26199 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:37.711475 26202 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:37.713387 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.720144 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:37.723467 28748 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.724710 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.727234 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:37.727635 28797 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:24:37.723955 28748 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:37.735842 26202 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
W20260709 12:24:37.737820 26791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.003s	sys 0.024s
W20260709 12:24:37.738274 26791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.003s	sys 0.024s
I20260709 12:24:37.737941 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.703469 28609 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:37.741044 26194 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.014s	sys 0.047s
W20260709 12:24:37.741765 26194 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.015s	sys 0.048s
W20260709 12:24:37.742290 26049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.004s	sys 0.023s
I20260709 12:24:37.701694 28601 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:37.742739 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.744541 26120 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:37.744813 26049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.004s	sys 0.023s
I20260709 12:24:37.745380 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.744837 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.745975 26121 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:37.728184 28764 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:37.754212 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 170992us
I20260709 12:24:37.734043 28748 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.758044 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.764302 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.765175 26199 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:37.810186 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47473.1us
I20260709 12:24:37.812273 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.811107 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 289) took 2236 ms. Trace:
I20260709 12:24:37.817869 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.820709 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:37.824416 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 495835us
I20260709 12:24:37.824828 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:37.761171 28708 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:24:37.828670 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101589us
I20260709 12:24:37.833241 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.849599 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.850419 26199 raft_consensus.cc:2376] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7. Re-sending same reply.
I20260709 12:24:37.847538 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:37.854188 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.815356 26049 rpcz_store.cc:276] 0709 12:24:35.537461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.548908 (+ 11447us) service_pool.cc:224] Handling call
0709 12:24:35.564273 (+ 15365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:35.564737 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.564778 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:35.569299 (+  4521us) log.cc:844] Serialized 52 byte log entry
0709 12:24:36.986299 (+1417000us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:37.033168 (+ 46869us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.745676 (+712508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.761007 (+ 15331us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.761065 (+    58us) raft_consensus.cc:1685] finished
0709 12:24:37.763869 (+  2804us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.773465 (+  9596us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43065,"prepare.queue_time_us":45361,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":137483,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":3666,"thread_start_us":1397123,"threads_started":2,"wal-append.queue_time_us":856383}
I20260709 12:24:37.832669 28764 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:37.866664 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.869005 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.869973 26121 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:37.870944 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.871568 26127 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:37.881093 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:37.887534 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:37.868563 28764 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:37.916700 26202 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:37.876260 28601 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:37.861572 28708 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.919747 26050 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:37.938455 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:37.940021 28801 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260709 12:24:37.943440 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:37.944942 28312 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:37.945299 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:37.936715 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:37.957139 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 95245us
I20260709 12:24:37.953991 26127 raft_consensus.cc:2376] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7. Re-sending same reply.
I20260709 12:24:37.957719 26196 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:37.924711 28693 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:37.960516 26788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.016s
W20260709 12:24:37.960935 26788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.000s	sys 0.016s
I20260709 12:24:37.961941 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:37.962308 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 268744us
I20260709 12:24:37.964475 26120 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:24:37.965349 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.953322 28601 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.978435 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:37.995174 28811 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:24:37.978554 28601 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.994990 28764 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:37.996632 26122 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:38.007561 28806 raft_consensus.cc:2764] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:38.012645 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:38.012907 28807 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:38.014393 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:38.015328 28606 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:38.016270 28380 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:38.016414 28795 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.018707 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:38.019580 26202 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:38.020437 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:38.028702 26202 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:38.030149 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:38.031622 26202 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:38.033432 28601 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.046231 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.046085 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.046590 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.061973 26129 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:38.038838 26203 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:38.048712 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:38.071547 26167 leader_election.cc:336] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.837s (SENT)
I20260709 12:24:38.049274 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.034371 26197 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:38.036155 28601 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:38.049755 28785 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.081650 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.082640 26127 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:38.082137 28785 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.036046 28724 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:38.102800 26199 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:38.108911 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.111486 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.070523 28756 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.109912 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56237.9us
I20260709 12:24:38.114369 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.117599 28601 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:38.119829 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.120893 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.123363 26121 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:38.121093 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.079952 28708 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:38.137967 26120 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:24:38.138496 26201 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.006s	sys 0.029s
W20260709 12:24:38.138849 26201 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.006s	sys 0.029s
W20260709 12:24:38.139479 26020 connection.cc:256] RPC call timeout handler was delayed by -1.3859077470001466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2676s for a response
W20260709 12:24:38.139855 26020 connection.cc:256] RPC call timeout handler was delayed by -1.342831417999605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 12:24:38.154042 26020 connection.cc:256] RPC call timeout handler was delayed by -1.3040077470000142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2767s for a response
I20260709 12:24:38.155561 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:38.106899 28724 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:38.167651 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 96136us
I20260709 12:24:38.167654 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:38.169767 26195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.017s	sys 0.016s
W20260709 12:24:38.170212 26195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.017s	sys 0.016s
W20260709 12:24:38.183737 26167 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.889s (SENT)
I20260709 12:24:38.162457 28714 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.191496 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:38.200860 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 577) took 1043 ms. Trace:
I20260709 12:24:38.201664 26787 rpcz_store.cc:276] 0709 12:24:37.145052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.147085 (+  2033us) service_pool.cc:224] Handling call
0709 12:24:37.176338 (+ 29253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.191047 (+ 14709us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:37.191585 (+   538us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.184658 (+993073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.187874 (+  3216us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:38.187920 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.187982 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.188094 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2355,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":4138,"raft.queue_time_us":366082,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":949240,"threads_started":2,"wal-append.queue_time_us":651696}
I20260709 12:24:38.165436 28609 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:38.213608 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:38.180464 28275 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.005s
W20260709 12:24:38.218221 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.215237 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 575) took 1089 ms. Trace:
I20260709 12:24:38.140758 28756 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.218746 26791 rpcz_store.cc:276] 0709 12:24:37.125573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.125709 (+   136us) service_pool.cc:224] Handling call
0709 12:24:37.145049 (+ 19340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.183495 (+ 38446us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:37.183832 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.214813 (+1030981us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.214876 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:38.214934 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.215017 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.215164 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1008,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":310721,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":1003557,"threads_started":2,"wal-append.queue_time_us":684283}
I20260709 12:24:38.157007 28601 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.157559 28607 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:24:38.200131 28714 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.207461 26785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.021s	sys 0.025s
W20260709 12:24:38.249114 26020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 2.083s (SENT) [suppressed 41 similar messages]
W20260709 12:24:38.258723 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75042.1us
W20260709 12:24:38.260311 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68934.5us
I20260709 12:24:38.261092 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:38.200372 28700 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:38.254760 26785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.022s	sys 0.025s
I20260709 12:24:38.276247 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:38.254637 28714 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.278357 26129 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.206081 28677 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:24:38.213416 28710 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:24:38.279337 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.273487 26127 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:38.293421 26127 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:38.237392 28275 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:38.295909 26020 connection.cc:256] RPC call timeout handler was delayed by -1.2744314179994944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 12:24:38.297974 26020 connection.cc:256] RPC call timeout handler was delayed by -1.2159759489995849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 12:24:38.296797 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:38.314478 26020 connection.cc:256] RPC call timeout handler was delayed by -1.1141077470001619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2978s for a response
I20260709 12:24:38.318117 26126 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:38.301047 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1177) took 1119 ms. Trace:
W20260709 12:24:38.302296 26125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.020s
I20260709 12:24:38.309374 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:38.299305 28649 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.310317 28375 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:38.301138 28785 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:38.284869 28606 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:24:38.330531 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:38.277809 28312 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:24:38.302349 28609 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.281956 28659 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:24:38.310750 28823 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
W20260709 12:24:38.336169 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52145.1us
I20260709 12:24:38.341053 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:38.342057 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.344251 28551 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:38.390511 28804 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.501s	user 0.005s	sys 0.000s
I20260709 12:24:38.390843 28551 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.391247 28700 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:38.391395 28551 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.393755 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2375) took 1937 ms. Trace:
I20260709 12:24:38.394460 26128 rpcz_store.cc:276] 0709 12:24:36.455842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.467381 (+ 11539us) service_pool.cc:224] Handling call
0709 12:24:38.393375 (+1925994us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:38.393638 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4102770944}
I20260709 12:24:38.344959 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.352603 28821 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:38.353508 28312 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.354879 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:38.403748 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48987.3us
I20260709 12:24:38.325748 28649 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.326001 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 573) took 1290 ms. Trace:
W20260709 12:24:38.412470 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84144.8us
I20260709 12:24:38.325484 26127 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.413792 26127 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:38.413815 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72866.6us
I20260709 12:24:38.358357 28814 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:38.361207 28790 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:38.365136 28819 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:38.343298 28785 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.386893 26129 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:38.388077 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:38.388325 28812 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:38.389657 28816 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:38.389537 28606 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:38.320019 26194 rpcz_store.cc:276] 0709 12:24:37.181537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.181741 (+   204us) service_pool.cc:224] Handling call
0709 12:24:37.184054 (+  2313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.184520 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:37.750057 (+565537us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.300521 (+550464us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.300634 (+   113us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:38.300710 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.300802 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.300962 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33114,"prepare.queue_time_us":679963,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":6078,"raft.queue_time_us":5768,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9140,"thread_start_us":1037279,"threads_started":2}
W20260709 12:24:38.421312 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76526.7us
W20260709 12:24:38.417563 26124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.032s	sys 0.032s
W20260709 12:24:38.427265 26124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.033s	sys 0.032s
I20260709 12:24:38.427549 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:38.436738 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 508256us
I20260709 12:24:38.442386 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:38.283535 28724 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:38.449494 26018 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.801s (SENT)
I20260709 12:24:38.412735 28649 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.413045 26785 rpcz_store.cc:276] 0709 12:24:37.033540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.033875 (+   335us) service_pool.cc:224] Handling call
0709 12:24:37.034387 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.034843 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:37.519067 (+484224us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.286241 (+767174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.296950 (+ 10709us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:38.301042 (+  4092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.313022 (+ 11980us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.324193 (+ 11171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26682,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":695921,"threads_started":1,"wal-append.queue_time_us":829161}
W20260709 12:24:38.460467 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.463457 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.415652 28601 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:38.425527 28785 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.473780 26126 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:38.474004 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.474293 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.474825 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:38.473347 26198 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.018s	sys 0.057s
W20260709 12:24:38.476958 26198 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.018s	sys 0.057s
I20260709 12:24:38.417280 28790 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:38.442709 28637 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:38.506907 28637 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.507303 28637 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.507422 28312 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.507840 28810 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:38.508870 28810 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.509431 28810 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.510332 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.511761 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:38.511890 28800 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:38.513592 28829 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:24:38.486133 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.489713 28825 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.486907 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.516867 28825 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:38.489878 28708 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:38.518036 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:38.519728 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 131438us
I20260709 12:24:38.516860 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.492134 28764 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.492678 28790 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.493920 28375 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:38.494601 28814 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:38.505231 26782 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:38.506451 28813 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:38.513639 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.477213 28785 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.556162 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1188) took 1122 ms. Trace:
I20260709 12:24:38.446439 28693 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:24:38.548987 26127 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:24:38.476017 28747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.024s	sys 0.051s
I20260709 12:24:38.518092 28700 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:38.555747 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 126447us
W20260709 12:24:38.559635 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 643604us
I20260709 12:24:38.560930 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1185) took 1127 ms. Trace:
W20260709 12:24:38.544025 28815 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:24:38.557217 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 131313us
I20260709 12:24:38.511787 28810 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:38.544418 28375 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.581815 26126 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:38.582651 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.527755 28601 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:38.529146 28764 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:38.585419 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.586143 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:38.544159 28814 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.508002 28312 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:38.605382 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.607218 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 179774us
I20260709 12:24:38.611320 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:38.578051 28747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.024s	sys 0.051s
I20260709 12:24:38.616956 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.617746 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.579027 26200 rpcz_store.cc:276] 0709 12:24:37.433041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.434704 (+  1663us) service_pool.cc:224] Handling call
0709 12:24:37.435416 (+   712us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.482272 (+ 46856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.482323 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.546414 (+ 64091us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.967958 (+421544us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:37.967995 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.968059 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.560657 (+592598us) raft_consensus.cc:1685] finished
0709 12:24:38.560688 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.560854 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12526,"raft.queue_time_us":43359,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":45860,"thread_start_us":367551,"threads_started":1,"wal-append.queue_time_us":698072}
I20260709 12:24:38.635177 28801 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.635741 28801 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.636878 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:38.637957 28801 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:38.638509 28801 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.638983 28801 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.640612 26122 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.639060 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:38.616891 28748 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.004s
I20260709 12:24:38.556746 26198 rpcz_store.cc:276] 0709 12:24:37.434130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.445284 (+ 11154us) service_pool.cc:224] Handling call
0709 12:24:37.445677 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.477650 (+ 31973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.477683 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.481039 (+  3356us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.878011 (+396972us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:38.046377 (+168366us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b48000b1b58. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:24:38.478315 (+431938us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.478519 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.478595 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.521395 (+ 42800us) raft_consensus.cc:1685] finished
0709 12:24:38.555855 (+ 34460us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.556077 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2527,"prepare.queue_time_us":59682,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":33827,"raft.run_cpu_time_us":9991,"raft.run_wall_time_us":565636,"spinlock_wait_cycles":43877120,"thread_start_us":866750,"threads_started":2,"wal-append.queue_time_us":714070}
I20260709 12:24:38.643324 26783 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:24:38.646116 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 318861us
I20260709 12:24:38.605795 28601 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.620546 28756 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:38.620388 28813 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:38.683127 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:38.699330 26199 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:38.694732 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1183) took 1278 ms. Trace:
W20260709 12:24:38.698472 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 112421us
I20260709 12:24:38.702572 26203 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:38.640084 28275 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:38.710006 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:38.711686 26197 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.710719 26194 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:38.715647 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:38.717020 26200 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:38.720845 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:38.654625 28601 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.663511 28726 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.731580 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:38.731930 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 114689us
I20260709 12:24:38.732770 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:38.733503 26054 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:38.690953 28659 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:38.738715 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:38.739358 26050 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.700567 26201 rpcz_store.cc:276] 0709 12:24:37.416884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.417305 (+   421us) service_pool.cc:224] Handling call
0709 12:24:37.485473 (+ 68168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.535262 (+ 49789us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:38.139178 (+603916us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.645347 (+506169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.645404 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:38.694105 (+ 48701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.694390 (+   285us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.694636 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":369591,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":4928,"raft.queue_time_us":105763,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":1042353,"threads_started":3,"wal-append.queue_time_us":484266}
I20260709 12:24:38.744436 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:38.743418 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:38.715723 28806 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:38.745944 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.645375 28800 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:38.698089 28479 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.627774 28825 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.718050 28756 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:38.753582 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 585) took 1569 ms (client timeout 1516 ms). Trace:
I20260709 12:24:38.730967 25949 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:38.748111 26045 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:38.713541 28812 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:38.726424 28609 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.654413 28748 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.783653 28756 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.785549 28827 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 12:24:38.786906 26053 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:38.738508 28726 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:38.771533 26790 rpcz_store.cc:269] 0709 12:24:37.184427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.264439 (+ 80012us) service_pool.cc:224] Handling call
0709 12:24:38.753341 (+1488902us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54005ce254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:38.753496 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200381696}
I20260709 12:24:38.825846 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.828197 26050 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:38.794536 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83127.4us
I20260709 12:24:38.811213 26202 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:24:38.824466 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 119120us
I20260709 12:24:38.771832 26049 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:38.829346 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.750352 28700 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:38.758298 28693 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:38.850615 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91694.4us
I20260709 12:24:38.831722 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:38.831465 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:38.852947 26199 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:38.830832 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113297us
I20260709 12:24:38.837507 26126 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 12:24:38.819008 28756 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:38.788879 28828 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:24:38.837610 26127 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:38.832224 28825 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.859594 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.734443 28764 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.819909 28821 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:38.865686 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.867697 26122 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.750579 28806 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.811389 28312 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:38.809269 28813 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:38.886162 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.888643 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:39.264604 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 653545us
I20260709 12:24:39.146479 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 368) took 1493 ms. Trace:
I20260709 12:24:39.041486 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 362) took 1411 ms. Trace:
I20260709 12:24:38.933146 26054 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:39.267942 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:38.953029 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79118.3us
I20260709 12:24:39.040886 28849 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.269966 28849 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.270480 28849 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.270699 28801 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:39.271479 28801 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:39.272094 28801 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:39.273346 28801 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.276624 28609 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:39.277436 28609 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.124835 28848 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.281385 28609 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:38.935576 28837 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 12:24:39.282430 28790 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.864272 28693 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:39.285095 28693 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.286638 25949 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.288630 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.289186 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.289820 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.290243 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.289186 25949 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 5, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:39.290755 26045 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:39.078678 28835 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.247s
I20260709 12:24:39.292400 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.285012 28724 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:39.293147 25949 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 4, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.293996 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.294936 26045 raft_consensus.cc:2410] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:38.869114 28747 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.436s
I20260709 12:24:39.296666 28747 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:39.296780 26045 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:39.297111 28747 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:39.188750 28380 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:39.298734 28380 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:39.299518 28380 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:38.931785 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.301689 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.290828 26049 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:39.302388 26050 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:39.186708 26194 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:39.294864 25949 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 5, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 5 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.266163 26125 rpcz_store.cc:276] 0709 12:24:37.652691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.686089 (+ 33398us) service_pool.cc:224] Handling call
0709 12:24:37.693144 (+  7055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.730580 (+ 37436us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:37.730615 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.742472 (+ 11857us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.325481 (+583009us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:38.325531 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.325610 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.146142 (+820532us) raft_consensus.cc:1685] finished
0709 12:24:39.146171 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.146391 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12056,"raft.queue_time_us":23191,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4817,"thread_start_us":548986,"threads_started":1,"wal-append.queue_time_us":1245299}
I20260709 12:24:39.266311 26124 rpcz_store.cc:276] 0709 12:24:37.632097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.633679 (+  1582us) service_pool.cc:224] Handling call
0709 12:24:37.672451 (+ 38772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.748938 (+ 76487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.748982 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.760988 (+ 12006us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.427663 (+666675us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:38.427702 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.427788 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.041205 (+613417us) raft_consensus.cc:1685] finished
0709 12:24:39.041230 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.041395 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35420,"raft.queue_time_us":10310,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11782,"thread_start_us":590711,"threads_started":1,"wal-append.queue_time_us":745354}
W20260709 12:24:39.215648 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.162842 28850 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.146s
I20260709 12:24:39.308393 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:39.311916 26045 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:39.313697 26045 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:39.314980 28849 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.315630 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.315986 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.316416 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.316817 26052 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
I20260709 12:24:39.316448 26049 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:39.075924 28846 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:39.317999 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.318275 28849 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:39.005982 26047 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:39.321177 28312 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:39.322257 26197 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.000s
I20260709 12:24:39.322698 26197 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:39.318231 28806 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.319694 26050 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:38.894824 28637 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:39.323410 26018 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.973s (SENT)
I20260709 12:24:39.324702 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.326547 28637 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:39.327291 28847 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:39.251628 28375 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.000s
W20260709 12:24:39.026561 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 163790us
I20260709 12:24:38.933933 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:38.934995 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:39.245383 28714 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.930s	user 0.007s	sys 0.006s
I20260709 12:24:38.888922 28726 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:39.335364 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:38.936646 28823 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.435s
I20260709 12:24:39.335520 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.298094 28747 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:39.337369 28649 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.002s
W20260709 12:24:38.935067 28838 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:24:39.355345 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:39.340628 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 475149us
I20260709 12:24:39.345264 25949 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:39.347810 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.040717 28839 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 12:24:39.329255 28375 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.245086 28748 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:39.395299 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 536451us
I20260709 12:24:39.333125 28609 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:24:39.411002 28649 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:38.933687 28800 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.427022 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:39.451072 26050 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.171885 28479 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:39.373526 28380 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:39.412698 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 584) took 2228 ms. Trace:
I20260709 12:24:39.417888 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:39.458439 26047 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:39.040361 28831 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
W20260709 12:24:39.370668 28814 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.741s	user 0.007s	sys 0.002s
W20260709 12:24:39.354512 28601 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.609s	user 0.005s	sys 0.005s
I20260709 12:24:39.353860 28275 raft_consensus.cc:1064] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:24:39.439050 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 504081us
I20260709 12:24:39.480552 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:39.409919 28785 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.822s	user 0.005s	sys 0.006s
I20260709 12:24:39.348491 28764 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.001132 28737 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.409500 28821 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:39.443408 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:39.379962 28846 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:39.454398 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:39.494553 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.409322 28848 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.459900 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.412621 28700 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:39.447391 28796 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.348734 28850 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:24:39.525713 26790 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:39.478107 28747 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.466065 25949 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:39.487587 28275 raft_consensus.cc:1068] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:39.456422 28659 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.469336 28726 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.471638 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:39.454167 28312 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:39.554949 26782 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.555003 28312 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:39.555595 28312 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:39.558106 26167 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.665s (SENT)
W20260709 12:24:39.559049 26167 leader_election.cc:336] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.703s (SENT)
I20260709 12:24:39.493930 28637 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.559715 26120 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:39.559703 28637 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.536149 25949 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 5, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 5 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.561631 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1690) took 1106 ms. Trace:
I20260709 12:24:39.458256 26788 rpcz_store.cc:276] 0709 12:24:37.184031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.288430 (+104399us) service_pool.cc:224] Handling call
0709 12:24:37.317356 (+ 28926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.974330 (+656974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.974371 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.993968 (+ 19597us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.441756 (+447788us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:38.441811 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.441909 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.412467 (+970558us) raft_consensus.cc:1685] finished
0709 12:24:39.412487 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.412635 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17417,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":34082,"thread_start_us":1033434,"threads_started":2,"wal-append.queue_time_us":589116}
I20260709 12:24:39.562119 25949 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 3, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 3 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.474188 26128 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:39.564154 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1684) took 1110 ms. Trace:
I20260709 12:24:39.564538 26128 rpcz_store.cc:276] 0709 12:24:38.453638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.453784 (+   146us) service_pool.cc:224] Handling call
0709 12:24:39.563989 (+1110205us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b5400369cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:39.564101 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2298619776}
I20260709 12:24:39.566160 28312 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.568673 28312 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:39.569491 28312 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.569900 26784 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:24:39.570338 26784 pending_rounds.cc:85] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 8
I20260709 12:24:39.570688 26784 pending_rounds.cc:107] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:24:39.571067 26784 raft_consensus.cc:2889] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:39.571184 26782 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:39.589751 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:39.491310 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.022384 28795 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.415s
I20260709 12:24:39.432186 28843 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:24:39.493046 26125 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.595249 28312 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.529824 28844 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:39.596581 28844 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:39.597865 28844 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.599715 28844 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.125061 28812 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.438215 28606 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:24:39.530325 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 194871us
I20260709 12:24:39.504927 28848 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:39.609637 28848 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.477717 28814 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.440397 26198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:39.622159 26198 raft_consensus.cc:606] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:39.622578 26198 raft_consensus.cc:614] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:39.536347 28659 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.537473 28649 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:39.540325 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.540652 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.541182 28821 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.495270 28748 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:39.503839 28601 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.428365 28714 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.554158 28609 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:39.543903 28375 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:39.561970 26120 rpcz_store.cc:276] 0709 12:24:38.455633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.511474 (+ 55841us) service_pool.cc:224] Handling call
0709 12:24:39.561407 (+1049933us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002ca1d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:39.561563 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391100672}
I20260709 12:24:39.526799 28796 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:39.491405 28785 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:39.581115 28756 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.001s
W20260709 12:24:39.582073 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87586.9us
W20260709 12:24:39.583032 25950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 274) took 17584 ms (client timeout 15000 ms). Trace:
W20260709 12:24:39.584079 28551 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.192s	user 0.014s	sys 0.000s
I20260709 12:24:39.563787 25949 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 5, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 5 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:39.559895 26167 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.069s (SENT)
I20260709 12:24:39.465129 28479 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.532447 28849 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:39.567458 28710 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.589105 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:39.483170 28847 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:24:39.494043 28851 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:39.589461 28840 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:39.592733 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109347us
I20260709 12:24:39.593645 28795 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:39.524504 28700 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.491751 28737 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.604859 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:39.688462 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 83666.2us
I20260709 12:24:39.608713 26125 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:39.562147 28637 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:39.647218 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 274564us
W20260709 12:24:39.652076 26018 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.991s (SENT)
I20260709 12:24:39.662915 26195 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 7. Ops: []
I20260709 12:24:39.661707 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:39.669602 26050 raft_consensus.cc:2410] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:39.671883 26128 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.638132 28748 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:39.637529 28609 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:39.692250 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46545.6us
I20260709 12:24:39.673027 26785 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:39.657239 28796 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:39.717895 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 157998us
I20260709 12:24:39.685890 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.697101 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:39.699007 28823 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:39.699903 28275 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:39.702764 28714 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:39.703454 28852 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.703938 28850 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:39.707914 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1193) took 1029 ms. Trace:
I20260709 12:24:39.743232 26195 rpcz_store.cc:276] 0709 12:24:38.678716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.678832 (+   116us) service_pool.cc:224] Handling call
0709 12:24:38.710293 (+ 31461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.691866 (+981573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.707669 (+ 15803us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b5400396554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:39.707853 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190326016}
W20260709 12:24:39.708778 26018 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.020s (SENT)
I20260709 12:24:39.743613 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:39.744526 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51344us
W20260709 12:24:39.745045 26018 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.057s (SENT)
I20260709 12:24:39.735302 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:39.745537 26018 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.151s (SENT)
I20260709 12:24:39.745908 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.724977 25949 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 3, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 3 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.665017 28649 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.671478 28849 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:39.762768 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91873.1us
I20260709 12:24:39.690307 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:39.725735 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 372804us
W20260709 12:24:39.737643 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.764899 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.747406 25949 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 4, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.657581 28724 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:24:39.716082 26124 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.691572 28700 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.712162 26128 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:39.714821 28737 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.680588 28795 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:39.768030 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:39.769163 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78944.2us
I20260709 12:24:39.725185 28860 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:39.715685 28748 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.732410 28637 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:39.665069 28853 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 12:24:39.670930 28551 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.666675 28756 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.781179 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.792500 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.742502 28852 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.782121 25949 catalog_manager.cc:5697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 2, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.804625 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.761341 28847 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.768014 28849 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.834695 26047 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:39.836373 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:39.814990 26124 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:39.816830 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:39.817050 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.838426 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.819164 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47587.9us
I20260709 12:24:39.820631 28785 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:39.821106 28714 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.808647 28693 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 12:24:39.822839 28859 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:39.791726 28823 consensus_queue.cc:1243] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:39.809212 28609 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.825176 28814 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:39.783885 28860 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:39.828902 28795 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:39.784267 28710 raft_consensus.cc:2764] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:24:39.891425 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 145892us
W20260709 12:24:39.892128 26018 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.160s (SENT)
I20260709 12:24:39.774646 28275 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:39.692754 28851 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:39.830863 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.725003 28840 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:39.662113 25950 rpcz_store.cc:269] 0709 12:24:22.002891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.003069 (+   178us) service_pool.cc:224] Handling call
0709 12:24:39.582690 (+17579621us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:32.820505 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:32.821162 (+   657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:32.821214 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:24:32.821233 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:32.939220 (+117987us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:32.939338 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:32.939367 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:24:32.940888 (+  1521us) tablet.cc:701] Row locks acquired
0709 12:24:32.940946 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:24:32.941131 (+   185us) write_op.cc:270] Start()
0709 12:24:32.941291 (+   160us) write_op.cc:276] Timestamp: P: 1783599872941062 usec, L: 0
0709 12:24:32.941314 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:24:32.964307 (+ 22993us) log.cc:844] Serialized 6889 byte log entry
0709 12:24:34.410277 (+1445970us) op_driver.cc:464] REPLICATION: finished
0709 12:24:34.935041 (+524764us) write_op.cc:301] APPLY: starting
0709 12:24:34.935169 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:34.935669 (+   500us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:34.935804 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:35.554587 (+618783us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:35.555027 (+   440us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:24:35.555078 (+    51us) write_op.cc:312] APPLY: finished
0709 12:24:35.555602 (+   524us) log.cc:844] Serialized 119 byte log entry
0709 12:24:36.260218 (+704616us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:36.260925 (+   707us) write_op.cc:454] Released schema lock
0709 12:24:36.261091 (+   166us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:36.261978 (+   887us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:36.262010 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13965,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":454577,"apply.run_cpu_time_us":25130,"apply.run_wall_time_us":1335414,"num_ops":17,"prepare.queue_time_us":265601,"prepare.run_cpu_time_us":18349,"prepare.run_wall_time_us":855479,"raft.queue_time_us":639339,"raft.run_cpu_time_us":47094,"raft.run_wall_time_us":607292,"replication_time_us":1321743,"thread_start_us":1563631,"threads_started":5,"wal-append.queue_time_us":1458105,"wal-append.run_cpu_time_us":44770,"wal-append.run_wall_time_us":2591360}]]}
I20260709 12:24:39.841135 26129 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:39.791728 28863 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.847820 26196 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:39.851862 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:39.769367 28649 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:39.802037 28852 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.878759 28637 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:39.793870 28606 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:24:39.847065 28850 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.863165 28800 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:39.860004 28847 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.853309 28790 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:24:39.895545 28856 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:24:39.896268 26195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
W20260709 12:24:39.896754 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.809347 28812 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
W20260709 12:24:39.899374 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 451771us
I20260709 12:24:39.725562 28846 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:39.913414 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108907us
I20260709 12:24:39.897585 28821 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:39.899924 28609 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:39.920531 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 128127us
I20260709 12:24:39.930920 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.934527 26047 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:39.934547 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:39.931149 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91516.3us
W20260709 12:24:39.946180 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.911899 28850 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.946411 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:39.957034 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:39.860785 28848 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:39.918109 28601 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:39.966157 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53139.7us
I20260709 12:24:39.969652 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:39.922286 28800 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:39.982602 28878 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:39.985049 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2416) took 1145 ms. Trace:
W20260709 12:24:39.987332 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56498.5us
I20260709 12:24:39.988409 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1192) took 1311 ms. Trace:
I20260709 12:24:39.935904 28795 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.933597 28821 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:39.989852 26047 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:39.963913 28850 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.988818 26196 rpcz_store.cc:276] 0709 12:24:38.676744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.676865 (+   121us) service_pool.cc:224] Handling call
0709 12:24:38.711457 (+ 34592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.905412 (+1193955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.988163 (+ 82751us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400302d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:39.988346 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607979776}
I20260709 12:24:39.969486 28785 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.985580 26123 rpcz_store.cc:276] 0709 12:24:38.839629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.839839 (+   210us) service_pool.cc:224] Handling call
0709 12:24:38.866659 (+ 26820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.001671 (+135012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.001701 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.333610 (+331909us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.492056 (+158446us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:39.492097 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.492166 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.975851 (+483685us) raft_consensus.cc:1685] finished
0709 12:24:39.975880 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.984978 (+  9098us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134470,"raft.queue_time_us":155488,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":18823,"thread_start_us":145393,"threads_started":1,"wal-append.queue_time_us":317128}
I20260709 12:24:39.965520 28693 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.928022 28862 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:24:39.974355 28843 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:40.018643 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:40.024757 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:40.019711 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:39.967418 28790 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:40.009032 28821 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:40.028147 28790 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:40.029124 28790 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.030481 28790 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.024045 26050 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:40.035851 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.004595 28847 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:40.038774 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81793.5us
I20260709 12:24:40.040017 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:40.013042 28601 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:40.055161 28726 raft_consensus.cc:1064] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:40.016747 28714 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:40.056669 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:40.057860 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.046761 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:40.023273 28693 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.055552 28873 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:40.064334 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:40.003770 28825 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:24:39.975284 28860 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.070143 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:40.037349 28843 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.042932 28756 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:40.027019 28785 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.009312 28275 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.020362 28637 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:40.294600 26050 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:40.205515 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:40.200680 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:40.212121 26048 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:40.295363 26051 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.204972 26052 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:40.111205 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.172853 26047 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:40.089778 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.167682 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:40.089419 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83695.8us
W20260709 12:24:40.142056 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.332916 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:40.301565 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75572.7us
I20260709 12:24:40.334822 26791 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:40.120512 28764 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.629s	user 0.003s	sys 0.008s
W20260709 12:24:40.336522 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 289772us
I20260709 12:24:40.367045 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:40.209789 28659 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:40.362843 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:40.202109 28806 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.868s	user 0.009s	sys 0.002s
W20260709 12:24:40.089587 28726 raft_consensus.cc:1068] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:40.211024 28813 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:40.121773 28677 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.259s
I20260709 12:24:40.364985 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:40.199680 28606 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.365890 26783 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:40.199209 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:40.091209 28693 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:40.336395 28649 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:40.361686 26054 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:40.292479 28843 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:40.374161 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.374774 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 345923us
I20260709 12:24:40.375555 26049 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:24:40.375669 26167 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.998s (SENT)
I20260709 12:24:40.336445 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.370370 26045 raft_consensus.cc:2376] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5. Re-sending same reply.
I20260709 12:24:40.065289 28551 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:40.110708 28883 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.365790 26048 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:40.389669 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:40.372082 28847 raft_consensus.cc:2764] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:24:40.409945 26052 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:24:40.121299 28868 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:24:40.395622 26787 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:40.204087 28796 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.367662 28840 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:40.426239 28790 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:40.428309 28865 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 12:24:40.429584 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2419) took 1002 ms. Trace:
I20260709 12:24:40.429716 28785 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:40.430006 26122 rpcz_store.cc:276] 0709 12:24:39.426745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.436347 (+  9602us) service_pool.cc:224] Handling call
0709 12:24:39.463401 (+ 27054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.463838 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.463914 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.464575 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.749882 (+285307us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:39.749942 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.750039 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.429359 (+679320us) raft_consensus.cc:1685] finished
0709 12:24:40.429378 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.429495 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9916,"thread_start_us":284699,"threads_started":1,"wal-append.queue_time_us":433158}
W20260709 12:24:40.432969 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 68124us
I20260709 12:24:40.433735 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:40.409308 28848 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:40.434659 26054 raft_consensus.cc:2955] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:40.435674 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:40.155421 28874 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.380s
I20260709 12:24:40.412906 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:40.182346 28800 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:40.300592 28864 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 12:24:40.423957 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:40.424611 28879 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:40.411563 28873 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.457868 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 383426us
I20260709 12:24:40.385469 28813 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:40.180492 28870 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.419s
I20260709 12:24:40.212704 28821 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.180326 28877 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.419s
W20260709 12:24:40.441877 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.204066 28748 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.449851 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.111093 28714 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:40.473222 28659 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:40.404510 28812 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:40.425987 28852 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.000s
W20260709 12:24:40.460343 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93357.9us
I20260709 12:24:40.460978 26120 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:40.552078 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:40.472134 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 96599.4us
I20260709 12:24:40.458348 28601 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:40.481449 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.482082 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.483592 26047 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:40.451313 28649 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:40.485291 26046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2411) took 1783 ms (client timeout 1744 ms). Trace:
I20260709 12:24:40.484320 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.505551 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:40.388466 28806 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.405547 28606 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:40.580350 26046 rpcz_store.cc:269] 0709 12:24:38.702120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.710240 (+  8120us) service_pool.cc:224] Handling call
0709 12:24:40.485044 (+1774804us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:40.485211 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2360222976}
I20260709 12:24:40.456115 28551 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.473842 28848 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:40.596772 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.423990 28764 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:24:40.508858 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:40.538375 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:40.424002 28867 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
I20260709 12:24:40.443423 28677 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 12:24:40.559928 26125 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:40.580668 26129 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
W20260709 12:24:40.579536 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63614.9us
W20260709 12:24:40.592654 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 87244.9us
I20260709 12:24:40.593425 26123 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:40.594316 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.604059 26052 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:40.613452 26053 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:40.614177 26122 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.610314 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:40.616160 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.616398 26047 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:40.617893 26126 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:40.621999 26129 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:40.622763 25949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.010s	sys 0.013s
I20260709 12:24:40.622889 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:40.623349 25949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.010s	sys 0.013s
I20260709 12:24:40.578015 28312 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:40.580294 28649 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.625334 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:40.631245 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.631475 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.639786 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.640302 26129 raft_consensus.cc:2376] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6. Re-sending same reply.
I20260709 12:24:40.640974 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.641660 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.642211 26129 raft_consensus.cc:2368] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6.
I20260709 12:24:40.642751 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.598162 28659 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:40.583284 28884 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.643778 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.644170 26121 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:40.644933 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:40.645280 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.572512 28852 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.478487 28873 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.645459 28649 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.603324 28551 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:40.659530 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:40.632135 28844 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.032s	user 0.015s	sys 0.000s
I20260709 12:24:40.648753 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.664994 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.678129 26128 raft_consensus.cc:2368] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8.
I20260709 12:24:40.666087 28714 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:40.666172 26122 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:40.667281 28886 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:40.661649 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:40.672714 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.671846 26052 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:40.628268 28312 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:40.688035 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.695806 26129 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:40.697088 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.697896 26129 raft_consensus.cc:2368] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:40.698712 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:40.699452 26129 raft_consensus.cc:2368] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5.
I20260709 12:24:40.660548 28862 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:40.656996 28876 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.709244 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:40.657473 28866 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:40.657387 28895 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:40.719237 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:40.739140 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:40.649850 28724 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:24:40.662199 28870 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:24:40.747501 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:40.748782 26047 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:40.707968 28862 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:40.756026 28880 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:24:40.756469 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2421) took 1467 ms. Trace:
I20260709 12:24:40.740033 26786 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:40.770385 26127 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:40.756834 26053 rpcz_store.cc:276] 0709 12:24:39.288801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.288914 (+   113us) service_pool.cc:224] Handling call
0709 12:24:40.756279 (+1467365us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400153454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:40.756409 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3035371776}
I20260709 12:24:40.766152 28825 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:40.662389 28813 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:40.773623 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:40.767269 28609 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.812s	user 0.006s	sys 0.002s
W20260709 12:24:40.812829 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.753160 28862 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:40.975880 26051 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.004s
I20260709 12:24:40.841269 26047 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 4: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:41.527329 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 382) took 1804 ms. Trace:
I20260709 12:24:41.532465 26051 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:41.463299 26018 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.592s (SENT)
I20260709 12:24:41.526588 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 386) took 1043 ms. Trace:
W20260709 12:24:40.855165 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 299180us
I20260709 12:24:41.532889 26120 rpcz_store.cc:276] 0709 12:24:39.723156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.723314 (+   158us) service_pool.cc:224] Handling call
0709 12:24:39.754541 (+ 31227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.552484 (+797943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.527033 (+974549us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54001860d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:41.527258 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622301568}
W20260709 12:24:40.994805 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 347635us
I20260709 12:24:41.534179 26123 rpcz_store.cc:276] 0709 12:24:40.482927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.483071 (+   144us) service_pool.cc:224] Handling call
0709 12:24:40.554167 (+ 71096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.612838 (+ 58671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.526195 (+913357us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:41.526470 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89795456}
I20260709 12:24:41.533741 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1202) took 1333 ms. Trace:
W20260709 12:24:41.536654 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6575478790000489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2762s for a response
I20260709 12:24:40.753446 28796 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:40.757745 28876 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:41.535446 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.563398 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 294) took 1127 ms. Trace:
W20260709 12:24:41.563870 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100620us
I20260709 12:24:40.871033 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:41.557891 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.536248 26051 rpcz_store.cc:276] 0709 12:24:40.200121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.205136 (+  5015us) service_pool.cc:224] Handling call
0709 12:24:41.533683 (+1328547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:41.580940 26047 rpcz_store.cc:276] 0709 12:24:40.435758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.773067 (+337309us) service_pool.cc:224] Handling call
0709 12:24:41.563298 (+790231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.847308 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:41.138659 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.112300 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 533299us
W20260709 12:24:41.417007 26048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.000s	sys 0.031s
I20260709 12:24:41.556102 26785 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:41.583360 26048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.000s	sys 0.038s
W20260709 12:24:40.880476 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.462805 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 743562us
I20260709 12:24:41.353025 26791 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:40.881073 26127 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:41.585182 26791 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:40.970757 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:41.581339 26167 connection.cc:256] RPC call timeout handler was delayed by -1.1766599140000835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260709 12:24:41.557339 26125 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:24:41.586412 26167 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.936s (SENT)
W20260709 12:24:41.587692 26168 connection.cc:256] RPC call timeout handler was delayed by -2.0605956499998683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 12:24:41.587651 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:41.463261 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:41.563313 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1212) took 1130 ms. Trace:
W20260709 12:24:40.875068 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 439462us
I20260709 12:24:41.586908 26053 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
W20260709 12:24:40.866825 28737 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.065s	user 0.006s	sys 0.006s
W20260709 12:24:40.877367 28872 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.914s	user 0.001s	sys 0.009s
I20260709 12:24:41.006206 28888 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.064077 28898 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:24:41.148429 28806 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.063126 28897 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.580940 26052 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:41.355541 28814 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:41.468525 28851 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:41.099633 28850 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.038s	user 0.009s	sys 0.004s
W20260709 12:24:41.349709 28700 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.565s	user 0.006s	sys 0.006s
I20260709 12:24:41.603307 26020 leader_election.cc:400] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6.
W20260709 12:24:41.464906 28896 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.762s	user 0.000s	sys 0.001s
W20260709 12:24:41.225230 28795 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.183s	user 0.004s	sys 0.008s
W20260709 12:24:41.534435 28714 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.003s	sys 0.013s
W20260709 12:24:41.558759 28801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.716s	user 0.009s	sys 0.009s
I20260709 12:24:40.776108 28825 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:41.062704 28724 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.983201 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 604) took 1347 ms. Trace:
W20260709 12:24:41.327087 28846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.012s	sys 0.025s
I20260709 12:24:41.614352 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:41.634569 28801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.816s	user 0.010s	sys 0.010s
I20260709 12:24:40.845113 28659 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.623957 28847 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:24:41.006927 28312 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:41.617662 28479 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.934s	user 0.012s	sys 0.000s
I20260709 12:24:41.618573 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:41.657174 26047 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:41.619234 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.589191 28551 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:40.851037 28895 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:41.355022 28726 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:41.458690 28865 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:24:41.366640 28677 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:41.631232 28850 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.591352 28813 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:40.894752 28606 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:41.279819 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.597993 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:41.666527 26054 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:41.667037 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1222) took 1022 ms. Trace:
I20260709 12:24:41.667330 26054 rpcz_store.cc:276] 0709 12:24:40.645193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.597711 (+952518us) service_pool.cc:224] Handling call
0709 12:24:41.666977 (+ 69266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:41.590803 26045 rpcz_store.cc:276] 0709 12:24:40.433002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.436352 (+  3350us) service_pool.cc:224] Handling call
0709 12:24:40.436716 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.617402 (+180686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.617431 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.626576 (+  9145us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.725886 (+ 99310us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:40.725938 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.726036 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.990359 (+264323us) raft_consensus.cc:1685] finished
0709 12:24:40.990388 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.117837 (+127449us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b4800080358. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:24:41.563223 (+445386us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21907,"raft.run_cpu_time_us":3582,"raft.run_wall_time_us":646282,"spinlock_wait_cycles":14929792,"thread_start_us":279158,"threads_started":2,"wal-append.queue_time_us":243937}
I20260709 12:24:41.668171 26789 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:41.603820 26791 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.592729 28852 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.636813 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:41.640056 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55415.1us
I20260709 12:24:41.670377 26020 leader_election.cc:400] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8.
I20260709 12:24:41.640723 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:41.627472 28748 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.627163 28814 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:41.672602 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:41.672765 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.672866 28847 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:41.593505 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 295) took 1157 ms. Trace:
I20260709 12:24:41.673662 26789 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:41.632325 28714 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.003s	sys 0.014s
I20260709 12:24:41.657990 28846 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:41.632367 28637 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.615724 28873 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.631255 28821 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.651141 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:41.652509 26018 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.884s (SENT)
W20260709 12:24:41.646600 28899 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.893s	user 0.000s	sys 0.001s
W20260709 12:24:41.644863 28889 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.895s	user 0.002s	sys 0.000s
W20260709 12:24:41.658087 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.658674 26019 connection.cc:256] RPC call timeout handler was delayed by -3.3096579100001691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:41.691385 26127 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:24:41.678165 28849 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.831s	user 0.014s	sys 0.000s
I20260709 12:24:41.646414 28710 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:41.621505 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1208) took 1282 ms. Trace:
I20260709 12:24:41.698154 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:41.701834 26123 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:41.664271 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:41.633045 28700 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.631695 28897 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.663904 28312 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:41.710203 28897 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.712116 28700 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:41.713047 26054 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:41.713618 26786 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:24:41.714087 28860 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:41.714848 28913 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:41.714769 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:41.708567 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56094.3us
I20260709 12:24:41.687444 28637 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:41.716847 28916 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:24:41.717401 26018 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.067s (SENT)
I20260709 12:24:41.718360 28637 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:41.718550 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1210) took 1286 ms. Trace:
W20260709 12:24:41.719748 28649 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.035s	user 0.008s	sys 0.000s
I20260709 12:24:41.720319 28895 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.721060 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.721856 26787 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:41.722769 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:41.664355 28677 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:41.724207 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 608) took 1823 ms. Trace:
I20260709 12:24:41.654512 28806 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:41.662950 28726 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:41.722698 28863 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.813s	user 0.012s	sys 0.000s
I20260709 12:24:41.674314 26053 rpcz_store.cc:276] 0709 12:24:40.436084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.871433 (+435349us) service_pool.cc:224] Handling call
0709 12:24:41.593419 (+721986us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:41.644937 28724 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.677098 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:41.735837 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58809.7us
I20260709 12:24:41.736738 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:41.738035 26020 leader_election.cc:400] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:41.738302 26020 leader_election.cc:403] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:41.728224 26047 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:41.673771 28847 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:41.644778 28825 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.684684 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.685859 26791 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:41.687405 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:41.740787 26789 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:41.740907 26051 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:41.623634 28888 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.673364 28814 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.661509 28479 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.706641 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:41.776111 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 69710us
I20260709 12:24:41.707893 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:41.778074 26123 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:41.778910 28846 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:41.779458 28846 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.779875 28846 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:41.782765 26049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.008s	sys 0.015s
I20260709 12:24:41.783524 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:41.708547 28551 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:41.783110 26049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.008s	sys 0.015s
I20260709 12:24:41.784399 28551 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:41.785701 28551 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.708807 28601 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:41.713897 28852 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:41.715085 28812 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.143s	user 0.008s	sys 0.000s
I20260709 12:24:41.673720 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:41.704409 26050 rpcz_store.cc:276] 0709 12:24:40.339081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.363798 (+ 24717us) service_pool.cc:224] Handling call
0709 12:24:40.381783 (+ 17985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.394967 (+ 13184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.395006 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.425436 (+ 30430us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.582824 (+157388us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:40.682795 (+ 99971us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.702037 (+ 19242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.702115 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.621121 (+919006us) raft_consensus.cc:1685] finished
0709 12:24:41.621152 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.621278 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30903,"prepare.queue_time_us":155219,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":63848,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":44804,"thread_start_us":236330,"threads_started":2,"wal-append.queue_time_us":282498}
I20260709 12:24:41.792825 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:41.630805 28878 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.724642 26782 rpcz_store.cc:276] 0709 12:24:39.900631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.902285 (+  1654us) service_pool.cc:224] Handling call
0709 12:24:39.932196 (+ 29911us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.989684 (+ 57488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.002330 (+ 12646us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.055649 (+ 53319us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.509353 (+453704us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.509417 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.509514 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.724012 (+1214498us) raft_consensus.cc:1685] finished
0709 12:24:41.724030 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.724153 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":696,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":12923,"thread_start_us":425250,"threads_started":1,"wal-append.queue_time_us":594621}
I20260709 12:24:41.600999 28737 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:41.719774 28912 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:41.699853 28850 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:41.808713 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.732120 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 605) took 2096 ms. Trace:
W20260709 12:24:41.732558 28880 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.976s	user 0.001s	sys 0.011s
W20260709 12:24:41.742321 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.733021 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.695904 28710 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:41.696650 28380 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.774003 26020 leader_election.cc:400] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5.
I20260709 12:24:41.814836 26020 leader_election.cc:403] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:41.718981 26048 rpcz_store.cc:276] 0709 12:24:40.432221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.441166 (+  8945us) service_pool.cc:224] Handling call
0709 12:24:40.476748 (+ 35582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.484884 (+  8136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.484925 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.559427 (+ 74502us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.583789 (+1024362us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:41.621939 (+ 38150us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.627320 (+  5381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.627409 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.718327 (+ 90918us) raft_consensus.cc:1685] finished
0709 12:24:41.718357 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.718490 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65120,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":3227,"raft.queue_time_us":207278,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":22691,"thread_start_us":959939,"threads_started":1,"wal-append.queue_time_us":1051685}
W20260709 12:24:41.815568 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.700410 28849 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.784423 26120 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:41.798925 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:41.769917 28806 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:41.809397 26789 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 4.
W20260709 12:24:41.809849 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.830463 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.810261 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.000s	sys 0.012s
I20260709 12:24:41.793733 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:41.831979 26047 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:41.833721 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.000s	sys 0.013s
I20260709 12:24:41.822674 26051 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:41.825665 26050 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:41.729210 28677 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.720177 28649 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.772709 28863 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:41.771150 28847 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:41.809466 26788 rpcz_store.cc:276] 0709 12:24:39.635748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.636203 (+   455us) service_pool.cc:224] Handling call
0709 12:24:39.656838 (+ 20635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.834457 (+177619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.834486 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.840686 (+  6200us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.212572 (+371886us) spinlock_profiling.cc:243] Waited 981 us on lock 0x7b18000c00a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:40.647199 (+434627us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.647265 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.647524 (+   259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.064323 (+416799us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18000c00a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8557 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:41.731890 (+667567us) raft_consensus.cc:1685] finished
0709 12:24:41.731912 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.732056 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152680,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12683,"spinlock_wait_cycles":9098368,"thread_start_us":604506,"threads_started":2,"wal-append.queue_time_us":830311}
I20260709 12:24:41.836892 26783 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:41.841063 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:41.844067 26048 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:41.808535 28878 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.810515 28812 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:41.842679 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.873662 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.812799 28700 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:41.813913 28897 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:41.880487 26791 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:41.880465 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.882105 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.883105 26782 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:41.885195 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:41.828810 28825 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:41.885667 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:41.886125 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54722.1us
I20260709 12:24:41.884202 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:41.887224 26791 pending_rounds.cc:85] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 5
I20260709 12:24:41.889034 26195 raft_consensus.cc:606] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:41.831478 28873 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7.
I20260709 12:24:41.894558 26791 pending_rounds.cc:107] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:41.840464 28821 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5.
I20260709 12:24:41.895072 26791 raft_consensus.cc:2889] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.828418 28380 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:41.852353 28275 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.000s	sys 0.027s
I20260709 12:24:41.897935 26053 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:41.909915 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1219) took 1272 ms. Trace:
I20260709 12:24:41.910378 26053 rpcz_store.cc:276] 0709 12:24:40.637369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.852875 (+1215506us) service_pool.cc:224] Handling call
0709 12:24:41.892541 (+ 39666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.909543 (+ 17002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.909842 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:41.849470 28870 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.916070 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.907532 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:41.854470 28814 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:41.893021 28880 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:24:41.886682 28801 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.933606 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:41.895380 28677 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:41.934418 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 348000us
I20260709 12:24:41.934118 26050 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:41.940442 26053 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:24:41.889132 28853 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.970795 26201 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:41.934726 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:41.974642 26199 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:41.935397 26782 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:41.950330 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:41.896517 28852 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.980746 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 100524us
I20260709 12:24:41.959081 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:41.964372 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:41.981954 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 189640us
W20260709 12:24:41.895843 28894 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.003s	sys 0.006s
I20260709 12:24:41.951241 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:41.915169 28275 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.001s	sys 0.027s
I20260709 12:24:41.967680 26048 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:41.957998 26053 pending_rounds.cc:85] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 7
I20260709 12:24:41.994489 26053 pending_rounds.cc:107] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:41.994864 26053 raft_consensus.cc:2889] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.978219 26200 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:41.979074 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:41.996441 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.936455 28843 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.498s	user 0.011s	sys 0.001s
I20260709 12:24:41.984575 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:42.001415 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 110505us
I20260709 12:24:42.004144 26202 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:41.879417 28726 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.987140 26194 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:41.987887 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 214062us
I20260709 12:24:41.981622 26196 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:41.990919 26203 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:41.642822 26784 rpcz_store.cc:276] 0709 12:24:39.635353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.637401 (+  2048us) service_pool.cc:224] Handling call
0709 12:24:39.726907 (+ 89506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.853789 (+126882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.856732 (+  2943us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.919470 (+ 62738us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.367632 (+448162us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:40.373801 (+  6169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.385812 (+ 12011us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.982764 (+596952us) raft_consensus.cc:1685] finished
0709 12:24:40.982813 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.983090 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74607,"raft.queue_time_us":83539,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":11091,"thread_start_us":309238,"threads_started":1,"wal-append.queue_time_us":503622}
I20260709 12:24:41.997394 26198 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:41.997907 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:42.051647 26197 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:41.955824 28863 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:42.005929 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:41.979317 28700 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:42.002365 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1220) took 1364 ms. Trace:
I20260709 12:24:42.058908 26053 rpcz_store.cc:276] 0709 12:24:40.637698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.911371 (+1273673us) service_pool.cc:224] Handling call
0709 12:24:41.940090 (+ 28719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.002076 (+ 61986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.002304 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:41.986644 26018 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.861s (SENT)
W20260709 12:24:41.987509 28894 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.330s	user 0.003s	sys 0.006s
W20260709 12:24:42.052500 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.064147 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 77602.1us
W20260709 12:24:42.066421 26018 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.970s (SENT)
I20260709 12:24:42.057777 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:42.069507 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:41.969725 28606 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 12:24:41.989668 28853 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.928861 28926 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 12:24:41.945922 28847 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.965443 28801 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.936610 28870 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:41.993134 28852 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:41.941643 28710 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:42.283864 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.069904 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:42.296633 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.317243 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.316112 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:42.146899 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.319576 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.320140 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.109799 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 106613us
I20260709 12:24:42.191425 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:42.327100 26196 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.114580 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:42.127988 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:42.331862 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 265465us
W20260709 12:24:42.339781 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.346571 28737 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:42.304003 28862 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.002s
W20260709 12:24:42.306202 28813 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.000s
W20260709 12:24:42.313275 28921 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:24:42.349440 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 443479us
W20260709 12:24:42.348959 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 279107us
W20260709 12:24:42.304720 28924 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
I20260709 12:24:42.356159 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:42.357663 26197 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:42.357923 26123 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 12:24:42.358619 26123 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:42.319801 28726 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.109266 28863 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:42.374691 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.338603 28918 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:24:42.110018 28796 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.361595 26790 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:42.367810 26194 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:42.368355 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:42.392083 26167 leader_election.cc:336] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.664s (SENT)
I20260709 12:24:42.374506 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:42.393724 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:42.349371 28873 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:42.189637 28922 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:42.402803 26197 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 5: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:42.401764 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:42.360718 28925 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:42.402168 26199 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:42.059260 28806 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.414775 28649 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:42.068779 28275 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.308s
I20260709 12:24:42.338163 28920 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 12:24:42.416975 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:42.049773 28843 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:42.393687 28932 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
W20260709 12:24:42.458637 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.419571 28312 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.710s	user 0.004s	sys 0.004s
W20260709 12:24:42.444962 28897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.049s
I20260709 12:24:42.361321 28862 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.421139 28863 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:42.421598 28949 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:42.425690 28795 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:42.428154 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.432304 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.444962 28849 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.059074 28864 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.314s
I20260709 12:24:42.432518 26123 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:42.437685 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:42.475412 26202 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
W20260709 12:24:42.482259 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 126174us
I20260709 12:24:42.482473 26199 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.483032 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:42.338505 28700 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.488279 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:42.467016 28935 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:42.452134 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:42.471101 28897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.005s	sys 0.050s
I20260709 12:24:42.402880 28813 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:42.450981 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:42.510289 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:42.458153 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.510948 26202 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:42.460775 26756 connection.cc:256] RPC call timeout handler was delayed by -4.95620496399988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:24:42.511466 26756 connection.cc:256] RPC call timeout handler was delayed by -4.8590049639997233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 12:24:42.511907 26756 connection.cc:256] RPC call timeout handler was delayed by -4.8482049639997058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 12:24:42.463811 28380 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:42.413667 28796 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.436758 28925 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.480423 26124 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:42.431800 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:42.487496 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.435746 28659 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 12:24:42.450757 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58707.6us
W20260709 12:24:42.528925 26167 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.685s (SENT)
W20260709 12:24:42.530434 26167 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.876s (SENT)
I20260709 12:24:42.530951 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:42.465849 28275 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:42.497942 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:42.412055 28934 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.466992 28939 raft_consensus.cc:1064] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:42.507670 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:42.499941 26791 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:42.509138 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:42.512404 26756 connection.cc:256] RPC call timeout handler was delayed by -4.8356049639996854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:24:42.534228 26756 connection.cc:256] RPC call timeout handler was delayed by -4.6727049639998768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 12:24:42.534696 26756 connection.cc:256] RPC call timeout handler was delayed by -4.617804963999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 12:24:42.534960 26756 connection.cc:256] RPC call timeout handler was delayed by -4.6160049639997851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:24:42.535369 26756 connection.cc:256] RPC call timeout handler was delayed by -3.5549049639998884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30950000000000005s for a response
I20260709 12:24:42.539671 26198 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:42.458500 28922 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:42.510370 26194 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:42.524609 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:42.549749 26050 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.540481 26199 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:42.541524 26202 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:42.462404 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44736 (request call id 23) took 2580 ms. Trace:
W20260709 12:24:42.544746 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.553094 26195 rpcz_store.cc:276] 0709 12:24:39.881986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.882142 (+   156us) service_pool.cc:224] Handling call
0709 12:24:42.462129 (+2579987us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:42.462271 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4529295104}
I20260709 12:24:42.555094 26196 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:42.500041 28897 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:42.525295 28693 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 2.087s	user 0.015s	sys 0.000s
I20260709 12:24:42.558995 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:42.560139 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47095us
I20260709 12:24:42.619186 26196 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:42.619410 26167 leader_election.cc:336] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.854s (SENT)
I20260709 12:24:42.622725 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:42.598802 26018 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.507s (SENT)
W20260709 12:24:42.624070 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.624110 26203 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.000s
I20260709 12:24:42.624804 26203 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:42.618343 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:42.626884 26782 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:42.616044 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:42.539000 28925 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.629314 26195 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:24:42.572912 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 212695us
I20260709 12:24:42.538098 28933 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:42.538722 28275 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:42.497179 28949 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.632192 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1213) took 2198 ms. Trace:
I20260709 12:24:42.627975 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:42.552052 28894 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:42.638618 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:42.515156 28813 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.524936 28795 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260709 12:24:42.650560 28936 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:24:42.651839 28862 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:42.653682 28677 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.002s
I20260709 12:24:42.654773 26194 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:42.658510 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:42.660601 28851 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:42.661741 28940 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:42.664608 28938 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:42.668361 26018 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.007s (SENT)
I20260709 12:24:42.670900 26195 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
W20260709 12:24:42.671332 28860 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.957s	user 0.010s	sys 0.001s
I20260709 12:24:42.660666 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:42.545586 28934 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.673388 28870 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.675501 28853 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:42.676973 28945 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:42.679798 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2439) took 1087 ms. Trace:
I20260709 12:24:42.686579 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:42.686753 26129 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:42.690824 26197 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:42.696689 26124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2425) took 2204 ms (client timeout 1610 ms). Trace:
I20260709 12:24:42.599336 28950 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:42.700255 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:42.651851 28814 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.001s
W20260709 12:24:42.745712 26124 rpcz_store.cc:269] 0709 12:24:40.481633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.481774 (+   141us) service_pool.cc:224] Handling call
0709 12:24:42.686178 (+2204404us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:42.686353 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4419303168}
I20260709 12:24:42.704538 28700 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:42.705246 28659 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:42.630522 28693 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:42.705542 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47136.1us
I20260709 12:24:42.705762 26194 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:42.645733 28929 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.133s	user 0.004s	sys 0.004s
I20260709 12:24:42.637092 28714 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:42.709206 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:42.713781 26196 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
W20260709 12:24:42.715267 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.639979 26049 rpcz_store.cc:276] 0709 12:24:40.433321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.434658 (+  1337us) service_pool.cc:224] Handling call
0709 12:24:40.434967 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.526569 (+ 91602us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:40.772666 (+246097us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.810385 (+1037719us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.848943 (+ 38558us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.884839 (+ 35896us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.887692 (+  2853us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:41.888025 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.231226 (+343201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.231335 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.631913 (+400578us) raft_consensus.cc:1685] finished
0709 12:24:42.631942 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.632103 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50208,"prepare.queue_time_us":212832,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":122,"raft.queue_time_us":165489,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":40877,"spinlock_wait_cycles":37760512,"thread_start_us":1614330,"threads_started":4,"wal-append.queue_time_us":999086}
I20260709 12:24:42.636885 28933 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:42.724047 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:42.745004 26048 rpcz_store.cc:276] 0709 12:24:41.592208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.994227 (+402019us) service_pool.cc:224] Handling call
0709 12:24:42.051075 (+ 56848us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.071572 (+ 20497us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:42.071615 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:42.117411 (+ 45796us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.453872 (+336461us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:42.453919 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.453991 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.679500 (+225509us) raft_consensus.cc:1685] finished
0709 12:24:42.679524 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.679692 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":257066,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":131043,"threads_started":1,"wal-append.queue_time_us":346573}
I20260709 12:24:42.703161 28949 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:42.707844 28846 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.927s	user 0.012s	sys 0.000s
I20260709 12:24:42.679628 28897 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.746376 26782 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:42.908136 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:42.705807 28795 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:42.908594 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:42.867447 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2438) took 1275 ms (client timeout 1661 ms). Trace:
W20260709 12:24:42.948482 26054 rpcz_store.cc:269] 0709 12:24:41.591683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.672326 (+ 80643us) service_pool.cc:224] Handling call
0709 12:24:42.867157 (+1194831us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b540006fc54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:42.867365 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90939648}
W20260709 12:24:42.790768 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 66046.9us
I20260709 12:24:42.937218 26203 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:42.823683 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.748097 26128 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:24:42.788504 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.952952 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.953496 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 285158us
W20260709 12:24:42.953728 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.708756 28801 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:42.884976 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:42.875406 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:42.960616 26788 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:42.729569 28852 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.838047 28950 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:42.891062 28847 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.967983 28950 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.823930 28870 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:42.970016 28870 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:42.970407 28870 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.973023 28863 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:42.954216 28726 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.001s
I20260709 12:24:42.816291 28275 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.981187 28275 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.982594 28795 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:42.983625 28795 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:42.946683 28846 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:42.985698 26786 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:42.986410 28846 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.989209 28878 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:42.989810 28693 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.990262 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1610) took 1052 ms. Trace:
I20260709 12:24:42.990602 26200 rpcz_store.cc:276] 0709 12:24:41.937874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.937986 (+   112us) service_pool.cc:224] Handling call
0709 12:24:41.977872 (+ 39886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.995575 (+ 17703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.989960 (+994385us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b5400154120. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a8a02 00007f5b743a4dd6 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:42.990200 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6818560}
I20260709 12:24:42.823135 26784 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:42.992463 28846 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.993229 28846 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:42.943718 28659 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:42.994917 28825 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.102s	user 0.010s	sys 0.005s
I20260709 12:24:42.995816 26787 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:42.995389 28659 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:42.996495 28659 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.996791 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:42.997102 26202 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:24:42.997406 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 768) took 1298 ms. Trace:
I20260709 12:24:42.997786 26787 rpcz_store.cc:276] 0709 12:24:41.698806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.832989 (+134183us) service_pool.cc:224] Handling call
0709 12:24:42.997196 (+1164207us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:42.997342 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541444352}
I20260709 12:24:42.998445 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:42.995733 28846 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.998940 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:42.999665 26787 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:42.999758 28846 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:43.000355 28846 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:42.992125 26200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:43.002090 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:43.002446 26200 raft_consensus.cc:606] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:43.002642 26194 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:42.997437 26202 pending_rounds.cc:85] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 7
I20260709 12:24:43.004652 26202 pending_rounds.cc:107] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:24:43.004940 26202 raft_consensus.cc:2889] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.004844 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:43.007819 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 755) took 2208 ms (client timeout 1998 ms). Trace:
W20260709 12:24:42.856724 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 196523us
W20260709 12:24:42.938046 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.911070 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.008405 26790 rpcz_store.cc:269] 0709 12:24:40.749526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.749661 (+   135us) service_pool.cc:224] Handling call
0709 12:24:42.895380 (+2145719us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54003c5854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:42.957785 (+ 62405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772733184}
I20260709 12:24:42.891883 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 756) took 2045 ms. Trace:
I20260709 12:24:42.886610 28938 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.020865 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2444) took 1421 ms. Trace:
I20260709 12:24:42.944058 26198 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:42.944435 26785 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:24:43.021366 26049 rpcz_store.cc:276] 0709 12:24:41.599472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.954503 (+1355031us) service_pool.cc:224] Handling call
0709 12:24:43.020173 (+ 65670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.020386 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.020408 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.020446 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:43.020473 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.020525 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.020809 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23808}
W20260709 12:24:42.938011 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.024338 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.958488 26201 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
W20260709 12:24:42.881543 28930 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.812s	user 0.003s	sys 0.001s
I20260709 12:24:42.936878 26052 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:42.857532 28941 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:24:42.964172 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.965680 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:42.878369 28479 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:42.962383 28949 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:43.012028 26783 rpcz_store.cc:276] 0709 12:24:40.846856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.847045 (+   189us) service_pool.cc:224] Handling call
0709 12:24:42.891562 (+2044517us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54004f4554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:42.891793 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584845568}
I20260709 12:24:42.908530 26124 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:43.031888 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:43.033707 26783 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4. Re-sending same reply.
I20260709 12:24:42.896572 28710 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.987979 28796 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:43.021996 26785 pending_rounds.cc:85] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 4
I20260709 12:24:43.042232 26785 pending_rounds.cc:107] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:43.042575 26785 raft_consensus.cc:2889] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:42.951102 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:43.043742 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92686.6us
I20260709 12:24:43.028507 26047 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:43.029541 26128 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:43.031889 26201 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:43.036538 26194 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:24:43.039597 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.995647 28888 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:43.040518 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.055838 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.058815 26051 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:43.058975 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.056849 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.060252 26790 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:43.061093 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.061755 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:43.061967 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.062335 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:43.062772 26783 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:43.061964 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.066331 26123 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:43.061447 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:43.066282 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.072603 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.073421 26783 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:43.074641 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:43.082050 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.105005 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.059878 28874 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:43.107218 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:43.065896 28952 raft_consensus.cc:2764] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:43.111532 26200 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:43.109948 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:43.110641 26788 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6. Re-sending same reply.
I20260709 12:24:43.117368 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.118240 26782 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:43.105701 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:43.108641 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.108254 26783 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:43.120695 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.120869 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.121289 26047 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:43.146555 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.159154 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:43.160621 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:43.160527 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:43.110338 28941 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:24:43.109172 28954 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:24:43.176635 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:43.176365 26788 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:43.121398 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:43.163424 28949 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.163561 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:43.104647 28796 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:24:43.125033 26790 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:43.166744 26051 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:43.171715 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:43.172967 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.108765 28700 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:24:43.036939 28938 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.181803 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:43.184392 26782 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:24:42.703222 28939 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.186940 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.210140 26047 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:43.175354 28920 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.173921 28952 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.191650 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.200505 26783 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:43.204619 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:43.207504 26046 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.207520 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:43.196708 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 125854us
I20260709 12:24:43.190786 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.220214 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.214522 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.218271 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 293) took 2782 ms. Trace:
I20260709 12:24:43.221484 26790 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 4: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:43.219105 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.218667 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.215395 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.216847 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1210) took 1156 ms. Trace:
I20260709 12:24:43.222995 26195 rpcz_store.cc:276] 0709 12:24:42.060579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.710009 (+649430us) service_pool.cc:224] Handling call
0709 12:24:42.744773 (+ 34764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.744970 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:42.814824 (+ 69854us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.216373 (+401549us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.216447 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:43.216505 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.216605 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.216763 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113009,"prepare.queue_time_us":70381,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":13334,"raft.queue_time_us":116106,"thread_start_us":287988,"threads_started":1}
I20260709 12:24:43.225282 26128 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:43.221330 26046 rpcz_store.cc:276] 0709 12:24:40.435471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.206951 (+2771480us) service_pool.cc:224] Handling call
0709 12:24:43.207314 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.218051 (+ 10737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.218219 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:43.072115 28846 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:43.229833 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.194717 28813 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:43.230834 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.230834 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:43.257328 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48993us
I20260709 12:24:43.257831 26195 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:43.258105 26782 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:43.229921 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.261873 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.262192 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.262643 26051 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:43.216061 28874 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:43.264890 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.186625 28949 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.219367 28693 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:43.261791 28862 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:43.286211 28862 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:43.286830 28862 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.287628 28938 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.287740 28949 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:43.290416 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.291718 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.292416 28825 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:43.292757 28953 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:43.293238 28953 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:43.293335 28825 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:43.286283 28955 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:24:43.295279 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2441) took 1698 ms. Trace:
I20260709 12:24:43.295661 26053 rpcz_store.cc:276] 0709 12:24:41.596692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.315923 (+719231us) service_pool.cc:224] Handling call
0709 12:24:42.350202 (+ 34279us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:42.405381 (+ 55179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.405537 (+   156us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:42.436323 (+ 30786us) log.cc:844] Serialized 52 byte log entry
0709 12:24:42.534864 (+ 98541us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:42.534901 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.534974 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.295041 (+760067us) raft_consensus.cc:1685] finished
0709 12:24:43.295060 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.295223 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50659,"raft.queue_time_us":105898,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6647,"thread_start_us":105191,"threads_started":2,"wal-append.queue_time_us":616911}
I20260709 12:24:43.296784 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.297513 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.297747 26051 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.298755 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.300330 28737 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:43.300750 28737 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:43.301674 28737 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.302659 28737 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.304638 28813 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.305119 28813 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.305490 28813 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.308883 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:43.309237 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.309651 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.313401 28814 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:43.314934 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.299911 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.260067 28849 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:43.319303 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.221213 28920 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:43.325959 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:43.331569 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.339351 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:43.340878 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.343014 26197 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [2.2-5.3]
I20260709 12:24:43.343098 26051 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:43.345340 26197 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.349984 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.362102 26199 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:43.370525 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.334932 28814 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:43.377692 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:43.350406 28941 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:24:43.397960 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.400825 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.221312 28952 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:43.401379 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0678286659999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 12:24:43.413358 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:43.404896 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.419610 26051 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.398873 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.422552 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:43.412109 28960 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.293s	user 0.001s	sys 0.003s
I20260709 12:24:43.398655 28825 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:43.423853 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46236us
W20260709 12:24:43.413653 28957 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:24:43.430881 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.397742 28860 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:43.438373 26197 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:43.069450 28933 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:43.440744 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.446345 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2439) took 1633 ms. Trace:
I20260709 12:24:43.412881 28956 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:43.294193 28953 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.457866 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:43.458742 26197 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:43.449473 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.423866 28659 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:43.461503 26052 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:43.462316 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2453) took 1127 ms. Trace:
I20260709 12:24:43.462637 26052 rpcz_store.cc:276] 0709 12:24:42.335020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.907883 (+572863us) service_pool.cc:224] Handling call
0709 12:24:43.462225 (+554342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:43.463588 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.434249 28973 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.465021 26052 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:43.446795 26121 rpcz_store.cc:276] 0709 12:24:41.813418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.813603 (+   185us) service_pool.cc:224] Handling call
0709 12:24:41.833462 (+ 19859us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:41.905488 (+ 72026us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:41.972972 (+ 67484us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.352564 (+379592us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:42.371951 (+ 19387us) log.cc:844] Serialized 52 byte log entry
0709 12:24:42.377093 (+  5142us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:42.377578 (+   485us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.500870 (+123292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.501103 (+   233us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.994125 (+493022us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b1800312e80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:43.446093 (+451968us) raft_consensus.cc:1685] finished
0709 12:24:43.446125 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.446282 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76365,"prepare.queue_time_us":58429,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":173401,"raft.run_cpu_time_us":648,"raft.run_wall_time_us":90540,"spinlock_wait_cycles":17227008,"thread_start_us":367038,"threads_started":2,"wal-append.queue_time_us":700253}
I20260709 12:24:43.465190 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.465893 26053 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:43.466465 26197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:43.467180 26197 raft_consensus.cc:606] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.434710 28825 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:43.469738 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.468051 28700 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.469189 28973 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.471419 28851 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:43.472757 28972 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.472821 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:43.474211 28969 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:24:43.489526 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.490242 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:43.450695 28860 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:43.505367 28812 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.505745 28973 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.506206 28860 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:43.506016 28812 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.506624 28860 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:43.507530 28860 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:43.484447 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71172.1us
I20260709 12:24:43.486215 26052 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:43.473412 28962 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:24:43.487216 28851 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:43.487684 28964 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.130s	user 0.001s	sys 0.001s
I20260709 12:24:43.487972 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2448) took 1887 ms. Trace:
I20260709 12:24:43.488452 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2451) took 1864 ms. Trace:
I20260709 12:24:43.478593 26051 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:43.533195 26048 rpcz_store.cc:276] 0709 12:24:41.600554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.954940 (+1354386us) service_pool.cc:224] Handling call
0709 12:24:43.020370 (+ 65430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.020696 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.020720 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.036823 (+ 16103us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.273785 (+236962us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:43.273824 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.273888 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.487760 (+213872us) raft_consensus.cc:1685] finished
0709 12:24:43.487781 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.487912 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10912,"spinlock_wait_cycles":47812480,"thread_start_us":167431,"threads_started":1,"wal-append.queue_time_us":359203}
I20260709 12:24:43.534268 26050 rpcz_store.cc:276] 0709 12:24:41.623626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.953272 (+1329646us) service_pool.cc:224] Handling call
0709 12:24:42.953629 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.953943 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.953974 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:42.954415 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.347710 (+393295us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:43.440098 (+ 92388us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.448992 (+  8894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.449097 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.488252 (+ 39155us) raft_consensus.cc:1685] finished
0709 12:24:43.488271 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.488399 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8545,"raft.queue_time_us":4605,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":18407,"spinlock_wait_cycles":52480,"thread_start_us":370410,"threads_started":2,"wal-append.queue_time_us":475078}
W20260709 12:24:43.495398 28966 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:24:43.495735 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 389) took 1652 ms. Trace:
I20260709 12:24:43.499100 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 415) took 1028 ms. Trace:
I20260709 12:24:43.500825 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1213) took 1112 ms. Trace:
I20260709 12:24:43.431466 28874 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.469766 28609 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:24:43.515988 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.547277 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.516433 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:43.517321 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.482213 28825 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.520419 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:43.573805 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53625.1us
I20260709 12:24:43.549455 28609 consensus_queue.cc:1243] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:43.575400 28952 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:43.577450 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.520912 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.481801 28953 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.482355 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:43.581637 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 99406.7us
I20260709 12:24:43.464943 28659 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.536826 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.536994 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:43.539497 26787 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:43.538643 26198 rpcz_store.cc:276] 0709 12:24:42.388702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.903049 (+514347us) service_pool.cc:224] Handling call
0709 12:24:42.903536 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.031314 (+127778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.500555 (+469241us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b54002b5f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:43.500767 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18181376}
I20260709 12:24:43.538542 26202 rpcz_store.cc:276] 0709 12:24:42.470579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.019970 (+549391us) service_pool.cc:224] Handling call
0709 12:24:43.020405 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.059392 (+ 38987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.059439 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.074076 (+ 14637us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.337941 (+263865us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:43.417345 (+ 79404us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.417929 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.418070 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.498856 (+ 80786us) raft_consensus.cc:1685] finished
0709 12:24:43.498880 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.499015 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25629,"prepare.queue_time_us":70099,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":25700,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":26475,"thread_start_us":176039,"threads_started":1,"wal-append.queue_time_us":278070}
I20260709 12:24:43.542894 26051 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:43.522991 28870 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.002s
I20260709 12:24:43.612737 28870 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.508742 28862 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.538482 26126 rpcz_store.cc:276] 0709 12:24:41.843426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.843674 (+   248us) service_pool.cc:224] Handling call
0709 12:24:41.844098 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.892356 (+ 48258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.892383 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.898908 (+  6525us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.013013 (+1114105us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b180045d0c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:43.214128 (+201115us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:43.214191 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.214285 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.495417 (+281132us) raft_consensus.cc:1685] finished
0709 12:24:43.495436 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.495665 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40658,"raft.queue_time_us":41164,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":37028,"spinlock_wait_cycles":10063104,"thread_start_us":200871,"threads_started":1,"wal-append.queue_time_us":1433783}
I20260709 12:24:43.518918 28814 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.519089 28956 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.485702 28700 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.557786 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:43.617727 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 298) took 1250 ms. Trace:
I20260709 12:24:43.618263 26051 rpcz_store.cc:276] 0709 12:24:42.367573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.542274 (+1174701us) service_pool.cc:224] Handling call
0709 12:24:43.542663 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.617441 (+ 74778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.617663 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:43.594333 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.553375 28860 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:43.606604 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.606492 26053 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.491118 28972 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:43.573216 28710 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.518s	user 0.005s	sys 0.006s
I20260709 12:24:43.609668 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.606411 26052 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:43.564157 28920 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.619720 26788 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.605715 28973 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:43.610131 28714 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.544077 28813 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.626550 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.630988 26120 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:24:43.619912 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.626935 26787 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:43.621097 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.632253 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.628376 28965 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:24:43.634613 28851 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.637073 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 408) took 1211 ms. Trace:
I20260709 12:24:43.637480 26196 rpcz_store.cc:276] 0709 12:24:42.425591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.938410 (+512819us) service_pool.cc:224] Handling call
0709 12:24:42.938741 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.967349 (+ 28608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.967398 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:42.968037 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.223151 (+255114us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:43.223201 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.223282 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.636802 (+413520us) raft_consensus.cc:1685] finished
0709 12:24:43.636819 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.637014 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24745,"raft.queue_time_us":29702,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":254649,"threads_started":1,"wal-append.queue_time_us":408289}
I20260709 12:24:43.639712 26198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:43.640345 26198 raft_consensus.cc:606] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.642170 28737 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:43.642850 28737 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:43.643278 28737 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.643836 26048 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:43.646528 26051 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:43.654873 26201 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:43.605474 28606 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:24:43.628126 28275 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.617976 28938 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 12:24:43.654299 28847 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.686s	user 0.006s	sys 0.006s
W20260709 12:24:43.668152 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.648545 28973 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:43.624101 28814 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.653419 28986 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.654019 28710 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:43.686002 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.682497 26051 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:24:43.694334 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:43.700650 26050 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:43.703608 26050 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 3
I20260709 12:24:43.703915 26050 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:24:43.704193 26050 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:43.685626 28925 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.039s	user 0.009s	sys 0.004s
I20260709 12:24:43.625146 28956 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.683835 28380 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:43.717869 28724 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:43.718197 28985 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:43.697126 28606 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 12:24:43.719245 28979 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:24:43.719372 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:43.707105 28986 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:43.818603 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99300us
I20260709 12:24:43.721493 28812 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.722075 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2438) took 1919 ms. Trace:
I20260709 12:24:43.822263 28812 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:43.822000 26122 rpcz_store.cc:276] 0709 12:24:41.802915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.803080 (+   165us) service_pool.cc:224] Handling call
0709 12:24:41.803572 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:41.808803 (+  5231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.808834 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:41.809380 (+   546us) log.cc:844] Serialized 52 byte log entry
0709 12:24:43.521493 (+1712113us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b1800311ec0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:43.609980 (+ 88487us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:43.610572 (+   592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.610618 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.721742 (+111124us) raft_consensus.cc:1685] finished
0709 12:24:43.721763 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.721996 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30293,"raft.queue_time_us":4900,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":16910,"spinlock_wait_cycles":3336576,"thread_start_us":86326,"threads_started":1,"wal-append.queue_time_us":1824573}
I20260709 12:24:43.722577 28993 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.824331 28993 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:43.825323 26168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.715s (SENT) [suppressed 31 similar messages]
I20260709 12:24:43.826460 28993 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:43.723647 28980 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:24:43.702586 28870 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:43.725579 28972 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.726886 28943 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.827739 28943 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.828140 28943 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.829828 28986 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.733958 28862 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.736739 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:43.833897 26202 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:43.738515 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 491) took 1202 ms. Trace:
I20260709 12:24:43.834959 26784 rpcz_store.cc:276] 0709 12:24:42.536187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.536568 (+   381us) service_pool.cc:224] Handling call
0709 12:24:42.537142 (+   574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.992552 (+455410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.738129 (+745577us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54004ffe54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:43.738429 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468927232}
I20260709 12:24:43.835860 26199 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:43.840678 26790 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:43.705264 28949 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.740460 26197 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.841517 28949 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.841894 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:43.842566 26783 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:43.837312 28973 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.838538 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 760) took 2237 ms. Trace:
I20260709 12:24:43.844105 26785 rpcz_store.cc:276] 0709 12:24:41.600786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.600979 (+   193us) service_pool.cc:224] Handling call
0709 12:24:42.927757 (+1326778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.043118 (+115361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.838328 (+795210us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003c5850. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:43.838479 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3050265984}
I20260709 12:24:43.839011 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:43.839673 28956 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.741425 28380 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:43.857682 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2434) took 2162 ms. Trace:
I20260709 12:24:43.859186 26127 rpcz_store.cc:276] 0709 12:24:41.692805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.694028 (+  1223us) service_pool.cc:224] Handling call
0709 12:24:41.703483 (+  9455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.257082 (+553599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.257151 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.257217 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:42.266200 (+  8983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.854826 (+1588626us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54001f54d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:43.854895 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.855092 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14886,"raft.queue_time_us":320552,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"spinlock_wait_cycles":603458432,"thread_start_us":261905,"threads_started":1}
I20260709 12:24:43.853199 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.863313 26199 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:43.866887 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.860765 28934 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.869369 28943 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.870288 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.871193 26201 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:43.871738 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.872378 26197 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:43.851583 28606 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:43.876714 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.855930 28724 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.741559 28925 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.856328 28878 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:24:43.880009 28606 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:43.900807 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 379917us
I20260709 12:24:43.913519 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:43.913380 28606 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.902294 28878 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:43.914408 28878 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:43.914770 28606 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.915299 28847 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.915422 28878 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.917057 28878 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.918172 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:43.918911 26196 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:43.903527 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.920150 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1205) took 1931 ms. Trace:
I20260709 12:24:43.920538 26203 rpcz_store.cc:276] 0709 12:24:41.988696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.625598 (+636902us) service_pool.cc:224] Handling call
0709 12:24:42.937039 (+311441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.951654 (+ 14615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.919977 (+968323us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400397e50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:43.920098 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715378048}
I20260709 12:24:43.921339 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.923957 28851 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.924669 28796 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:43.925460 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:43.926033 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 764) took 2231 ms (client timeout 1664 ms). Trace:
W20260709 12:24:43.926436 26786 rpcz_store.cc:269] 0709 12:24:41.694254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.714431 (+ 20177us) service_pool.cc:224] Handling call
0709 12:24:43.925818 (+2211387us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:43.925964 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2873834368}
I20260709 12:24:43.927724 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 772) took 2227 ms. Trace:
I20260709 12:24:43.928087 26791 rpcz_store.cc:276] 0709 12:24:41.699878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.956743 (+256865us) service_pool.cc:224] Handling call
0709 12:24:41.986827 (+ 30084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.532932 (+546105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.927488 (+1394556us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400660154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:43.927667 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179552512}
I20260709 12:24:43.929163 28796 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:43.932113 28796 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:43.934526 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2441) took 1600 ms. Trace:
I20260709 12:24:43.934964 26120 rpcz_store.cc:276] 0709 12:24:42.333951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.334094 (+   143us) service_pool.cc:224] Handling call
0709 12:24:42.365933 (+ 31839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.631360 (+1265427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.934296 (+302936us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:43.934464 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901889408}
I20260709 12:24:43.936213 28796 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:43.937286 28796 consensus_queue.cc:1243] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:43.937690 28726 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:43.938020 28726 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:43.938843 28726 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:43.940399 28726 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.947098 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:43.948856 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.952397 28940 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:43.954145 28940 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:43.954490 28940 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.954815 28940 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.956629 28933 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.894529 28659 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:43.962044 28821 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:43.962632 28821 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:43.962982 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:43.962795 28737 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.908675 28813 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.912302 28700 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:43.901826 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 278183us
I20260709 12:24:43.964511 26046 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:43.962674 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.987481 26053 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:43.988054 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2462) took 1001 ms. Trace:
I20260709 12:24:43.988269 26051 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:43.988394 26053 rpcz_store.cc:276] 0709 12:24:42.986316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.509261 (+522945us) service_pool.cc:224] Handling call
0709 12:24:43.988000 (+478739us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:43.988736 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.882637 28724 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.971907 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:43.990782 26203 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:43.990976 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:43.977154 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.991911 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 475542us
I20260709 12:24:43.962625 28940 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.982110 28821 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:43.995324 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:43.996141 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:43.996688 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.002835 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.005236 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:44.009532 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.011631 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.013280 28812 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:44.009069 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:43.993417 28724 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.015399 28813 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:44.015815 28813 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:44.016196 28813 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:44.009514 28878 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:44.020828 28878 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.019989 28853 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:44.023254 28878 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.023582 28853 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:44.024037 28853 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.024350 28825 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.027324 28813 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:43.944089 28949 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:44.034950 26045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2445) took 2435 ms (client timeout 1715 ms). Trace:
I20260709 12:24:43.948082 28925 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.038257 26787 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:44.035338 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.011545 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:44.040434 26045 rpcz_store.cc:269] 0709 12:24:41.599707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.673447 (+ 73740us) service_pool.cc:224] Handling call
0709 12:24:44.034703 (+2361256us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400229554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:44.034838 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3380945152}
I20260709 12:24:44.013041 26046 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:44.042393 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:44.043704 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.043778 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.047045 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.047626 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.050827 26048 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:44.054679 26203 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:44.038946 28987 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.353s	user 0.001s	sys 0.001s
I20260709 12:24:43.983052 28806 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:44.057399 26120 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:43.985507 28677 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:44.058246 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.056121 28949 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:44.037099 29007 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:24:44.010761 28700 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:44.008845 28312 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.040568 28993 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:44.071857 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.067008 28949 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:43.964886 28380 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.066862 28860 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:44.067085 28677 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.091179 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.088112 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.074642 28934 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:44.058905 29003 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:44.081991 28814 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.081163 26203 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:44.112962 26048 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 12:24:44.097138 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:44.086712 28956 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:44.120357 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.120916 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.121713 26785 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:44.119645 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:44.077607 29005 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:44.126647 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.124049 26783 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:44.124536 28748 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:44.066068 28853 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.129400 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.131207 26791 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:44.132192 26790 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEARNER]: Advancing to term 7
I20260709 12:24:44.135942 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.139137 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.142325 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:44.142545 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.143015 26786 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.143296 26789 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:44.093981 28943 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.098886 28677 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.143194 26784 pending_rounds.cc:85] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 4
I20260709 12:24:44.177969 26784 pending_rounds.cc:107] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:44.100047 28934 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:44.113897 28312 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:44.189622 28920 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.154069 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.192090 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.167927 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.128826 29003 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:44.149566 28956 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:44.180615 28714 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:44.132367 28997 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.183327 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.183722 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.196839 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.198736 26048 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:44.185457 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.186903 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.092489 28862 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:24:44.066977 28806 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:44.202109 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.178190 28813 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.182080 28888 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:44.193325 26201 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:44.095899 28993 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:24:44.149863 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.178615 26784 raft_consensus.cc:2889] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.190829 28934 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:44.197050 28953 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.003s
I20260709 12:24:44.118201 28996 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:24:44.149962 28748 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:44.210358 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1234) took 1041 ms. Trace:
I20260709 12:24:44.219048 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.219003 26050 rpcz_store.cc:276] 0709 12:24:43.168841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.846209 (+677368us) service_pool.cc:224] Handling call
0709 12:24:43.846593 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.846959 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.846988 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.847453 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.006156 (+158703us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:44.006197 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.006264 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.210161 (+203897us) raft_consensus.cc:1685] finished
0709 12:24:44.210182 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.210304 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1594,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":91,"thread_start_us":158303,"threads_started":1,"wal-append.queue_time_us":242361}
I20260709 12:24:44.202950 28986 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:44.228536 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.231776 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.209401 28479 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:44.199415 28920 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.216984 28860 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:44.219956 28821 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:44.243201 28748 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.244115 28860 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:44.248608 28978 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.627s	user 0.001s	sys 0.003s
I20260709 12:24:44.251159 28806 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.254009 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1216) took 1136 ms. Trace:
I20260709 12:24:44.254464 26194 rpcz_store.cc:276] 0709 12:24:43.117141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.158143 (+ 41002us) service_pool.cc:224] Handling call
0709 12:24:43.174581 (+ 16438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.219823 (+ 45242us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:43.250382 (+ 30559us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.435072 (+184690us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.446413 (+ 11341us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.446586 (+   173us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:43.446619 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.446688 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.843510 (+396822us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b48000b1b58. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:24:44.246094 (+402584us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:44.253773 (+  7679us) raft_consensus.cc:1685] finished
0709 12:24:44.253798 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.253935 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48967,"prepare.queue_time_us":37972,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":50318,"raft.run_cpu_time_us":2726,"raft.run_wall_time_us":630588,"spinlock_wait_cycles":58576384,"thread_start_us":103759,"threads_started":1,"wal-append.queue_time_us":222047}
W20260709 12:24:44.263443 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 66664.5us
W20260709 12:24:44.284862 29010 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:24:44.285763 29011 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:24:44.288017 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1237) took 1113 ms. Trace:
I20260709 12:24:44.288419 26049 rpcz_store.cc:276] 0709 12:24:43.175005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.922088 (+747083us) service_pool.cc:224] Handling call
0709 12:24:43.922446 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.922822 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:24:43.922851 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.923313 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.064002 (+140689us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:44.064426 (+   424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.065275 (+   849us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.287786 (+222511us) raft_consensus.cc:1685] finished
0709 12:24:44.287807 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.287953 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":139009,"threads_started":1,"wal-append.queue_time_us":161088}
I20260709 12:24:44.290241 28853 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.282785 29003 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.297132 29004 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:44.297818 29004 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:44.298704 29004 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.299813 29004 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:44.302739 28737 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.265731 28479 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:44.282881 28724 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.314309 28724 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.317124 26786 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:44.310410 28920 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.318648 28677 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:44.330896 26790 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:44.335301 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.328127 28479 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.327724 28714 consensus_queue.cc:1243] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:44.262876 28956 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.352641 26195 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:44.355895 28677 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.346939 28951 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:44.356720 28951 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:44.357806 28951 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.358805 28951 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:44.342788 28479 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.360684 26047 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:44.362267 28479 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:44.364502 28888 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:44.366382 28888 raft_consensus.cc:993] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:24:44.366781 28888 raft_consensus.cc:1081] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:24:44.367496 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2468) took 1090 ms. Trace:
I20260709 12:24:44.367617 28847 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:44.359845 28934 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.370105 28934 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.370504 26123 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:24:44.370877 26123 pending_rounds.cc:85] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 4
I20260709 12:24:44.370788 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:44.371240 26123 pending_rounds.cc:107] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:24:44.371516 26123 raft_consensus.cc:2889] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.371662 26126 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:44.372965 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:44.373469 28380 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:44.373808 26125 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:44.381562 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.367866 26053 rpcz_store.cc:276] 0709 12:24:43.277082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.989234 (+712152us) service_pool.cc:224] Handling call
0709 12:24:43.989521 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.989738 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:44.045900 (+ 56162us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.099867 (+ 53967us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.367032 (+267165us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b18003678a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:24:44.367165 (+   133us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.367213 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:44.367253 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.367327 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.367433 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16396,"prepare.queue_time_us":177497,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":269503,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":10067,"spinlock_wait_cycles":6419200,"thread_start_us":52193,"threads_started":1,"wal-append.queue_time_us":130885}
I20260709 12:24:44.384913 28949 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:44.385073 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:44.385586 28949 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:44.386109 26120 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:44.380687 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.387318 26127 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:44.387480 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 290421us
I20260709 12:24:44.388180 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:44.388978 26128 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:44.389564 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.398114 26124 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:44.401234 28940 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.403388 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1707) took 3784 ms. Trace:
I20260709 12:24:44.398165 26122 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:44.407233 26128 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:44.406687 26129 rpcz_store.cc:276] 0709 12:24:40.618905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.700022 (+ 81117us) service_pool.cc:224] Handling call
0709 12:24:42.686469 (+1986447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.745381 (+ 58912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.403207 (+1657826us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54001874d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:44.403331 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4567519616}
I20260709 12:24:44.409206 28897 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.409736 28897 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:44.419616 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 217576us
I20260709 12:24:44.406260 28312 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:44.424566 29005 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:44.425839 29014 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.430799 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.432298 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.432961 26199 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:44.434239 26201 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:44.435276 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.436046 26202 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:44.439023 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.425570 28312 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.444517 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.444344 28949 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.447009 28897 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.451352 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.454545 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:44.457027 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.418543 29029 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.457916 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.460690 26045 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:44.459972 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:44.416720 29001 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.475502 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.468580 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:44.462437 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44570.7us
W20260709 12:24:44.482366 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.484695 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.425627 29005 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:44.486503 28748 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.469340 28993 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:44.480024 28847 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:44.511159 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.457058 29000 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:44.484341 28933 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.479663 28940 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:44.472295 28897 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:44.494299 28479 raft_consensus.cc:993] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:24:44.480669 28950 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:24:44.492524 29005 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:44.557260 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.602267 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.614615 28796 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.615357 28796 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:44.670424 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.487197 29001 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.537464 28897 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.683375 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:44.684127 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 198883us
I20260709 12:24:44.698644 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.712726 26045 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:44.496315 28748 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.528332 28993 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.679932 28814 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.722721 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.722929 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.539805 28940 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.706676 28677 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:44.737476 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1230) took 1576 ms (client timeout 1739 ms). Trace:
W20260709 12:24:44.737905 26047 rpcz_store.cc:269] 0709 12:24:43.160647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.219958 (+ 59311us) service_pool.cc:224] Handling call
0709 12:24:44.737255 (+1517297us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:44.737413 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2620730112}
I20260709 12:24:44.732821 26046 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:44.733063 26048 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
W20260709 12:24:44.732926 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.704319 28897 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.706477 28843 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.710323 28934 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:44.742187 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.743676 26052 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:24:44.743623 26045 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:44.744190 26052 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 2
I20260709 12:24:44.744516 26052 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:24:44.744771 26052 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.748652 26049 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:24:44.751088 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.753671 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2463) took 1764 ms. Trace:
I20260709 12:24:44.530772 29000 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.754320 26052 rpcz_store.cc:276] 0709 12:24:42.989631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.626150 (+636519us) service_pool.cc:224] Handling call
0709 12:24:44.740765 (+1114615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.745190 (+  4425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.753464 (+  8274us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540008e8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:44.753611 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2533349760}
W20260709 12:24:44.762229 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77184.7us
I20260709 12:24:44.786332 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.752262 28479 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.746477 28934 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.792726 29005 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.835013 28825 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:44.740105 28814 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.916009 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.926677 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:44.925477 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.948711 26049 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:44.949710 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:44.950995 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:44.915644 26045 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:44.901732 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1233) took 1733 ms. Trace:
I20260709 12:24:44.905164 29001 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.906139 26194 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:44.952203 26051 rpcz_store.cc:276] 0709 12:24:43.168522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.704467 (+535945us) service_pool.cc:224] Handling call
0709 12:24:43.988008 (+283541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.988686 (+   678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.821489 (+832803us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54000f9950. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:44.901657 (+ 80168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587276544}
I20260709 12:24:44.952739 26785 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:24:44.909994 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.918643 26047 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:44.733453 29036 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.913326 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.914606 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:44.956564 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.956207 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.957329 26203 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.959131 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:44.959707 26202 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.947218 26201 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.966794 28790 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:44.967155 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.967401 28790 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.968144 26784 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:44.969192 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:44.969457 28790 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.969868 26787 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:44.971242 26790 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:44.972708 26788 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:44.972891 26786 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:44.973747 28812 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:44.974267 28812 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.974462 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:44.976024 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:44.976557 29036 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.976702 26127 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:44.957712 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.979254 29036 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:44.909047 28380 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.001s
I20260709 12:24:44.980422 26195 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:44.985702 26053 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:44.984215 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:44.987972 26054 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:44.990427 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.980556 26045 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.982600 26127 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 12:24:44.993507 26127 pending_rounds.cc:85] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 1
I20260709 12:24:44.993758 26127 pending_rounds.cc:107] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:44.994119 26127 raft_consensus.cc:2889] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.915426 28693 raft_consensus.cc:1081] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:24:44.999289 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:44.986871 26202 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
W20260709 12:24:44.934049 29028 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.488s	user 0.004s	sys 0.009s
I20260709 12:24:45.004063 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:44.982280 28479 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.006460 28993 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.006336 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.007499 26123 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.007606 26124 raft_consensus.cc:2410] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:45.007802 28479 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:45.008656 28999 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:45.009032 29044 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.009426 28999 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.012758 29041 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:45.012316 29042 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:45.013478 29041 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:45.014161 29014 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.588s	user 0.015s	sys 0.000s
I20260709 12:24:45.014605 29014 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.015307 26048 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:44.997565 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:44.922717 29000 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:44.994681 26129 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:24:44.946518 28934 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.983908 29003 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:44.911772 28933 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:45.009634 29044 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:44.976316 28806 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:24:44.923817 28796 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.019507 26129 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:45.020089 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.034933 26129 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.035632 26052 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:24:45.021773 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.994701 26127 pending_rounds.cc:107] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:24:45.037911 26127 raft_consensus.cc:2889] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:44.966431 28951 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.607s	user 0.014s	sys 0.000s
I20260709 12:24:45.022360 26199 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:45.024495 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:45.025527 28860 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.025954 29047 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:45.026337 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.017339 26122 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:45.020102 28999 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:44.930346 28897 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:44.996443 28994 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.032648 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:45.041451 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.072676 26124 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:45.044831 26127 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:45.047469 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:45.049722 26201 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:45.073925 26199 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 4.
I20260709 12:24:45.034060 28814 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.053470 26203 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:45.030165 28847 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:45.026957 28312 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.027679 28876 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.062994 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.053565 26129 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:45.061828 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 149004us
I20260709 12:24:45.066658 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.068634 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:45.070123 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:45.051841 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.076253 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.086606 26127 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:45.031730 29003 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:45.087409 29033 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.404s	user 0.003s	sys 0.000s
I20260709 12:24:45.061213 28796 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.090903 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.091212 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.091725 26124 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:45.092096 26790 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:45.034369 28700 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:24:45.106215 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73687.1us
I20260709 12:24:45.107220 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:45.107554 26046 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.044086 29001 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:24:45.041625 29036 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:24:45.062779 28956 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.030419 29014 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:24:45.029884 28933 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.089607 26788 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:45.097606 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.142518 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:45.098937 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:45.100365 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:45.145135 26126 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:45.147064 26126 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:45.100648 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:45.149627 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.089466 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:45.152582 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63186.6us
I20260709 12:24:45.111056 26129 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.112082 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:45.113302 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44769.8us
I20260709 12:24:45.154860 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:45.155398 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:45.115175 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.117722 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.118783 28934 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:45.120944 29034 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:24:45.122444 29044 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.123980 28843 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:45.124506 29046 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:45.124891 29047 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.126798 28938 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:45.127876 28677 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:45.128471 29003 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.132018 26045 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:45.132450 29041 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.082345 28847 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:45.157886 26124 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:45.189813 26126 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:45.155524 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:45.191186 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.109360 28951 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:45.192761 28956 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:45.236949 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.250769 26045 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
W20260709 12:24:45.252149 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72730.1us
W20260709 12:24:45.248394 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92960.8us
I20260709 12:24:45.255554 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.258540 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:45.219532 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.255443 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:45.289104 29003 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:45.290344 29003 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.292131 29003 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.292878 29003 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.295931 29003 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.296981 29003 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:45.105942 28876 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.191915 28860 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.189075 29014 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.190061 28312 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:45.328333 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 215906us
I20260709 12:24:45.107366 28796 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.238143 29044 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:45.336699 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:45.193380 28814 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:24:45.247197 28933 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.264564 28847 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.251029 28956 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.347597 26045 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7. Re-sending same reply.
I20260709 12:24:45.186537 28994 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 12:24:45.357659 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.358928 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.339600 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:45.344403 29048 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:24:45.264678 28951 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.338863 28312 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:45.378100 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:45.346712 29044 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.346299 28796 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.359869 28956 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.349238 29014 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.386092 28312 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.382064 29036 raft_consensus.cc:2764] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:45.387784 28973 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:45.416451 28934 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:45.417724 29044 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.434598 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:45.435972 26045 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.436980 29037 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:45.440984 29049 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:45.442873 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:45.443605 26050 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.443849 26200 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:45.443760 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:45.449025 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 181155us
I20260709 12:24:45.422039 28973 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.412774 29055 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:24:45.422863 28814 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.384774 29063 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:45.422653 28843 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.453987 28878 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:45.471196 28878 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.471690 28878 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.473474 26049 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:45.474689 26049 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.489751 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:45.492309 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 382139us
I20260709 12:24:45.496708 26050 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
W20260709 12:24:45.502316 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 243904us
I20260709 12:24:45.506587 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.503209 26050 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:45.470561 28951 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:45.454327 28897 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.460374 28973 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.459846 28860 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.481976 29037 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.511224 26045 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:45.468446 28812 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.525976 28888 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:45.527205 28812 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.527870 28951 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.528539 28951 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.502070 28934 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.532675 28934 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.537650 29014 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.464605 29064 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:24:45.516590 29047 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 12:24:45.541687 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.517294 29062 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:24:45.545269 28821 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.545789 28821 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.548017 28821 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.468833 28843 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.548650 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 625) took 2847 ms. Trace:
I20260709 12:24:45.549067 26782 rpcz_store.cc:276] 0709 12:24:42.701437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.289682 (+588245us) service_pool.cc:224] Handling call
0709 12:24:43.739538 (+449856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.739745 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:43.848322 (+108577us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.059688 (+211366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.059733 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:44.059783 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.059845 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.548464 (+1488619us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54004ffe50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:45.548588 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12947,"prepare.queue_time_us":71762,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":8202,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3601,"spinlock_wait_cycles":1033819904,"thread_start_us":210931,"threads_started":1,"wal-append.queue_time_us":242132,"wal-append.run_cpu_time_us":2099,"wal-append.run_wall_time_us":1192200}
I20260709 12:24:45.550413 28843 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.462435 28814 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.553037 28814 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.553377 28897 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:45.553900 28897 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.555929 28897 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.500059 29038 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:24:45.557512 28714 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:45.557905 28888 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.559082 28888 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.560393 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:45.558036 28714 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.561255 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:45.561887 26197 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:45.561573 28714 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.562422 29005 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.562647 28714 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.570194 29042 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.570691 29042 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.572989 29052 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:45.573642 29052 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:45.574854 29052 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.571666 29042 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.577155 29042 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.579210 28951 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.580075 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:45.581077 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55818us
I20260709 12:24:45.582259 29029 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:45.582669 29029 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.583031 29029 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.583865 29062 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:45.597119 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 90612us
I20260709 12:24:45.611828 26786 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:45.612023 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.613619 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.612998 26782 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 4.
I20260709 12:24:45.616653 26782 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:45.613376 29013 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.376s	user 0.008s	sys 0.005s
I20260709 12:24:45.628465 28973 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.629971 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.590054 28677 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.611404 28992 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:45.623919 26782 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:24:45.624878 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:45.625164 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.624698 28847 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.625974 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:45.632063 26786 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.632102 28677 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.632910 26787 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:45.627777 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:45.631211 28992 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.634905 29037 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.635991 28992 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:45.639986 26782 pending_rounds.cc:85] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
I20260709 12:24:45.640360 26782 pending_rounds.cc:107] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:45.640607 26782 raft_consensus.cc:2889] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:45.640632 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.014s	sys 0.005s
W20260709 12:24:45.641731 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.014s	sys 0.005s
I20260709 12:24:45.642473 28806 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:45.643292 29024 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.643054 28806 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.645255 28796 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.645648 28796 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:45.645941 28312 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.646505 28796 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.647006 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.646917 28956 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.647737 28796 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.648695 28312 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.648823 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:45.647701 26052 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:24:45.650295 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 394953us
I20260709 12:24:45.649816 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.649467 26049 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.653952 28677 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.657308 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:45.659128 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.661103 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.661739 26124 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:45.661856 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.662428 26129 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:45.662739 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.639564 26786 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:45.664601 26127 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.670253 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:45.671195 26121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:24:45.671870 26121 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:45.672420 26121 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:45.671396 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.013s	sys 0.038s
W20260709 12:24:45.673040 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.014s	sys 0.038s
I20260709 12:24:45.675014 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:45.645829 28806 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.622136 28951 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:45.684712 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:45.649458 28973 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.673756 28992 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:24:45.659989 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:45.688422 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.719189 26786 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:45.649160 28956 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.727713 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:45.681996 28312 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:45.669145 28999 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.730643 26784 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.731304 29036 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:45.658007 29046 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.680631 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.689334 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:45.643693 29024 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:45.726725 26782 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:24:45.744567 26782 pending_rounds.cc:85] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
I20260709 12:24:45.744959 26782 pending_rounds.cc:107] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:45.745249 26782 raft_consensus.cc:2889] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.727443 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:45.738161 26125 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:45.685505 29075 raft_consensus.cc:2764] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:45.682566 28950 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 12:24:45.748315 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59375.9us
I20260709 12:24:45.736341 29036 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:45.688798 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.775020 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.717437 29062 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:24:45.751647 28999 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:45.751288 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70781.2us
I20260709 12:24:45.751719 28951 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:45.683939 29029 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.752218 29047 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:45.691216 28479 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:24:45.772193 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1249) took 1045 ms. Trace:
I20260709 12:24:45.789000 26048 rpcz_store.cc:276] 0709 12:24:44.726859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.739870 (+ 13011us) service_pool.cc:224] Handling call
0709 12:24:44.909599 (+169729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.015764 (+106165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.771859 (+756095us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540022b354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:45.772111 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242678656}
I20260709 12:24:45.737900 29045 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:24:45.764400 28693 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:45.752108 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:45.776173 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.779866 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.773675 29059 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:45.726094 29072 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:24:45.785467 28950 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:45.781340 29023 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:24:45.781661 28951 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.776026 29036 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.781946 29075 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:24:45.797201 26782 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:45.797367 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.777877 28999 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:45.816956 26788 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.017s	sys 0.015s
W20260709 12:24:45.834924 26788 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.017s	sys 0.016s
I20260709 12:24:45.833534 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:45.816700 29059 consensus_queue.cc:1243] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:45.823781 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:45.825860 26784 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:45.829546 28874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.003s	sys 0.024s
I20260709 12:24:45.837352 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:45.820813 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 458989us
I20260709 12:24:45.816483 28312 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:45.820876 29072 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:24:45.838737 26754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.024s	sys 0.026s
W20260709 12:24:45.867952 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.024s	sys 0.027s
I20260709 12:24:45.820499 28950 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:45.839213 28806 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:45.873904 28806 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.821198 29046 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.867758 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:45.864804 28897 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:45.875923 28897 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.876351 28897 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.874332 28806 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.831571 28847 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:45.815877 28714 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:24:45.835896 28999 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.879726 28821 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.830430 28951 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.827334 28973 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 12:24:45.880604 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56948.5us
I20260709 12:24:45.813830 28693 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.795976 29024 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.877799 28992 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.903349 28821 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:45.827831 28878 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:45.904188 26051 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:45.904564 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.898828 26121 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:45.908850 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 465992us
I20260709 12:24:45.879638 29014 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:45.927201 26123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.019s
I20260709 12:24:45.952739 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:45.951783 26123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.019s
I20260709 12:24:45.892689 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:45.839972 29045 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:24:45.953541 26197 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.952500 29014 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:45.955220 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.956115 26052 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:45.956475 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:45.882642 28812 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:45.895426 29070 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:45.895658 28479 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:45.883138 28312 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.969040 26121 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:45.971714 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.974807 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:45.976925 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:45.979665 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:46.008976 26784 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:45.981747 26782 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:45.984306 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.984724 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:46.010408 26791 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:45.985244 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:45.985957 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:45.908056 28973 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:46.001878 29070 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.011195 26789 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.888824 29038 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 12:24:46.012830 26127 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:46.013391 26783 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:46.015480 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:46.019162 26200 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:46.030838 29081 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:45.905247 28950 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:46.372210 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.235061 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 451991us
I20260709 12:24:46.146214 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.173920 26789 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:46.051689 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:46.375370 26203 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:46.079126 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 121330us
I20260709 12:24:45.963766 28812 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:46.379132 28812 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:46.379484 29069 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.962899 28992 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.380630 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.381428 26195 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
W20260709 12:24:46.232470 26124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.008s	sys 0.008s
W20260709 12:24:46.382550 26124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.011s	sys 0.009s
W20260709 12:24:46.061017 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.078997 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 186405us
W20260709 12:24:46.384085 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.384516 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:46.242960 29055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.004s	sys 0.017s
I20260709 12:24:46.373997 26197 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:46.187199 29065 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:46.385445 29055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.005s	sys 0.019s
I20260709 12:24:46.391039 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:46.177812 26128 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:46.393056 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1728) took 1310 ms. Trace:
I20260709 12:24:46.410938 26123 rpcz_store.cc:276] 0709 12:24:45.083030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.083854 (+   824us) service_pool.cc:224] Handling call
0709 12:24:45.108686 (+ 24832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.108938 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:45.399766 (+290828us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.392621 (+992855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.392702 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:46.392756 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.392857 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.392996 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":438987,"prepare.queue_time_us":329953,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":439688,"spinlock_wait_cycles":178704256,"thread_start_us":843846,"threads_started":2,"wal-append.queue_time_us":633554}
W20260709 12:24:46.078941 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.393282 26129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.028s	sys 0.041s
W20260709 12:24:46.119578 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.414779 26129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.028s	sys 0.044s
I20260709 12:24:46.110205 29005 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.031675 29062 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:46.200395 28876 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:46.176097 28748 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:46.424702 29005 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:46.003785 29067 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.424s
I20260709 12:24:46.428925 29067 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:46.408504 29065 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:46.022837 28312 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.432313 29078 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:46.021507 28948 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.412s
I20260709 12:24:46.110699 29076 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:45.994997 29074 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.513s
I20260709 12:24:46.409384 29081 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.423698 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:46.189909 29024 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:46.031896 28821 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:46.037299 29070 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.455204 29081 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:46.461614 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:46.468556 28693 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:46.469806 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:46.471355 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:46.418133 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:46.473230 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55185.6us
I20260709 12:24:46.473505 29060 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:46.474049 29060 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.474462 29060 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:46.475713 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.476186 26197 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 7. Ops: [5.4-5.4]
I20260709 12:24:46.476827 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:46.477828 28999 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.001s
I20260709 12:24:46.478346 28999 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.479002 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.480715 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:46.481129 28796 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.812s	user 0.009s	sys 0.002s
I20260709 12:24:46.481845 29075 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:24:46.483188 28992 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:46.487375 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 102932us
I20260709 12:24:46.201650 28878 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:46.432493 28312 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:46.394393 28814 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.999178 28479 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:46.385963 29037 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.005s	sys 0.012s
W20260709 12:24:46.420836 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.443322 26786 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.040s	sys 0.035s
W20260709 12:24:46.507139 26786 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.040s	sys 0.035s
I20260709 12:24:46.452178 28993 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:46.452380 29038 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:46.452523 28876 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.036265 29071 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.450s
I20260709 12:24:46.466123 28940 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:46.013798 28973 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:46.130087 28677 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.339s
I20260709 12:24:46.433249 29065 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.467674 28934 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:46.429316 29005 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:46.491753 26203 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:46.572743 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:46.573716 26203 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:46.492089 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:46.576299 26195 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:46.426175 29062 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:46.496019 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:46.434473 29086 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:24:46.497387 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:46.577996 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 85977.9us
I20260709 12:24:46.393268 28380 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:46.502414 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:46.580178 26201 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:46.580425 26787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.022s	sys 0.022s
W20260709 12:24:46.582137 26787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.022s	sys 0.022s
I20260709 12:24:46.509750 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.434993 28847 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:46.433588 29026 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:46.433764 29056 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:46.517079 26788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.009s	sys 0.020s
I20260709 12:24:46.393913 28951 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.435712 29069 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.525413 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.525030 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:46.609221 26194 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.530550 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:46.497829 29076 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:46.571128 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.513945 29078 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:46.512388 29037 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.006s	sys 0.012s
I20260709 12:24:46.578608 26200 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:46.552268 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.625388 26202 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:46.581362 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:46.626447 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45212.4us
I20260709 12:24:46.594069 29065 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.593914 28973 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.514370 28814 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.594848 28972 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:46.595135 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:46.643760 26123 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:24:46.597116 26788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.009s	sys 0.020s
I20260709 12:24:46.517277 29055 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:46.597430 26203 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:46.591890 26201 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:46.498443 29070 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.610136 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.664760 26196 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:46.548766 28999 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:46.615393 26198 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 4.
I20260709 12:24:46.621264 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:46.515381 28876 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.629483 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.683005 26202 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:46.515540 28993 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.631912 26783 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:46.602237 28677 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:46.618465 29005 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:46.637198 28806 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.736s	user 0.009s	sys 0.002s
W20260709 12:24:46.633136 29047 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.012s	sys 0.012s
I20260709 12:24:46.630874 29071 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.600584 28380 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.638757 28790 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:46.609831 28847 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.616742 29026 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:46.687741 29036 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.855s	user 0.012s	sys 0.000s
I20260709 12:24:46.663733 29055 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:46.615880 29056 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.670908 29075 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:46.681028 29081 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:46.708180 29047 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.012s	sys 0.012s
I20260709 12:24:46.610378 28479 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.664366 28796 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:24:46.749130 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:46.750366 26789 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 5: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:46.751896 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:46.752907 26789 raft_consensus.cc:2368] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8.
I20260709 12:24:46.723335 28934 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.765692 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:46.734076 29046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.029s	sys 0.051s
I20260709 12:24:46.766747 26789 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7. Re-sending same reply.
I20260709 12:24:46.731124 28790 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:46.631208 29078 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:46.755493 28897 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.862s	user 0.009s	sys 0.003s
I20260709 12:24:46.712702 28806 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.661053 29062 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:46.771525 29046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.029s	sys 0.052s
I20260709 12:24:46.734324 29036 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.744557 29047 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:46.718922 29071 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.762243 28812 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.768399 28934 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.742353 29075 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.755919 29081 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.778905 28897 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:46.733610 29076 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.103430 28940 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:46.965297 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.004s	sys 0.006s
W20260709 12:24:47.115312 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.005s	sys 0.006s
W20260709 12:24:46.966255 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.004s
W20260709 12:24:47.116626 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.006s	sys 0.005s
I20260709 12:24:47.117424 29056 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.119066 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:24:46.966864 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.005s	sys 0.005s
W20260709 12:24:47.120527 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.005s	sys 0.005s
W20260709 12:24:47.121816 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.121944 29108 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:47.122670 29108 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:47.122852 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.971114 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1255) took 2045 ms. Trace:
I20260709 12:24:47.123776 26046 rpcz_store.cc:276] 0709 12:24:44.925343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.925631 (+   288us) service_pool.cc:224] Handling call
0709 12:24:44.925901 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.108104 (+182203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.970269 (+1862165us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540008e3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:46.970848 (+   579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384503680}
I20260709 12:24:47.124637 29106 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.971966 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.003s	sys 0.027s
I20260709 12:24:46.983300 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:47.125887 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.003s	sys 0.028s
I20260709 12:24:47.125663 29106 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:47.126384 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 747546us
I20260709 12:24:47.127305 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:47.124015 29108 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:46.689338 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:46.990141 26120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.012s	sys 0.024s
W20260709 12:24:47.130461 26120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.013s	sys 0.024s
W20260709 12:24:46.990837 26121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.004s	sys 0.021s
W20260709 12:24:46.991178 26122 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.000s	sys 0.032s
I20260709 12:24:47.131922 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:47.132519 26122 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.000s	sys 0.033s
I20260709 12:24:46.994247 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2474) took 1267 ms. Trace:
I20260709 12:24:47.133423 29071 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.133811 26127 rpcz_store.cc:276] 0709 12:24:45.726419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.196670 (+470251us) service_pool.cc:224] Handling call
0709 12:24:46.197107 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.394515 (+197408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.394562 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.395192 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.993276 (+598084us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:46.993394 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.993558 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.993688 (+   130us) raft_consensus.cc:1685] finished
0709 12:24:46.993725 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.993984 (+   259us) inbound_call.cc:177] Queueing success response
0709 12:24:47.109333 (+115349us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
Metrics: {"mutex_wait_us":196939,"raft.queue_time_us":219476,"raft.run_cpu_time_us":3585,"raft.run_wall_time_us":674908,"spinlock_wait_cycles":11691904,"thread_start_us":496415,"threads_started":1,"wal-append.queue_time_us":455204}
W20260709 12:24:46.995142 26129 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.003s	sys 0.012s
W20260709 12:24:47.136466 26129 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.004s	sys 0.013s
I20260709 12:24:46.994247 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 412) took 1203 ms. Trace:
I20260709 12:24:47.136940 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:46.999738 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.010425 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.018s	sys 0.033s
W20260709 12:24:47.010464 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.017226 26786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.013s	sys 0.029s
I20260709 12:24:47.018635 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.029850 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.030428 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 786) took 1042 ms. Trace:
I20260709 12:24:47.033092 28748 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:47.133632 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.033142 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.036347 28479 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.137658 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.035566 28790 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:47.159715 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.018s	sys 0.034s
I20260709 12:24:47.046994 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:47.159732 26786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.013s	sys 0.032s
I20260709 12:24:47.058352 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:47.063500 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.069347 25950 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 5, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 5 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:47.077855 28814 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.079064 28825 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.082418 28878 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.869167 28992 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:46.848224 29005 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:47.178043 26123 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:47.178395 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2464) took 1516 ms. Trace:
I20260709 12:24:46.870316 29023 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:47.179111 28897 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.179497 29023 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:46.920127 29026 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.180492 28843 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:46.843515 29029 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:46.894918 29036 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:46.826848 28951 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.095592 28876 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.182456 29077 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:46.882988 29052 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.184010 29119 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:47.184826 29119 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.184877 28878 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.187422 29119 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:46.780710 29055 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.187654 28878 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:46.922128 29060 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.191057 29060 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:46.907547 29062 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.192348 28938 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:47.193065 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:47.193851 28938 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:47.194281 28938 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:47.194999 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.098847 28934 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:46.837363 29069 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.196811 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:46.914565 29075 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:46.614377 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83974.6us
I20260709 12:24:47.197633 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:46.904857 29078 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:46.846834 29080 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.814s	user 0.004s	sys 0.007s
I20260709 12:24:46.917770 29081 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:46.865979 29083 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.832s	user 0.000s	sys 0.003s
I20260709 12:24:47.202673 26195 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:46.841733 29085 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 12:24:46.867209 29088 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.834s	user 0.007s	sys 0.007s
W20260709 12:24:46.887264 29089 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 12:24:46.902437 29090 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:24:46.848448 29092 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:24:47.107476 28847 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:46.847500 29094 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 12:24:46.893324 29096 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:24:46.840152 29097 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:24:46.852648 29098 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 12:24:47.239567 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.240675 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.241570 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 535) took 2086 ms. Trace:
I20260709 12:24:47.195552 28938 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.242879 28938 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:47.241932 26788 rpcz_store.cc:276] 0709 12:24:45.154574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.186983 (+ 32409us) service_pool.cc:224] Handling call
0709 12:24:45.187519 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.212528 (+ 25009us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:45.836405 (+623877us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.644648 (+808243us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.677186 (+ 32538us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.677401 (+   215us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:46.677458 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.677554 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.240976 (+563422us) raft_consensus.cc:1685] finished
0709 12:24:47.241180 (+   204us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.241373 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24985,"prepare.queue_time_us":819589,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":35629,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":29693,"thread_start_us":1278040,"threads_started":2,"wal-append.queue_time_us":829383}
W20260709 12:24:46.841002 29099 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 12:24:47.245658 25949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 284) took 22249 ms (client timeout 15000 ms). Trace:
I20260709 12:24:47.245925 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2475) took 1455 ms. Trace:
I20260709 12:24:47.246349 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2471) took 1556 ms. Trace:
W20260709 12:24:47.138043 26121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.005s	sys 0.021s
I20260709 12:24:47.249243 28796 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:47.249773 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 538) took 1267 ms. Trace:
I20260709 12:24:47.250201 26787 rpcz_store.cc:276] 0709 12:24:45.982893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.983087 (+   194us) service_pool.cc:224] Handling call
0709 12:24:45.983433 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.983850 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.983879 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.984445 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.582640 (+598195us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:46.582700 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.582801 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.249343 (+666542us) raft_consensus.cc:1685] finished
0709 12:24:47.249380 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.249587 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"thread_start_us":597738,"threads_started":1,"wal-append.queue_time_us":744483}
I20260709 12:24:47.131999 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 407) took 1475 ms. Trace:
I20260709 12:24:47.043042 28380 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:47.252375 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.004s	sys 0.009s
I20260709 12:24:47.252781 29071 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.253669 29071 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.256881 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:47.257193 26194 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.008s
I20260709 12:24:47.257716 26194 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:47.258108 28992 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:47.259735 29077 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.054386 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.260586 29077 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.261525 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 792) took 1270 ms. Trace:
I20260709 12:24:47.261937 26791 rpcz_store.cc:276] 0709 12:24:45.991355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.053662 (+1062307us) service_pool.cc:224] Handling call
0709 12:24:47.261434 (+207772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.137398 26128 rpcz_store.cc:276] 0709 12:24:45.791047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.438026 (+646979us) service_pool.cc:224] Handling call
0709 12:24:46.438835 (+   809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.439310 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.439358 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.439982 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.993050 (+553068us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:46.993393 (+   343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.993553 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.993705 (+   152us) raft_consensus.cc:1685] finished
0709 12:24:46.993731 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.994000 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35188,"raft.queue_time_us":941,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":456064,"threads_started":1,"wal-append.queue_time_us":326760}
I20260709 12:24:47.146627 26783 rpcz_store.cc:276] 0709 12:24:45.988412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.390708 (+402296us) service_pool.cc:224] Handling call
0709 12:24:47.029867 (+639159us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54006bf8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:47.030181 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496400128}
I20260709 12:24:47.135896 28973 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.263111 29077 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.263865 28825 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.266243 28825 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.146831 28940 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:47.252393 26120 rpcz_store.cc:276] 0709 12:24:45.656648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.656775 (+   127us) service_pool.cc:224] Handling call
0709 12:24:45.684889 (+ 28114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.685373 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.685420 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.686055 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.183831 (+497776us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b1800068440. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:47.131178 (+947347us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:47.131286 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.131407 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.131499 (+    92us) raft_consensus.cc:1685] finished
0709 12:24:47.131515 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.131901 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":372,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":28665,"spinlock_wait_cycles":4337024,"thread_start_us":946867,"threads_started":1,"wal-append.queue_time_us":1197909}
I20260709 12:24:47.177030 26785 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:47.176838 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.177296 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 789) took 1184 ms. Trace:
I20260709 12:24:47.267773 28940 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.268669 26789 rpcz_store.cc:276] 0709 12:24:45.989815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.767647 (+777832us) service_pool.cc:224] Handling call
0709 12:24:47.170922 (+403275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.269212 28940 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.269405 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2465) took 1605 ms. Trace:
I20260709 12:24:47.269644 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.270016 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 790) took 1279 ms. Trace:
I20260709 12:24:46.853307 29047 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.270473 26785 rpcz_store.cc:276] 0709 12:24:45.990373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.057785 (+1067412us) service_pool.cc:224] Handling call
0709 12:24:47.269957 (+212172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.191195 29065 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:47.170881 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.272782 29065 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:47.272816 26127 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:47.273396 29065 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:46.911662 29070 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.246409 26122 rpcz_store.cc:276] 0709 12:24:45.790702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.392461 (+601759us) service_pool.cc:224] Handling call
0709 12:24:46.393052 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.393290 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:47.133154 (+739864us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.245380 (+112226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.245496 (+   116us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:47.245643 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.245734 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.245861 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71941,"prepare.queue_time_us":423089,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2548,"thread_start_us":769977,"threads_started":2}
I20260709 12:24:47.178845 26129 rpcz_store.cc:276] 0709 12:24:45.663110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.664132 (+  1022us) service_pool.cc:224] Handling call
0709 12:24:45.664479 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.664687 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:45.674045 (+  9358us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.137047 (+1463002us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.169534 (+ 32487us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.175520 (+  5986us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:47.178012 (+  2492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.178136 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.178335 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52213,"prepare.queue_time_us":402854,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":91,"raft.queue_time_us":104897,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":6144,"spinlock_wait_cycles":20864,"thread_start_us":1437391,"threads_started":2,"wal-append.queue_time_us":1057595}
I20260709 12:24:47.233297 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.277455 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:47.234069 25950 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 5 to 7, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.279126 25950 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 5, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.177474 26782 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:47.168833 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 753202us
I20260709 12:24:47.164508 29106 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.277566 26195 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:47.281121 25950 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.249262 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2472) took 1544 ms. Trace:
W20260709 12:24:47.282713 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.287432 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:47.252806 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.009s
I20260709 12:24:47.267323 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:47.231971 26784 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:47.301033 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.308573 26782 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:47.309551 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 543) took 1323 ms. Trace:
W20260709 12:24:47.314354 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 144066us
I20260709 12:24:47.310735 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:47.283717 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:47.310004 26782 rpcz_store.cc:276] 0709 12:24:45.986006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.046389 (+1060383us) service_pool.cc:224] Handling call
0709 12:24:47.309495 (+263106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.322798 29038 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.323835 29038 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.325160 26784 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:47.326021 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 796) took 1331 ms. Trace:
I20260709 12:24:47.326447 26784 rpcz_store.cc:276] 0709 12:24:45.994486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.324351 (+1329865us) service_pool.cc:224] Handling call
0709 12:24:47.324797 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.325695 (+   898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.325940 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.246898 26125 rpcz_store.cc:276] 0709 12:24:45.689916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.826525 (+136609us) service_pool.cc:224] Handling call
0709 12:24:45.826837 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.834507 (+  7670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.834556 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.838240 (+  3684us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.199657 (+361417us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:24:46.371609 (+171952us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:46.371661 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.371747 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.245834 (+874087us) raft_consensus.cc:1685] finished
0709 12:24:47.245872 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.246199 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":196312,"raft.queue_time_us":2668,"raft.run_cpu_time_us":2555,"raft.run_wall_time_us":367121,"spinlock_wait_cycles":8117504,"thread_start_us":480141,"threads_started":1,"wal-append.queue_time_us":751045}
I20260709 12:24:47.333649 28814 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:47.334637 26787 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:47.334398 28814 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.334856 29118 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.335202 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 784) took 1348 ms. Trace:
I20260709 12:24:47.335628 26787 rpcz_store.cc:276] 0709 12:24:45.987358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.251955 (+1264597us) service_pool.cc:224] Handling call
0709 12:24:47.335122 (+ 83167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.335538 29118 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.337110 28814 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:47.338063 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.338169 29118 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:47.339107 28677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.015s
W20260709 12:24:47.339566 28677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.009s	sys 0.015s
I20260709 12:24:47.339856 28677 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:47.341138 28677 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.327153 29038 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.313678 28843 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:47.345901 28843 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.351125 28876 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.351236 29106 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:47.351688 26783 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:47.351914 28876 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.352288 29106 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:47.353714 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 86495.8us
I20260709 12:24:47.354272 29056 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:47.354972 29056 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.355896 29005 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:47.356575 29014 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.357203 29014 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.359758 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 544) took 1373 ms. Trace:
I20260709 12:24:47.269907 26124 rpcz_store.cc:276] 0709 12:24:45.663406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.663508 (+   102us) service_pool.cc:224] Handling call
0709 12:24:45.663837 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.670040 (+  6203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.670070 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.670550 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.382973 (+712423us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:46.383013 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.383077 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.269049 (+885972us) raft_consensus.cc:1685] finished
0709 12:24:47.269088 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.269243 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5778,"raft.queue_time_us":10235,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9178,"thread_start_us":665657,"threads_started":1,"wal-append.queue_time_us":1009439}
I20260709 12:24:47.365257 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.366693 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.368497 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.368191 29023 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.369611 29023 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:47.327338 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 669809us
I20260709 12:24:47.372572 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.373982 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.374109 29117 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:47.374683 26127 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:47.375675 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:47.376145 29090 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:24:47.345252 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:47.296679 26121 rpcz_store.cc:276] 0709 12:24:45.705394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.024674 (+319280us) service_pool.cc:224] Handling call
0709 12:24:46.025117 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.382484 (+357367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.382514 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.383095 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.248749 (+865654us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:47.248824 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.248947 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.249035 (+    88us) raft_consensus.cc:1685] finished
0709 12:24:47.249051 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.249201 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153239,"raft.queue_time_us":366919,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":3831,"thread_start_us":865143,"threads_started":1,"wal-append.queue_time_us":445662}
I20260709 12:24:47.378309 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.379124 28993 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.379312 26785 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:47.379498 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:47.363175 29114 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:24:47.363979 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.377259 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.362615 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.365682 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.390954 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.380085 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 793) took 1387 ms. Trace:
I20260709 12:24:47.411079 26783 rpcz_store.cc:276] 0709 12:24:45.986495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.263348 (+1276853us) service_pool.cc:224] Handling call
0709 12:24:47.351070 (+ 87722us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:47.351328 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.352168 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.359665 (+  7497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74573184}
I20260709 12:24:47.380503 26789 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:47.411815 26785 rpcz_store.cc:276] 0709 12:24:45.992714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.271224 (+1278510us) service_pool.cc:224] Handling call
0709 12:24:47.378703 (+107479us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:47.378939 (+   236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.379824 (+   885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.380034 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48555392}
I20260709 12:24:47.413723 26123 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:47.414675 26120 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:47.417230 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:47.417469 29106 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.379180 28806 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.418140 26046 raft_consensus.cc:2376] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8. Re-sending same reply.
I20260709 12:24:47.259652 28992 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.418838 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 783) took 1432 ms. Trace:
I20260709 12:24:47.419591 28992 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.419317 26789 rpcz_store.cc:276] 0709 12:24:45.986884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.269500 (+1282616us) service_pool.cc:224] Handling call
0709 12:24:47.379694 (+110194us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:24:47.379904 (+   210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.418547 (+ 38643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.418760 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52414720}
I20260709 12:24:47.419845 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:47.420693 26049 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.420809 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:47.384711 29038 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.422663 29014 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.423061 29038 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:47.420714 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.424516 26125 raft_consensus.cc:2410] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:47.425673 26791 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:47.426275 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 795) took 1432 ms. Trace:
I20260709 12:24:47.428395 29055 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.428592 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.429814 28814 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:47.430332 28814 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.430611 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:47.430768 28814 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.432883 26788 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:24:47.433342 26788 pending_rounds.cc:85] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 7
I20260709 12:24:47.433068 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:47.433624 26788 pending_rounds.cc:107] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:24:47.433923 26788 raft_consensus.cc:2889] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.434249 28951 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:47.434648 28951 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.434689 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 794) took 1441 ms. Trace:
I20260709 12:24:47.435109 28951 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.435585 28950 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.436661 28693 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:47.437054 28693 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:47.437417 28693 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:47.429723 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 146159us
I20260709 12:24:47.441959 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:47.442873 29067 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:47.443361 29067 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:47.445120 28951 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.445339 29067 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.446998 29056 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.447137 26790 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:47.447917 29069 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.448436 29069 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.450254 28876 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.452170 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.452344 26198 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:47.372970 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:47.453928 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81102.7us
I20260709 12:24:47.454353 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.455436 26129 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:47.456460 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.457264 26128 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.457727 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:47.458292 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:47.382534 29065 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.458956 26053 raft_consensus.cc:2376] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6. Re-sending same reply.
I20260709 12:24:47.459874 29065 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.458652 26046 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:47.462502 28814 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.466018 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 649) took 1739 ms. Trace:
I20260709 12:24:47.466504 26786 rpcz_store.cc:276] 0709 12:24:45.726419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.727881 (+  1462us) service_pool.cc:224] Handling call
0709 12:24:45.732710 (+  4829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.733100 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:46.507654 (+774554us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.188415 (+680761us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.190244 (+  1829us) log.cc:844] Serialized 564 byte log entry
0709 12:24:47.190457 (+   213us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:47.190497 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.190608 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.465768 (+275160us) raft_consensus.cc:1685] finished
0709 12:24:47.465794 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.465953 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14734,"prepare.queue_time_us":299346,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":23161,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":230,"thread_start_us":1412272,"threads_started":2,"wal-append.queue_time_us":390210}
I20260709 12:24:47.468894 26121 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.298436 28847 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:47.470263 28847 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.435117 26788 rpcz_store.cc:276] 0709 12:24:45.993225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.331453 (+1338228us) service_pool.cc:224] Handling call
0709 12:24:47.432377 (+100924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.434453 (+  2076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.434640 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.428334 26791 rpcz_store.cc:276] 0709 12:24:45.993951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.262858 (+1268907us) service_pool.cc:224] Handling call
0709 12:24:47.390748 (+127890us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:24:47.426214 (+ 35466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75376000}
I20260709 12:24:47.385807 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:47.456770 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 106167us
I20260709 12:24:47.452978 26124 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:47.486364 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.471940 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.501783 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.388937 29082 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:47.409543 29071 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.512317 26129 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:47.512599 26046 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:47.509498 25950 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.447865 28812 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.471027 29038 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:47.496619 28876 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.538043 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.541054 26123 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 5: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:47.507519 29082 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:47.511615 28380 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.473099 28693 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.547772 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:47.552845 26120 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:47.555444 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.576583 26049 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:47.574717 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.538561 25950 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 4, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130), now has a pending config: VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129), VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130), VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131), VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 4 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:47.568965 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:47.574169 28876 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.587368 29110 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:47.587602 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.587823 26045 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:47.588954 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.590727 26201 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:24:47.590914 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80820.2us
I20260709 12:24:47.591735 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:47.593026 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.593444 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:47.593884 26123 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:47.595114 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.595288 26123 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:24:47.595674 26123 pending_rounds.cc:85] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 2
I20260709 12:24:47.595944 26123 pending_rounds.cc:107] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:47.596293 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:47.596585 26125 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:47.597184 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1737) took 1165 ms. Trace:
I20260709 12:24:47.597507 26125 rpcz_store.cc:276] 0709 12:24:46.431480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.596051 (+1164571us) service_pool.cc:224] Handling call
0709 12:24:47.596406 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.596951 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.597113 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.598879 26125 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:47.605746 28693 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:47.606604 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:24:47.607290 28951 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.608268 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:47.608829 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1736) took 1177 ms. Trace:
I20260709 12:24:47.572813 29108 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:24:47.609287 26129 rpcz_store.cc:276] 0709 12:24:46.431203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.456123 (+1024920us) service_pool.cc:224] Handling call
0709 12:24:47.608596 (+152473us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b54001c3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:47.608771 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126383488}
I20260709 12:24:47.610692 28876 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:47.246170 25949 rpcz_store.cc:269] 0709 12:24:24.995625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.030275 (+ 34650us) service_pool.cc:224] Handling call
0709 12:24:47.245307 (+22215032us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:40.661036 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:41.148852 (+487816us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:41.188446 (+ 39594us) write_op.cc:435] Acquired schema lock
0709 12:24:41.220440 (+ 31994us) tablet.cc:662] Decoding operations
0709 12:24:41.461054 (+240614us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18001eb920. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b77dda16c 00007f5b77d9ac18 00007f5b7a29afa1
0709 12:24:41.729038 (+267984us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:41.729152 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:41.729183 (+    31us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:24:41.730406 (+  1223us) tablet.cc:701] Row locks acquired
0709 12:24:41.730451 (+    45us) write_op.cc:260] PREPARE: finished
0709 12:24:41.730656 (+   205us) write_op.cc:270] Start()
0709 12:24:41.730903 (+   247us) write_op.cc:276] Timestamp: P: 1783599881730586 usec, L: 0
0709 12:24:41.730929 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:24:41.898489 (+167560us) log.cc:844] Serialized 5164 byte log entry
0709 12:24:42.849823 (+951334us) op_driver.cc:464] REPLICATION: finished
0709 12:24:42.873119 (+ 23296us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a0e28 00007f5b77ddb581 00007f5b77ddad58 00007f5b77ddf9f1 00007f5b77ddf9a9 00007f5b77ddf939 00007f5b77ddf901 00007f5b77ddeb2d 0000000000388634 0000000000388479
0709 12:24:43.206924 (+333805us) write_op.cc:301] APPLY: starting
0709 12:24:43.207032 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:43.207292 (+   260us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:43.207330 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:43.305507 (+ 98177us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:43.306042 (+   535us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:24:43.306128 (+    86us) write_op.cc:312] APPLY: finished
0709 12:24:43.306738 (+   610us) log.cc:844] Serialized 95 byte log entry
0709 12:24:43.306975 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:43.307129 (+   154us) write_op.cc:454] Released schema lock
0709 12:24:43.307186 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:43.307631 (+   445us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:43.307660 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1553993,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":345745,"apply.run_cpu_time_us":5348,"apply.run_wall_time_us":100657,"num_ops":13,"prepare.queue_time_us":532149,"prepare.run_cpu_time_us":33259,"prepare.run_wall_time_us":2228873,"raft.queue_time_us":142671,"raft.run_cpu_time_us":43762,"raft.run_wall_time_us":183759,"replication_time_us":1118693,"spinlock_wait_cycles":97880192,"thread_start_us":1083596,"threads_started":4,"wal-append.queue_time_us":571384,"wal-append.run_cpu_time_us":5883,"wal-append.run_wall_time_us":1832623}]]}
I20260709 12:24:47.613610 29069 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.617590 28814 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:47.618706 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 637) took 2014 ms (client timeout 1990 ms). Trace:
I20260709 12:24:47.591684 28843 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:47.619156 26790 rpcz_store.cc:269] 0709 12:24:45.605317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.613314 (+  7997us) service_pool.cc:224] Handling call
0709 12:24:47.618494 (+2005180us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54004f79d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:47.618619 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4214327552}
I20260709 12:24:47.596203 26123 raft_consensus.cc:2889] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.620386 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.609210 26054 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:47.621047 26120 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:47.622004 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:47.622534 26120 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.625656 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:47.626654 28814 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:47.627015 28814 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.627333 28814 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.573073 28380 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.630285 28380 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:47.628485 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.630873 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.661374 26195 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
W20260709 12:24:47.662086 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.631268 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1732) took 1202 ms. Trace:
W20260709 12:24:47.663882 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.589781 26127 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:47.608876 26126 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 12:24:47.665403 26126 pending_rounds.cc:85] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 1
I20260709 12:24:47.616744 25950 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 4 to 5, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.663251 26123 rpcz_store.cc:276] 0709 12:24:46.422396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.594516 (+1172120us) service_pool.cc:224] Handling call
0709 12:24:47.594912 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.623673 (+ 28761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.625131 (+  1458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.666631 26126 pending_rounds.cc:107] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:47.667076 26126 raft_consensus.cc:2889] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.648730 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:47.670215 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107167us
I20260709 12:24:47.672125 26122 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:47.672840 26129 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:47.674659 26196 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:47.678056 25950 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 3, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.584586 28748 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:47.688354 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:47.685261 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.696631 26201 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:47.711457 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:47.713714 26198 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 12:24:47.719252 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 150413us
I20260709 12:24:47.720185 26789 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:47.722855 25950 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.730485 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.728766 26046 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:47.734122 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.683964 29117 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:47.705852 28748 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.736595 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.748621 26054 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:47.724694 28940 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:47.590734 28972 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.228s
I20260709 12:24:47.760170 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:47.737789 26049 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.749361 26202 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:24:47.773188 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.731911 28843 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:47.782418 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 189060us
I20260709 12:24:47.706342 28876 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:47.775799 25950 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 5 to 6, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 6 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.782804 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:47.789903 26045 pending_rounds.cc:85] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 2
I20260709 12:24:47.790333 26045 pending_rounds.cc:107] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:24:47.790649 26045 raft_consensus.cc:2889] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.791102 28714 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:47.791940 26052 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:47.792862 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:47.793499 26047 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.793677 28814 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.797039 26199 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:47.796252 26048 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:24:47.783176 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 267307us
W20260709 12:24:47.799342 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 250648us
I20260709 12:24:47.750074 29117 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:47.808821 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:47.759790 28796 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.818648 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 131921us
I20260709 12:24:47.825604 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.826565 26785 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:47.829989 26195 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
W20260709 12:24:47.833045 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.839398 26197 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:47.831703 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.846123 25950 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 5 to 6, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 6 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.803802 28814 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:47.797770 28847 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.804036 28940 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:47.852540 26046 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:47.808594 29117 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.854007 26790 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:47.869583 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.832649 29069 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.875097 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.870898 25950 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 3, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:47.862299 26167 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.661s (SENT)
I20260709 12:24:47.882483 26791 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:24:47.883795 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113344us
I20260709 12:24:47.846225 28812 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:47.908860 28843 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:47.909718 28843 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:47.912663 26195 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:47.916280 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:47.917958 26127 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
W20260709 12:24:47.918658 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.922279 29082 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.849653 28796 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.925134 29069 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:47.932394 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:47.933248 26051 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:47.889245 26786 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:47.888901 28940 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:47.891695 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51116.8us
I20260709 12:24:47.928390 26785 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEARNER]: Advancing to term 8
I20260709 12:24:47.895112 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:47.900556 29023 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.005s
I20260709 12:24:47.904383 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:47.935389 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:47.939637 26049 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:47.936537 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.948446 26047 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:47.948589 26128 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:24:47.949208 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.895213 29130 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:24:47.940744 26126 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.953786 26194 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:47.956133 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.924198 29082 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.941771 26120 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:47.960093 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:47.960366 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:47.963405 26127 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:47.963543 29064 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:47.964148 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 79538us
I20260709 12:24:47.964619 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:47.964994 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:47.965361 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:47.966383 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:47.966567 26784 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:24:47.968729 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 118634us
I20260709 12:24:47.968869 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:47.969285 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.969976 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:47.970197 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:47.970867 26786 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:47.971328 26167 leader_election.cc:400] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8.
I20260709 12:24:47.971621 26167 leader_election.cc:403] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:47.956934 28951 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:47.972102 28951 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.972426 29023 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:47.973166 29056 raft_consensus.cc:2764] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:47.957260 25950 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 6, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 6 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.967588 28677 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:47.972497 28951 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:47.978513 26121 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.000s
I20260709 12:24:47.979234 26121 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:47.960749 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:47.933272 29069 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.929000 28843 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:47.994004 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:47.927148 28796 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:47.990833 26051 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:47.996678 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:48.052376 26125 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:48.053783 26049 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:48.057027 26202 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:47.963650 29037 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:48.062179 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71685.5us
W20260709 12:24:48.063170 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91998.5us
I20260709 12:24:48.025597 25950 catalog_manager.cc:5697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.070866 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:47.990374 29108 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:48.079754 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.085641 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116832us
I20260709 12:24:48.003059 29065 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:48.090569 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 95756.6us
I20260709 12:24:48.093711 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:48.102614 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.101621 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:48.113273 25950 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 5 to 6, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 6 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:48.094818 29114 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:24:48.116459 26049 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:24:48.117167 29047 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:48.116529 29108 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:48.088856 28992 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:48.091859 29065 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.060683 29023 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:48.120522 29047 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.122148 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.112934 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:48.055564 29056 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:24:48.105423 29069 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:48.129606 25950 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 4, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:48.132596 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.134690 26049 raft_consensus.cc:2376] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5. Re-sending same reply.
I20260709 12:24:48.137633 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.138815 26126 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:48.140480 26046 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:48.133463 29075 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8.
I20260709 12:24:48.141875 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.143144 29037 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:48.143729 29037 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:48.144353 29037 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:48.144937 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.146687 29055 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:48.147416 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.148198 28812 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:48.148720 28812 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:48.149158 28812 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.150574 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.150733 29055 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:48.153123 28714 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:48.152664 25950 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:48.153780 28950 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.718s	user 0.009s	sys 0.003s
I20260709 12:24:48.153661 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.124740 28972 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:48.154681 28847 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:48.126824 28992 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.129562 29047 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.129901 29064 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:48.127732 29065 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.113598 29110 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 12:24:48.172845 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.169062 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.168506 29056 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:48.171124 25950 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 4 to 7, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.155154 28897 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.159725 28972 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.102365 28843 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:24:48.166134 28847 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:48.167068 28992 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:48.196815 26128 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:48.210947 26051 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:24:48.185134 29056 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.188783 29038 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:48.221730 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:48.222486 26126 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.191843 28972 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:48.188105 28938 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:48.214426 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:48.224309 25950 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 5, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.225703 26785 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:48.220134 29056 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.196100 28847 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:48.246272 25950 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 4 to 8, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 8 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.189813 29023 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.248080 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.192044 28897 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.217115 29055 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.257028 26046 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:24:48.246059 28796 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:48.244930 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75902us
I20260709 12:24:48.252996 25950 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 4 to 5, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:48.254841 26124 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.760s	user 0.011s	sys 0.001s
I20260709 12:24:48.173002 29108 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.230816 28938 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:48.238938 29128 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.418s	user 0.005s	sys 0.001s
I20260709 12:24:48.257705 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:48.258051 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:48.229715 28876 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:48.266273 28714 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:24:48.271363 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.262645 26124 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:48.276506 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2476) took 1273 ms. Trace:
I20260709 12:24:48.277515 26124 rpcz_store.cc:276] 0709 12:24:47.003485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.451878 (+448393us) service_pool.cc:224] Handling call
0709 12:24:48.276336 (+824458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:48.283723 26126 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:48.255627 29067 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:48.289812 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.285333 25950 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 6, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 6 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.288480 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:48.255430 28950 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:48.292848 26123 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:48.306191 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.267812 28938 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.267177 28973 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:48.251197 28843 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:48.260543 28796 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:48.330843 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.386526 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:48.397500 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:48.399855 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:48.401086 26121 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:48.397043 25950 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 2 to 5, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.417321 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:48.417675 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 156572us
I20260709 12:24:48.420226 26052 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:48.401356 28973 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:48.421702 25950 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 4, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:48.431737 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.423429 28825 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.436683 28825 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:48.440893 29141 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 12:24:48.439858 28825 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:48.425936 28876 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:48.458559 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 303967us
W20260709 12:24:48.461694 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.457947 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 394819us
W20260709 12:24:48.463583 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 192411us
I20260709 12:24:48.474794 26052 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:48.476099 26121 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:24:48.476660 26121 pending_rounds.cc:85] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 2
I20260709 12:24:48.476943 26121 pending_rounds.cc:107] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:48.477260 26121 raft_consensus.cc:2889] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:48.469746 26046 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.477056 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.467705 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:48.498968 26168 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.630s (SENT)
I20260709 12:24:48.462795 29069 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:48.520606 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108516us
I20260709 12:24:48.530406 26052 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:48.534374 29023 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:48.542557 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.493990 28796 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:48.566793 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92455us
I20260709 12:24:48.529326 28940 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:48.568018 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:48.562040 28748 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.004s
I20260709 12:24:48.547400 29166 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:48.580996 26052 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:48.565604 28796 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.579281 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:48.570415 29117 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.005s
I20260709 12:24:48.608856 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 656) took 1026 ms. Trace:
I20260709 12:24:48.536454 29067 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:48.620312 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.618291 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.570943 29023 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.575495 29062 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:48.645591 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.635249 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:48.653622 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.634287 26052 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:24:48.662420 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 194145us
W20260709 12:24:48.665593 26168 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.726s (SENT)
I20260709 12:24:48.599175 29166 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:48.604846 29151 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:48.675015 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 254571us
I20260709 12:24:48.677167 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.609377 26783 rpcz_store.cc:276] 0709 12:24:47.572652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.578025 (+  5373us) service_pool.cc:224] Handling call
0709 12:24:47.594438 (+ 16413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.595016 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:47.803605 (+208589us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.174087 (+370482us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.174691 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.174869 (+   178us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:48.174907 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.174974 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.598595 (+423621us) raft_consensus.cc:1685] finished
0709 12:24:48.598618 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.598766 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51106,"prepare.queue_time_us":329128,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":290249,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":12811,"thread_start_us":435057,"threads_started":2,"wal-append.queue_time_us":576889}
I20260709 12:24:48.646862 29152 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.690037 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.695968 26785 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:48.702240 29149 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:24:48.701922 29067 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.657420 28825 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.658810 29062 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:48.570608 28973 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.604642 28748 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.605468 28940 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.729970 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 417) took 1628 ms. Trace:
I20260709 12:24:48.732671 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:48.737749 26785 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:48.737463 26123 rpcz_store.cc:276] 0709 12:24:47.102050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.713432 (+611382us) service_pool.cc:224] Handling call
0709 12:24:47.734116 (+ 20684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.299973 (+565857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.729675 (+429702us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54002cabd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:48.729902 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284302080}
I20260709 12:24:48.621925 29117 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:48.743459 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.753604 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2485) took 1686 ms. Trace:
I20260709 12:24:48.692629 29151 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:48.754220 26125 rpcz_store.cc:276] 0709 12:24:47.066936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.599737 (+532801us) service_pool.cc:224] Handling call
0709 12:24:47.996706 (+396969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.055777 (+ 59071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.753355 (+697578us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54001f54d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:48.753522 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912190336}
I20260709 12:24:48.770781 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.773037 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.777645 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.782891 26120 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:48.784430 26122 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:48.795348 26129 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:48.778306 29150 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 12:24:48.817188 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.728746 28825 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.675460 29069 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.774976 29118 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.868217 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81889.1us
W20260709 12:24:48.872877 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.895804 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.712863 29152 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:48.896504 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.873734 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:48.897758 26125 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:48.869042 26045 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:48.887189 26052 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:48.900135 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:48.883513 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 183026us
I20260709 12:24:48.816252 28693 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.915616 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:48.930992 26790 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:48.931947 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.932783 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:48.938330 26045 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:48.910462 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 134602us
I20260709 12:24:48.945690 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:48.901930 28876 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:48.953831 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:48.940975 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:48.907251 29118 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.959327 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.960669 28843 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.961205 28843 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.961544 29118 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:48.962330 29118 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:48.963786 28479 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:48.830832 29062 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.964864 29062 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:48.966396 28812 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.817s	user 0.010s	sys 0.001s
I20260709 12:24:48.966861 28812 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.968456 29155 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:48.968912 29179 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:48.969398 29179 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:48.969350 29155 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.969877 28876 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.970680 29179 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.905795 29168 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 12:24:48.952980 28992 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.002s
I20260709 12:24:48.917411 28693 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.972389 28677 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.981686 28677 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:48.972707 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:48.881487 29166 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:48.971910 29155 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:48.973161 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:48.979985 26052 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:24:48.977321 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 331189us
I20260709 12:24:48.985523 26046 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:48.987196 26045 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:48.997678 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:48.980273 26125 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:48.998342 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:48.999482 29147 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.518s	user 0.000s	sys 0.005s
W20260709 12:24:48.973049 29170 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:24:49.001549 28825 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:48.998425 26045 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:48.991372 26053 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:48.935791 29082 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.947573 29146 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:24:49.002393 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:48.973125 29161 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 12:24:48.973690 29159 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:24:49.000123 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:49.024475 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 156261us
I20260709 12:24:49.024878 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2488) took 1199 ms. Trace:
I20260709 12:24:48.975723 28748 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:48.976416 29117 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:48.980872 28812 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:48.975144 29152 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:49.033470 28951 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.022s	user 0.015s	sys 0.002s
W20260709 12:24:49.036403 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62882.3us
W20260709 12:24:49.038143 28847 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.773s	user 0.010s	sys 0.001s
I20260709 12:24:49.038259 29176 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.976790 28940 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:48.972571 29175 raft_consensus.cc:2764] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:49.039301 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:49.043480 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:49.063776 29153 raft_consensus.cc:2764] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:49.064622 29156 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:49.022392 29151 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.065070 29156 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.065469 29156 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.066964 28843 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.067971 29187 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:49.027029 29082 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.057192 28951 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.071238 26784 raft_consensus.cc:2368] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:24:49.071336 29082 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.072268 28951 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:49.059926 28748 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:49.059648 28847 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.059509 28693 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.074256 29172 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:49.074846 29172 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:49.075103 28847 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:49.075783 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 450) took 1310 ms. Trace:
I20260709 12:24:49.075831 29172 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.076417 26196 rpcz_store.cc:276] 0709 12:24:47.765162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.765970 (+   808us) service_pool.cc:224] Handling call
0709 12:24:47.766429 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.852667 (+ 86238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.852697 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.898278 (+ 45581us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:47.898911 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.187697 (+288786us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:48.187744 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.187813 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.075380 (+887567us) raft_consensus.cc:1685] finished
0709 12:24:49.075402 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.075692 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82556,"raft.queue_time_us":70802,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":9244,"spinlock_wait_cycles":3115392,"thread_start_us":288322,"threads_started":1,"wal-append.queue_time_us":552565}
I20260709 12:24:49.077766 29164 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:49.081183 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.081076 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.081868 26197 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:48.972199 29179 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:49.045181 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91464.9us
I20260709 12:24:48.990505 28950 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.025354 29118 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:49.033471 29056 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.001s
I20260709 12:24:49.029538 26048 rpcz_store.cc:276] 0709 12:24:47.825887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.826030 (+   143us) service_pool.cc:224] Handling call
0709 12:24:47.873076 (+ 47046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.905395 (+ 32319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.905442 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.905996 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.197581 (+291585us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:48.197625 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.197773 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.998731 (+800958us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:49.001943 (+  3212us) raft_consensus.cc:1685] finished
0709 12:24:49.001963 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.024784 (+ 22821us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3484,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":16448896,"thread_start_us":291152,"threads_started":1,"wal-append.queue_time_us":519727}
I20260709 12:24:49.027089 29110 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:49.028672 29155 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:49.055932 26168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.750s (SENT) [suppressed 13 similar messages]
W20260709 12:24:49.060585 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60547.3us
I20260709 12:24:49.031160 29189 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.086465 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47242.8us
I20260709 12:24:48.979205 28992 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.094465 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.132261 29185 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.132656 26202 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:49.132959 29185 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.047127 29166 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:49.096616 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.118358 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.136047 26196 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:49.136265 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.079523 28972 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.828s	user 0.007s	sys 0.006s
I20260709 12:24:49.137051 28972 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.139424 28972 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.043973 29162 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:24:49.131196 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:49.123901 29176 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:49.130983 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75093.6us
I20260709 12:24:49.135322 29185 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.120551 28748 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:49.151540 26199 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:49.100720 29164 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:49.122426 29178 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:49.159709 26196 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:49.125787 28714 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.156044 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 224542us
I20260709 12:24:49.129684 29155 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.182252 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.182590 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.183496 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.186666 26046 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:49.152190 28479 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.126514 29056 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.197659 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.154456 28847 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:24:49.204012 28748 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.177646 29155 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.208704 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.209244 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:49.203163 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 299906us
I20260709 12:24:49.217362 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:49.227325 26050 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:49.228991 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:49.146538 29189 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.219900 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.213085 29062 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.258558 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.260942 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43680.2us
W20260709 12:24:49.260063 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81987.8us
W20260709 12:24:49.262797 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53629.1us
I20260709 12:24:49.182127 28714 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:49.260808 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.267328 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.216296 29108 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:49.194739 28479 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.270077 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:49.257798 29110 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:49.215847 28973 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:24:49.268584 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
W20260709 12:24:49.281402 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 86870.8us
I20260709 12:24:49.284641 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:49.288276 26196 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:49.257588 29190 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:49.265035 28677 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:49.277779 29181 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:24:49.269456 29151 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:49.265733 28693 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:49.273674 29162 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:24:49.265540 29062 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.308957 26201 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:49.339118 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.345368 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2502) took 1124 ms. Trace:
I20260709 12:24:49.346643 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:49.348263 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.275784 29194 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.304826 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.345950 26125 rpcz_store.cc:276] 0709 12:24:48.220454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.064192 (+843738us) service_pool.cc:224] Handling call
0709 12:24:49.064599 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.099321 (+ 34722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.099381 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.142566 (+ 43185us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.258995 (+116429us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:49.259053 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.259255 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.345049 (+ 85794us) raft_consensus.cc:1685] finished
0709 12:24:49.345085 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.345267 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41341,"raft.queue_time_us":34758,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5575,"thread_start_us":70036,"threads_started":1,"wal-append.queue_time_us":115354}
I20260709 12:24:49.281255 29157 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.349313 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2489) took 1464 ms. Trace:
I20260709 12:24:49.299867 29110 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.305042 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 797) took 3310 ms. Trace:
I20260709 12:24:49.353025 26048 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:24:49.353276 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83063.5us
I20260709 12:24:49.353780 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.282315 29180 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.363129 26202 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:49.315634 29198 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:24:49.340400 29192 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:24:49.363631 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:49.344024 29047 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.152s	user 0.011s	sys 0.000s
W20260709 12:24:49.342482 29037 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.197s	user 0.011s	sys 0.004s
I20260709 12:24:49.306413 29175 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:49.361827 26782 rpcz_store.cc:276] 0709 12:24:45.996080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.330164 (+1334084us) service_pool.cc:224] Handling call
0709 12:24:47.505173 (+175009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.564202 (+ 59029us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:47.842782 (+278580us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.385818 (+543036us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.980347 (+594529us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b1800433480. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:49.302452 (+322105us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b1800433480. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287
0709 12:24:49.304571 (+  2119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.304642 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:49.304707 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.304797 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.304963 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":364744,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":330531,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":14595,"spinlock_wait_cycles":70770816,"thread_start_us":785179,"threads_started":3,"wal-append.queue_time_us":1132561}
I20260709 12:24:49.289737 29164 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.362978 28951 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:49.370608 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:49.412679 26202 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:49.371196 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:49.371557 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.414199 26046 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:49.415299 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:49.416070 26046 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.363214 28693 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:49.372473 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.373147 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2501) took 1182 ms. Trace:
I20260709 12:24:49.373028 29180 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.373821 28714 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:49.373734 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:49.374282 29067 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:49.376060 29200 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:24:49.377229 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.372345 28843 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:49.363981 29151 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:49.357762 26054 rpcz_store.cc:276] 0709 12:24:47.888345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.932269 (+ 43924us) service_pool.cc:224] Handling call
0709 12:24:47.932698 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:47.933169 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.933200 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:49.138422 (+1205222us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b2c000014c8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:49.139181 (+   759us) log.cc:844] Serialized 52 byte log entry
0709 12:24:49.203575 (+ 64394us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:49.254787 (+ 51212us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.265985 (+ 11198us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.273208 (+  7223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.273302 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.349073 (+ 75771us) raft_consensus.cc:1685] finished
0709 12:24:49.349102 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.349254 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26886,"prepare.queue_time_us":57579,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":6164,"raft.queue_time_us":25746,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":13248,"spinlock_wait_cycles":55381120,"thread_start_us":63801,"threads_started":1,"wal-append.queue_time_us":125195}
I20260709 12:24:49.439968 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.440914 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:49.442085 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.387993 29056 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.388288 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:49.364377 29157 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.389295 29037 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.390355 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.443307 26053 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:49.332520 29108 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.063s
W20260709 12:24:49.390982 29055 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.126s	user 0.013s	sys 0.000s
W20260709 12:24:49.391419 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 106848us
W20260709 12:24:49.391642 28796 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.000s
I20260709 12:24:49.337327 29189 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:49.283231 29185 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
W20260709 12:24:49.392542 29205 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:24:49.391023 29047 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.309176 29190 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.394708 28479 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:49.395294 29175 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.367054 29110 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:49.396733 29065 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.203s	user 0.014s	sys 0.002s
W20260709 12:24:49.398733 28938 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.999s	user 0.012s	sys 0.000s
I20260709 12:24:49.396142 29155 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.403497 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2500) took 1222 ms. Trace:
I20260709 12:24:49.452577 29055 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:49.340590 28992 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:49.424551 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.473845 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101500us
W20260709 12:24:49.474804 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.433279 26125 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:49.433753 26046 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:49.418959 26123 rpcz_store.cc:276] 0709 12:24:48.191208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.738836 (+547628us) service_pool.cc:224] Handling call
0709 12:24:48.739086 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.996088 (+257002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.996160 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.027232 (+ 31072us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.155533 (+128301us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:49.155594 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.155698 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.372939 (+217241us) raft_consensus.cc:1685] finished
0709 12:24:49.372958 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.373089 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69364,"raft.queue_time_us":25071,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":7147,"thread_start_us":318924,"threads_started":2,"wal-append.queue_time_us":176531}
I20260709 12:24:49.443068 26048 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:49.356359 29178 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:24:49.476598 29178 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:49.477010 29178 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:49.478144 26048 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:49.477903 29178 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.479259 29178 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.476598 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:49.445562 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:49.451809 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.453547 26054 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:24:49.435885 29152 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:49.460011 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.482970 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:49.483526 26200 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:49.463791 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:49.466228 26053 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:49.489943 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:49.360594 29194 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.467727 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.498253 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:49.446064 28843 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:49.470389 26128 rpcz_store.cc:276] 0709 12:24:48.181524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.710286 (+528762us) service_pool.cc:224] Handling call
0709 12:24:48.742701 (+ 32415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.783626 (+ 40925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.783655 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.870626 (+ 86971us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.161440 (+290814us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:49.161509 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.161609 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.403247 (+241638us) raft_consensus.cc:1685] finished
0709 12:24:49.403269 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.403432 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70268,"raft.queue_time_us":28292,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9825,"thread_start_us":224780,"threads_started":1,"wal-append.queue_time_us":356044}
I20260709 12:24:49.496964 26200 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:49.499176 26202 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:49.504722 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:49.505505 26200 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.506841 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:49.507229 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:49.465554 29175 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.512275 26199 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:49.512856 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:49.513628 26125 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:49.518395 29055 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.508633 29152 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:49.519089 29055 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.519428 29152 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.521957 29055 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.527104 29155 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:49.515407 29179 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.530938 29179 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:49.518878 29209 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:49.537793 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:49.541445 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:49.543916 26200 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:49.547349 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2495) took 1326 ms. Trace:
I20260709 12:24:49.502583 28973 raft_consensus.cc:2764] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:49.554234 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:49.470417 29065 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:49.562054 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72165.8us
I20260709 12:24:49.472602 28938 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:49.603441 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.604503 29037 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.547924 26051 rpcz_store.cc:276] 0709 12:24:48.221348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.223711 (+  2363us) service_pool.cc:224] Handling call
0709 12:24:48.249118 (+ 25407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:48.416489 (+167371us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:48.730546 (+314057us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.929538 (+198992us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:48.958308 (+ 28770us) log.cc:844] Serialized 52 byte log entry
0709 12:24:48.959178 (+   870us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:48.989798 (+ 30620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.989900 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.547053 (+557153us) raft_consensus.cc:1685] finished
0709 12:24:49.547084 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.547269 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21590,"prepare.queue_time_us":57179,"prepare.run_cpu_time_us":4683,"prepare.run_wall_time_us":11778,"raft.queue_time_us":348077,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":25771,"thread_start_us":514903,"threads_started":3,"wal-append.queue_time_us":339479}
W20260709 12:24:49.607234 29023 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.949s	user 0.012s	sys 0.001s
I20260709 12:24:49.607847 29023 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:49.608585 26196 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:49.609100 26196 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:49.610098 26196 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.611459 26196 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:49.555269 29155 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:49.572836 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70833.9us
I20260709 12:24:49.573956 26201 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:49.576551 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 423) took 2076 ms. Trace:
I20260709 12:24:49.577520 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 670) took 1452 ms. Trace:
I20260709 12:24:49.617798 29179 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.618129 26784 rpcz_store.cc:276] 0709 12:24:48.125201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.125368 (+   167us) service_pool.cc:224] Handling call
0709 12:24:49.577366 (+1451998us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54004f79d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:49.577465 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144694016}
W20260709 12:24:49.579383 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1270) took 2289 ms (client timeout 1651 ms). Trace:
I20260709 12:24:49.580859 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:49.620700 26194 rpcz_store.cc:269] 0709 12:24:47.297409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.297601 (+   192us) service_pool.cc:224] Handling call
0709 12:24:49.579162 (+2281561us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:49.579258 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223560960}
I20260709 12:24:49.621526 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:49.622241 26194 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:49.622390 29194 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:49.623361 29154 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.405s	user 0.000s	sys 0.003s
I20260709 12:24:49.616982 26129 rpcz_store.cc:276] 0709 12:24:47.500603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.773855 (+273252us) service_pool.cc:224] Handling call
0709 12:24:47.790008 (+ 16153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.795994 (+1005986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.576279 (+780285us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54003d1654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:49.576475 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2303414528}
I20260709 12:24:49.624214 28843 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.625433 26129 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:49.625780 26129 pending_rounds.cc:85] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 3
I20260709 12:24:49.626067 26129 pending_rounds.cc:107] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:49.626462 26129 raft_consensus.cc:2889] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.617017 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:49.581723 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2496) took 1406 ms. Trace:
I20260709 12:24:49.543887 28748 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:49.510874 28693 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.585862 29187 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:49.593250 29213 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:49.581569 29064 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:49.649062 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141956us
W20260709 12:24:49.683008 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.628729 26127 rpcz_store.cc:276] 0709 12:24:48.175191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.188497 (+ 13306us) service_pool.cc:224] Handling call
0709 12:24:48.211474 (+ 22977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.225284 (+ 13810us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:48.427771 (+202487us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.581109 (+1153338us) spinlock_profiling.cc:243] Waited 7.12 ms on lock 0x7b1800450e20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287
0709 12:24:49.581245 (+   136us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.581315 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:49.581376 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.581481 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.581650 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12304,"prepare.queue_time_us":360245,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":109694,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":17006,"spinlock_wait_cycles":16379648,"thread_start_us":564603,"threads_started":3,"wal-append.queue_time_us":535393}
I20260709 12:24:49.703801 26196 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:49.719914 26046 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:49.704321 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:49.884167 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 267241us
I20260709 12:24:49.885308 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.886840 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.887725 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 805) took 1201 ms. Trace:
I20260709 12:24:49.888357 26790 rpcz_store.cc:276] 0709 12:24:48.686724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.686899 (+   175us) service_pool.cc:224] Handling call
0709 12:24:48.715792 (+ 28893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.934186 (+218394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.855370 (+921184us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54005c35d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:49.887636 (+ 32266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493994240}
I20260709 12:24:49.854907 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2499) took 1678 ms. Trace:
I20260709 12:24:49.891103 26125 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:24:49.891040 26121 rpcz_store.cc:276] 0709 12:24:48.181037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.514543 (+333506us) service_pool.cc:224] Handling call
0709 12:24:48.549655 (+ 35112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.778142 (+228487us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:48.799947 (+ 21805us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.144392 (+344445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.144450 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:49.144502 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.854567 (+710065us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:49.854635 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.854823 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20492,"prepare.queue_time_us":22315,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":13890,"raft.queue_time_us":75360,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4151,"spinlock_wait_cycles":524853120,"thread_start_us":178377,"threads_started":1,"wal-append.queue_time_us":494374}
I20260709 12:24:49.741240 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 321) took 1024 ms. Trace:
W20260709 12:24:49.896797 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 201200us
I20260709 12:24:49.703898 28814 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:49.562934 29151 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.906100 26050 rpcz_store.cc:276] 0709 12:24:48.717068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.798572 (+ 81504us) service_pool.cc:224] Handling call
0709 12:24:49.740939 (+942367us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540006fc54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:49.741142 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594012032}
I20260709 12:24:49.569150 29055 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:49.923015 28951 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.923585 28951 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:49.924029 28951 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.924839 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:49.925812 28380 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:49.926330 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:49.927790 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:49.651167 28748 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:24:49.770462 29156 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.006s
I20260709 12:24:49.924889 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:49.833458 29164 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:49.792675 29154 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:24:49.932456 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:49.929968 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:49.905784 29187 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:49.947875 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 366575us
I20260709 12:24:49.928714 28380 raft_consensus.cc:2764] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:49.923051 29069 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.904956 29155 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.934636 28748 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:49.968933 29212 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
W20260709 12:24:49.977926 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.959046 28992 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:49.980435 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.964429 28876 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.963011 28973 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.969390 28897 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:49.954612 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:49.952836 29156 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:49.982699 28814 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:49.975268 29155 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:49.993556 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61399.3us
I20260709 12:24:49.991869 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:49.994819 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.005071 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.006131 28380 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:50.006356 26048 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 8 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:50.006708 28380 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:50.007313 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:50.007601 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:50.007941 26051 raft_consensus.cc:2376] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6. Re-sending same reply.
I20260709 12:24:50.008406 26050 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.008630 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.009296 26196 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.991979 29175 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.985823 28973 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:50.017627 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.018054 28973 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.018918 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.019763 26053 raft_consensus.cc:2410] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:49.972437 29164 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:49.913542 29064 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:49.988190 28876 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.016508 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.017356 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.024158 29175 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.024425 29164 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:49.975172 29069 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.026698 28938 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.030100 26196 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:50.049085 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2490) took 2073 ms. Trace:
I20260709 12:24:50.036294 26201 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:50.037391 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.037712 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2491) took 2559 ms. Trace:
I20260709 12:24:50.053514 26122 rpcz_store.cc:276] 0709 12:24:47.469420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.743613 (+274193us) service_pool.cc:224] Handling call
0709 12:24:47.769774 (+ 26161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.785049 (+1015275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.028361 (+1243312us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001f86d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:50.028563 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2321263360}
I20260709 12:24:50.002218 29047 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.005054 29155 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.059041 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:50.046490 26050 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:49.991281 29190 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:50.074407 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.029309 29214 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.446s	user 0.000s	sys 0.004s
I20260709 12:24:50.059022 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.013096 29065 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.007654 28380 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.097154 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.063323 29069 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.066998 28796 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:50.028653 29055 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:24:50.071382 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.080613 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:50.081375 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.110841 26785 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:50.081911 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.111939 26782 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:50.082597 29157 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:50.080065 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.083873 29152 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.078940 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.115365 26203 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.057098 26194 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.049688 26047 rpcz_store.cc:276] 0709 12:24:47.975258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.993308 (+ 18050us) service_pool.cc:224] Handling call
0709 12:24:48.038466 (+ 45158us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:48.112705 (+ 74239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.112751 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:48.119307 (+  6556us) log.cc:844] Serialized 78 byte log entry
0709 12:24:48.404836 (+285529us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:48.431650 (+ 26814us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.463040 (+ 31390us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.681538 (+218498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.681681 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.623040 (+941359us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b2400136fe8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:24:50.048721 (+425681us) raft_consensus.cc:1685] finished
0709 12:24:50.048779 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.049010 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92471,"prepare.queue_time_us":154892,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":2360,"raft.queue_time_us":499995,"raft.run_cpu_time_us":567,"raft.run_wall_time_us":49225,"spinlock_wait_cycles":10962304,"thread_start_us":503193,"threads_started":2,"wal-append.queue_time_us":1033065}
I20260709 12:24:50.097889 26196 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:50.054260 29187 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.100790 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.103287 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:50.121019 26784 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:50.104014 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.106257 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.108063 29065 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.104925 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.117120 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.121920 26788 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:50.104106 29064 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:50.127867 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47913.2us
I20260709 12:24:50.145192 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.119436 29194 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:50.153357 26196 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:50.161880 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.146138 29187 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.176080 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.148290 29064 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.125928 29065 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.126156 29155 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.212371 29155 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:50.152045 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:50.190371 26203 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:24:50.127528 29190 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:24:50.172355 29179 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:50.220667 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45643.7us
I20260709 12:24:50.226687 26194 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:50.222414 26786 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.208155 29108 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:50.247110 26019 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.570s (SENT)
I20260709 12:24:50.209296 28951 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:50.226442 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1636188389998097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 12:24:50.260599 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1546188389997951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 12:24:50.215224 29064 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:50.265066 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 243947us
W20260709 12:24:50.263202 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1411188389997733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
I20260709 12:24:50.252307 29156 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.231503 28992 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:50.276791 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:50.286873 26019 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.609s (SENT)
I20260709 12:24:50.223686 28814 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:50.293644 26787 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.309284 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1277) took 1228 ms. Trace:
W20260709 12:24:50.327581 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50888.6us
I20260709 12:24:50.328624 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:50.310150 26198 rpcz_store.cc:276] 0709 12:24:49.081316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.093959 (+ 12643us) service_pool.cc:224] Handling call
0709 12:24:49.094554 (+   595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.094819 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:49.143963 (+ 49144us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.130578 (+986615us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b18003d2c40. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:24:50.308700 (+178122us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.308779 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:50.308841 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.308939 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.309204 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22716,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"spinlock_wait_cycles":11481344,"thread_start_us":177824,"threads_started":1}
I20260709 12:24:50.276271 29194 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:24:50.261579 29108 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.391248 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1282) took 1231 ms. Trace:
W20260709 12:24:50.444352 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 176933us
W20260709 12:24:50.356849 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70036.7us
W20260709 12:24:50.368155 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91451.5us
I20260709 12:24:50.323164 29023 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:50.570608 29179 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:50.571202 29179 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.571954 29023 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.567799 26199 rpcz_store.cc:276] 0709 12:24:49.159759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.159905 (+   146us) service_pool.cc:224] Handling call
0709 12:24:50.390965 (+1231060us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:50.391139 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756256512}
W20260709 12:24:50.391960 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63460.3us
W20260709 12:24:50.467579 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.573324 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.572932 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.578626 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.578765 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.579483 26052 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:50.528939 29182 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:24:50.577142 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.574569 28677 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:50.580746 28677 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:50.581562 28677 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:50.527598 28973 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.000s
I20260709 12:24:50.583226 28677 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.583570 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.584278 26051 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:50.583767 28973 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.574621 29226 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:24:50.585945 28950 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.600093 29151 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:50.600965 29176 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.587109 29047 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:50.601271 29166 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.601598 29047 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:50.601996 29216 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.555s	user 0.008s	sys 0.021s
I20260709 12:24:50.588209 26051 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:50.499372 28992 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.602626 29047 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.603051 29185 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.574802 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:50.603907 29047 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.605793 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.606791 28938 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:50.574743 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:50.608583 29230 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:50.524030 29110 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.051s	user 0.011s	sys 0.001s
I20260709 12:24:50.609975 29110 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.529920 29062 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:50.612715 29062 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.507943 28814 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.591547 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:50.593981 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:50.613754 26052 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.595352 29179 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:50.368533 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 143103us
I20260709 12:24:50.589856 29189 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:24:50.585338 29226 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:50.599370 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:50.600664 28950 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.523044 29065 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:50.502990 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.573558 29023 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.612787 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1288) took 1237 ms. Trace:
I20260709 12:24:50.631343 26046 rpcz_store.cc:276] 0709 12:24:49.375129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.477302 (+102173us) service_pool.cc:224] Handling call
0709 12:24:49.477884 (+   582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.882585 (+404701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.612519 (+729934us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400131354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:50.612717 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555864064}
I20260709 12:24:50.613235 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.619181 26198 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:50.613318 28677 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.590778 28973 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.622236 26194 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:50.626897 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.642171 26053 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.642580 26203 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:50.394451 29178 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.915s	user 0.010s	sys 0.010s
W20260709 12:24:50.565176 29211 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.949s	user 0.007s	sys 0.000s
I20260709 12:24:50.333467 28876 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.601610 29176 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:50.623037 29067 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.180s	user 0.013s	sys 0.001s
W20260709 12:24:50.392608 29224 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:24:50.390926 29228 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.622102 29226 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:50.645730 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.671435 26050 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:50.646376 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.646461 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.393754 29064 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.489866 29163 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.647207 26195 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:50.332983 29229 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.650422 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.655689 29151 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:50.613598 29062 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:50.659267 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.624202 29230 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:50.392501 29225 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.644891 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.666556 26047 raft_consensus.cc:606] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.673194 26197 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.544914 29055 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 12:24:50.688740 26047 raft_consensus.cc:614] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:50.641873 28951 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:50.650684 26120 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:50.682464 26201 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:50.628983 29190 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:24:50.686138 26202 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:50.680601 29229 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.682184 28693 raft_consensus.cc:1064] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:50.675434 29228 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.709149 29237 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:50.709834 29237 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:50.674695 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 333877us
I20260709 12:24:50.689625 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.675519 29067 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:50.660583 29176 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:50.687953 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 286297us
I20260709 12:24:50.696098 26053 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:50.698081 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 119513us
I20260709 12:24:50.649627 28677 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.679122 28973 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:50.683153 29151 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:50.690276 29178 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.717214 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:50.724223 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.728334 26201 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:50.726925 26093 leader_election.cc:400] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:24:50.725677 29176 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.731000 29179 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.731142 29062 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.731604 29237 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.683550 29163 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.710852 29228 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.738879 26054 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:50.740514 29226 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.741302 28950 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.742762 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:50.704042 29157 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:50.744546 29157 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.744920 29157 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:50.745561 29234 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.746012 29234 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.746333 29234 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.705065 29230 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:50.749017 29230 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.749471 29180 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.750352 28677 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.753968 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.756285 26197 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:50.756832 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1661) took 1413 ms. Trace:
W20260709 12:24:50.735986 29187 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.000s
I20260709 12:24:50.712373 29229 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.768187 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.763195 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.770282 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.771019 26197 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.740037 29163 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.750229 29151 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.785519 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:50.786254 28938 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:24:50.786489 26782 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:50.786769 28938 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:50.787068 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.787827 26784 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:50.789294 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.791509 26790 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:50.793124 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.793663 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.794261 26046 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:50.794514 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.795081 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:50.795130 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.795753 26048 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:50.796079 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.759951 29225 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:50.759773 29178 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:50.762030 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.766997 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44926.2us
I20260709 12:24:50.760985 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.798018 29023 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:50.798964 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:50.767650 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.800781 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.802281 26124 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:50.785637 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.728443 28973 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:50.796686 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.757458 26200 rpcz_store.cc:276] 0709 12:24:49.343157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.555804 (+212647us) service_pool.cc:224] Handling call
0709 12:24:49.556710 (+   906us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:49.697142 (+140432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.697206 (+    64us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:49.910701 (+213495us) log.cc:844] Serialized 52 byte log entry
0709 12:24:50.129864 (+219163us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:50.164502 (+ 34638us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.172094 (+  7592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.172184 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.756551 (+584367us) raft_consensus.cc:1685] finished
0709 12:24:50.756581 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.756753 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81502,"prepare.queue_time_us":34662,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":150549,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":58515,"thread_start_us":160512,"threads_started":1,"wal-append.queue_time_us":316005}
I20260709 12:24:50.798978 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.786396 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.805689 26202 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:50.800176 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:50.806375 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.796656 26049 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:50.803994 26791 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:50.815663 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:50.816882 26048 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:50.824679 26052 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:50.833880 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:50.835439 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1278) took 1482 ms. Trace:
I20260709 12:24:50.838245 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.838394 26197 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:50.842733 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.843762 26197 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:50.844830 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.852684 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:50.853386 26129 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.835975 26045 rpcz_store.cc:276] 0709 12:24:49.353283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.353646 (+   363us) service_pool.cc:224] Handling call
0709 12:24:49.354172 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.354628 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:49.354909 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.444754 (+ 89845us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.469024 (+ 24270us) log.cc:844] Serialized 564 byte log entry
0709 12:24:49.469213 (+   189us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:49.469277 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.469373 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.564348 (+1094975us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:24:50.804319 (+239971us) raft_consensus.cc:1685] finished
0709 12:24:50.835187 (+ 30868us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.835367 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":156,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":8340,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6019,"spinlock_wait_cycles":27523968,"thread_start_us":82565,"threads_started":1,"wal-append.queue_time_us":153751}
I20260709 12:24:50.835664 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.866250 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.868151 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.869769 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:50.871179 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.865155 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.760705 29067 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:24:50.885461 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.880316 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.865489 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.874193 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:50.859419 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:50.887940 26122 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:50.875821 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:50.888475 26200 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:50.875155 26128 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:24:50.878703 26202 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:50.885015 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.890268 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:50.890434 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:50.897696 26124 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.890882 26049 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:50.898604 26049 pending_rounds.cc:85] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 3
I20260709 12:24:50.898840 26049 pending_rounds.cc:107] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:24:50.899065 26049 raft_consensus.cc:2889] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:50.899452 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.710867 28693 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:50.913458 26788 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:50.915233 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.922487 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63470.1us
I20260709 12:24:50.891491 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.927805 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.873565 29180 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:50.783561 29187 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 12:24:50.871840 29023 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:50.852869 29065 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:50.935570 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.913424 28876 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:50.852885 29179 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.930305 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 148256us
I20260709 12:24:50.853788 29242 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.956461 29242 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.956900 29242 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.958171 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.959335 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:50.959924 26129 raft_consensus.cc:2410] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:50.872830 29163 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.960705 29179 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.961758 26129 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:50.962984 29179 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.890714 28950 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.964133 26129 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:50.965703 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2509) took 1568 ms. Trace:
I20260709 12:24:50.966104 26120 rpcz_store.cc:276] 0709 12:24:49.397572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.739006 (+341434us) service_pool.cc:224] Handling call
0709 12:24:49.739404 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.691526 (+952122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.965451 (+273925us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b540037c154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:50.965646 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095095040}
I20260709 12:24:50.940389 26051 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:50.931587 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.933669 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.935268 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.973021 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:50.973012 26121 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:50.928889 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.975809 26197 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:50.933027 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:50.951340 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:50.954141 26788 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:50.973292 26200 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:50.854030 29237 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.983209 26129 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:50.984189 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.984866 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:50.931191 29243 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.918676 29166 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 12:24:50.980899 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50831us
W20260709 12:24:50.991236 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 374017us
I20260709 12:24:50.949596 29065 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.935923 29110 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:50.994246 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.989393 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.002610 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:50.998188 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99039.5us
I20260709 12:24:51.005935 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:50.997602 29065 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:51.012035 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:50.988070 29187 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.918443 29175 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:24:51.018209 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.023216 26786 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:50.995710 28843 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.009s	sys 0.023s
I20260709 12:24:51.019418 26783 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:51.024977 28843 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.010s	sys 0.023s
I20260709 12:24:50.994942 29243 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.027442 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:50.946476 28973 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:50.988757 29237 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.019654 29055 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.028481 28748 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.043s	user 0.008s	sys 0.006s
I20260709 12:24:51.022471 29187 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.048206 26199 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:24:51.045652 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.049808 26783 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:51.051556 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:51.056840 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.047547 26124 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:51.051707 29187 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.072314 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.073001 26786 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:51.073941 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.030524 29243 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.074766 26789 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:51.075560 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.076557 29166 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.076325 28748 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.077123 29166 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.077688 26126 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:51.078959 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 428) took 2334 ms. Trace:
I20260709 12:24:51.079092 29166 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:51.052074 29055 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.079288 26126 rpcz_store.cc:276] 0709 12:24:48.744426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.622410 (+877984us) service_pool.cc:224] Handling call
0709 12:24:49.622705 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.078186 (+1455481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.078750 (+   564us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400287654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:51.078904 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3345616640}
I20260709 12:24:51.080819 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1663) took 1737 ms. Trace:
I20260709 12:24:51.077355 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.076135 26789 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:51.087718 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:51.088521 26197 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.089844 26198 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:51.081449 26195 rpcz_store.cc:276] 0709 12:24:49.343726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.589124 (+245398us) service_pool.cc:224] Handling call
0709 12:24:49.589680 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.680282 (+1090602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.080578 (+400296us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:51.080755 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2431364864}
I20260709 12:24:51.049587 29062 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:51.105551 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.115466 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:51.154152 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.153923 26788 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:51.154613 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:51.154935 26790 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:51.155341 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:51.031428 29157 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.052335 29180 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:51.158651 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.159648 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:51.153913 29062 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.159080 26194 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:51.168048 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:51.183079 26051 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.079231 28748 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:51.177289 26122 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.170192 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:51.185782 26202 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:51.185842 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:51.186677 26789 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:51.188436 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.191419 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.191059 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 193879us
I20260709 12:24:51.191675 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:51.192142 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.192631 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:51.201082 26791 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11.
I20260709 12:24:51.154242 28876 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:51.202351 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:51.156975 29243 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.215327 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:51.216555 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 212116us
I20260709 12:24:51.189710 29062 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.213697 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:51.224994 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2508) took 1828 ms (client timeout 1711 ms). Trace:
W20260709 12:24:51.249022 26123 rpcz_store.cc:269] 0709 12:24:49.396877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.475978 (+ 79101us) service_pool.cc:224] Handling call
0709 12:24:51.224658 (+1748680us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400186ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:51.224857 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3599330048}
I20260709 12:24:51.227947 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:51.230719 26786 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:51.231367 26054 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:51.237975 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60878 (request call id 11) took 1111 ms. Trace:
I20260709 12:24:51.240638 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:51.290498 26047 rpcz_store.cc:276] 0709 12:24:50.144642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.146934 (+  2292us) service_pool.cc:224] Handling call
0709 12:24:51.237762 (+1090828us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540022bd54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:51.237876 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038187264}
I20260709 12:24:51.240443 29179 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:51.291090 26126 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:51.241058 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.240866 28380 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:51.241539 26129 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:51.242547 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:51.243194 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.285032 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:24:51.292341 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49865.8us
I20260709 12:24:51.293875 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:51.294119 26167 leader_election.cc:336] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.713s (SENT)
W20260709 12:24:51.294806 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80213us
I20260709 12:24:51.296180 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:51.287961 26053 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.300276 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.301213 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:51.310506 26048 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:51.315301 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:51.318351 26124 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:51.211505 29230 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:24:51.329619 26053 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:51.303184 29236 raft_consensus.cc:2764] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:51.354166 26052 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:51.221450 28814 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.105s
W20260709 12:24:51.350106 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.307411 29261 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:51.351763 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 239781us
I20260709 12:24:51.354614 26052 pending_rounds.cc:85] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 5
W20260709 12:24:51.351194 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 158669us
I20260709 12:24:51.247884 29225 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:24:51.304754 28950 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:51.349776 29226 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.362964 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:51.361670 26052 pending_rounds.cc:107] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:51.374341 26052 raft_consensus.cc:2889] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:51.362385 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 68584.4us
I20260709 12:24:51.306810 29183 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:24:51.357944 29110 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:51.380306 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:51.360392 28677 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.458714 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78816.1us
I20260709 12:24:51.367035 28951 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:51.363327 29261 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:51.360479 29272 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:51.381870 29225 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:51.359498 29269 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:24:51.397293 29287 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.500097 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:51.441035 29055 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:51.461220 29067 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.529070 29272 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.496479 29185 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:51.545096 29268 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:51.492107 29235 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.581663 29279 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:51.582175 29274 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:51.581843 29268 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.559476 29237 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:51.573311 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:51.360090 29267 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.183s
I20260709 12:24:51.363132 29180 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.439482 29064 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.128s
I20260709 12:24:51.463305 28677 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.460755 29236 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 12:24:51.382536 28876 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.532454 29110 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.500907 28951 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:51.719200 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 146102us
W20260709 12:24:51.697192 29251 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:24:51.698173 29280 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.742268 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 377067us
I20260709 12:24:51.754904 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:51.755668 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 259309us
I20260709 12:24:51.725378 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:51.763723 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:51.752919 29185 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.582963 29157 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:51.768653 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:51.770673 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 290253us
I20260709 12:24:51.774575 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:51.707103 29272 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:51.786341 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:51.814747 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.737497 29225 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.825275 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:51.817878 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 92593.9us
I20260709 12:24:51.771669 29067 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:51.762693 29175 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:51.760025 29166 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:51.840776 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:51.830047 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:51.853500 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:51.742475 29055 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:24:51.741472 29110 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:51.884428 26122 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.700s	user 0.006s	sys 0.005s
I20260709 12:24:51.884826 26122 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:51.864030 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:51.882380 26789 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.001s
I20260709 12:24:51.887689 26789 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:51.813699 29243 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:51.707058 28950 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.575251 29287 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:51.831095 29279 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:51.742475 29182 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 12:24:51.777640 28380 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:24:51.829550 29284 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:51.816797 29283 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:51.909547 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.837662 29269 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:51.912424 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48490.2us
I20260709 12:24:51.889074 29185 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:51.893278 29288 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:24:51.766899 29235 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.861591 29157 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:51.938643 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 164732us
I20260709 12:24:51.862087 29262 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:51.881349 29166 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.768450 29280 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:51.903344 29064 raft_consensus.cc:2764] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:51.852047 29267 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 12:24:51.956176 26051 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.772s	user 0.011s	sys 0.001s
I20260709 12:24:51.905344 29067 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:51.723246 29274 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:51.967967 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2519) took 1154 ms. Trace:
I20260709 12:24:51.979964 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:52.015621 26051 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:52.015873 26045 rpcz_store.cc:276] 0709 12:24:50.813292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.885154 (+ 71862us) service_pool.cc:224] Handling call
0709 12:24:50.921497 (+ 36343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.062606 (+141109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.062744 (+   138us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.064489 (+  1745us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.327775 (+263286us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:51.359628 (+ 31853us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.359827 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.359900 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.967725 (+607825us) raft_consensus.cc:1685] finished
0709 12:24:51.967752 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.967902 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25850,"prepare.queue_time_us":32269,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":2812,"raft.queue_time_us":409263,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":32980,"thread_start_us":392900,"threads_started":2,"wal-append.queue_time_us":303381}
I20260709 12:24:51.940864 29273 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:24:51.962867 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 188378us
I20260709 12:24:52.017472 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:51.939572 29284 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.970058 29260 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:51.967846 29155 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:51.880702 29175 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:51.960651 29185 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.955226 29055 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:51.965687 29236 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.839357 29225 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:52.000324 29047 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.387s	user 0.008s	sys 0.005s
I20260709 12:24:51.957641 28677 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.030185 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:51.957000 29269 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:52.115459 28951 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:51.918694 29237 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:24:51.966171 29243 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:52.116418 29157 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.116861 29243 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:52.033831 29284 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.118175 29243 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.118928 29055 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.081789 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:52.121277 29055 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.121690 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.123489 29225 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.123808 26053 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:52.086432 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:51.957424 28380 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:24:52.023645 29300 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:52.227563 29300 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:52.228868 29300 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.225490 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:52.233373 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:52.033159 29276 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.459s	user 0.000s	sys 0.001s
I20260709 12:24:52.100881 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:52.237176 26045 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:52.171459 29229 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.368s	user 0.011s	sys 0.002s
I20260709 12:24:52.237959 29229 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.239138 26785 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:52.101835 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.240948 26052 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:52.242280 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.243040 26052 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
W20260709 12:24:52.112222 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82581us
I20260709 12:24:52.245229 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.113922 26196 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:52.115691 29269 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.246438 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 471) took 1135 ms. Trace:
I20260709 12:24:52.246767 29269 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:52.247445 29305 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:52.246860 26198 rpcz_store.cc:276] 0709 12:24:51.110758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.156832 (+ 46074us) service_pool.cc:224] Handling call
0709 12:24:51.157255 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.431933 (+274678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.460444 (+ 28511us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.492578 (+ 32134us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.794009 (+301431us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:51.794068 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.794170 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.246239 (+452069us) raft_consensus.cc:1685] finished
0709 12:24:52.246259 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.246385 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2949,"raft.queue_time_us":127979,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11688,"thread_start_us":391144,"threads_started":2,"wal-append.queue_time_us":684458}
W20260709 12:24:52.248688 29271 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.116s	user 0.001s	sys 0.002s
W20260709 12:24:52.249552 29298 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.230s	user 0.003s	sys 0.002s
W20260709 12:24:52.030828 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 177394us
I20260709 12:24:52.252588 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2523) took 1352 ms. Trace:
I20260709 12:24:52.252764 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.254412 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.252962 26049 rpcz_store.cc:276] 0709 12:24:50.900275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.931914 (+ 31639us) service_pool.cc:224] Handling call
0709 12:24:50.948939 (+ 17025us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:51.297493 (+348554us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:51.348982 (+ 51489us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.589125 (+240143us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:51.686070 (+ 96945us) log.cc:844] Serialized 52 byte log entry
0709 12:24:51.686271 (+   201us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:51.686648 (+   377us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.758730 (+ 72082us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.763523 (+  4793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.763608 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.252368 (+488760us) raft_consensus.cc:1685] finished
0709 12:24:52.252389 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.252534 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174818,"prepare.queue_time_us":46096,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":3868,"raft.queue_time_us":380272,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":69806,"spinlock_wait_cycles":22813056,"thread_start_us":466406,"threads_started":2,"wal-append.queue_time_us":520719}
I20260709 12:24:52.085278 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:52.125072 29151 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.326s	user 0.010s	sys 0.002s
I20260709 12:24:52.256611 29310 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:52.256889 29151 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.253760 26053 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:52.258173 26053 pending_rounds.cc:85] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 5
I20260709 12:24:52.258504 26053 pending_rounds.cc:107] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:52.258853 26053 raft_consensus.cc:2889] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.259171 29151 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.125819 29236 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:52.260277 26049 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:24:52.134785 29067 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:52.262838 29176 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.533s	user 0.010s	sys 0.003s
W20260709 12:24:52.187963 29187 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.117s	user 0.012s	sys 0.000s
I20260709 12:24:52.137701 29287 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:52.098346 29242 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.141s	user 0.008s	sys 0.004s
I20260709 12:24:52.266378 29242 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.031796 29273 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.269207 29242 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.269521 29273 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.270324 26127 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:52.270533 29267 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.139796 29268 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.085937 29064 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:52.169236 29108 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.086143 29155 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:52.202382 29234 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.456s	user 0.010s	sys 0.003s
W20260709 12:24:52.206308 29062 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.001s
I20260709 12:24:52.212319 29306 raft_consensus.cc:476] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 12:24:52.215449 29311 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.217839 29280 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.221930 29175 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:52.222657 29295 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.239s	user 0.004s	sys 0.001s
I20260709 12:24:52.257300 29310 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.272153 29064 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:52.264922 29187 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.256623 26047 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:52.272889 26052 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:52.127372 29235 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.247140 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.280269 26053 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:52.303891 29108 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.263584 29176 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:52.305459 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:52.240978 29110 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.237360 29274 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.284192 29229 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.302852 29062 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.091885 29185 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:52.325364 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78272.3us
I20260709 12:24:52.333077 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.334542 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.336350 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:52.302054 29065 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.275s	user 0.010s	sys 0.003s
I20260709 12:24:52.329421 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.329669 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 812) took 1256 ms. Trace:
I20260709 12:24:52.338766 29305 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.325364 29151 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:52.320283 29310 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.342069 26791 rpcz_store.cc:276] 0709 12:24:51.073278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.088642 (+ 15364us) service_pool.cc:224] Handling call
0709 12:24:52.329466 (+1240824us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400639f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:52.329602 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38357248}
I20260709 12:24:52.330646 29108 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.354576 26047 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:52.346751 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.384884 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2520) took 1571 ms. Trace:
I20260709 12:24:52.373914 26053 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:52.384305 26123 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260709 12:24:52.387673 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.387812 26123 pending_rounds.cc:85] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 7
I20260709 12:24:52.377606 26052 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:52.388288 26123 pending_rounds.cc:107] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:52.389061 26123 raft_consensus.cc:2889] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.389146 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.391407 26046 rpcz_store.cc:276] 0709 12:24:50.813603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.914542 (+100939us) service_pool.cc:224] Handling call
0709 12:24:50.936579 (+ 22037us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.315179 (+378600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.315209 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.344438 (+ 29229us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.753355 (+408917us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:51.753438 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.753707 (+   269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.384671 (+630964us) raft_consensus.cc:1685] finished
0709 12:24:52.384700 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.384828 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132753,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"thread_start_us":728454,"threads_started":2,"wal-append.queue_time_us":450115}
I20260709 12:24:52.393105 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.337520 29306 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:52.386067 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:52.372828 29108 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.369454 29176 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.399096 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:52.345438 29225 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:52.397782 29277 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:52.394995 28748 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:52.402420 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.411285 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:52.403080 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 441) took 1154 ms. Trace:
I20260709 12:24:52.412477 26123 rpcz_store.cc:276] 0709 12:24:51.248407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.383322 (+1134915us) service_pool.cc:224] Handling call
0709 12:24:52.383775 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.402785 (+ 19010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.403016 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:52.314350 29311 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.344009 29157 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:24:52.398196 29190 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:24:52.427373 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:52.396059 29229 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:52.404409 28380 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.420475 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:52.294826 29234 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.419358 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:52.429265 26128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.016s	sys 0.033s
I20260709 12:24:52.441772 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:52.442847 26128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.016s	sys 0.033s
W20260709 12:24:52.443212 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1292) took 1798 ms (client timeout 2249 ms). Trace:
W20260709 12:24:52.443607 26196 rpcz_store.cc:269] 0709 12:24:50.644462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.644632 (+   170us) service_pool.cc:224] Handling call
0709 12:24:52.443065 (+1798433us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400301954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:52.443160 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3277562624}
I20260709 12:24:52.444205 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.438762 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:52.445456 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.446173 26786 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:52.431981 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:52.442234 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1766) took 1638 ms. Trace:
I20260709 12:24:52.397058 29306 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.442525 28380 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.452050 28380 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.451563 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:52.368381 28677 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:24:52.447330 26789 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:52.460708 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.448217 26127 rpcz_store.cc:276] 0709 12:24:50.803660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.927567 (+123907us) service_pool.cc:224] Handling call
0709 12:24:50.948389 (+ 20822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.270878 (+1322489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.441922 (+171044us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001f5ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:52.442157 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039503616}
I20260709 12:24:52.346871 29065 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.449229 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.347783 29110 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:52.423203 29277 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.463717 26125 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11.
I20260709 12:24:52.463821 29277 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:52.465713 29065 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.458277 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.466691 26786 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:52.456805 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.455636 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.400759 29242 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.468544 26782 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:52.469373 29110 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:52.471822 29110 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:52.476346 26195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:24:52.477092 26195 raft_consensus.cc:606] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.477481 26195 raft_consensus.cc:614] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:52.478693 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.479426 26195 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:24:52.480644 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.481024 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.482477 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.484467 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.485131 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.486020 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.486232 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.486925 26201 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:52.487030 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.487249 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:52.487799 26783 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.368561 29307 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:24:52.475633 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.474357 29291 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.417389 29176 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.425172 29264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.023s	sys 0.023s
I20260709 12:24:52.490248 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:52.501008 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:52.493552 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:52.498240 26196 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:52.431849 29225 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:52.508397 29302 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:24:52.509043 29225 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:52.511778 29281 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.926s	user 0.009s	sys 0.000s
I20260709 12:24:52.460798 29151 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:52.512670 29151 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:52.513041 29308 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:24:52.513654 29151 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.514995 29308 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:52.516160 29242 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.516671 29242 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.517033 29242 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.345808 29155 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.519253 29284 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.519608 29155 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:52.520376 29267 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:52.520816 29267 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.521145 29267 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:52.524434 29176 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.526752 29178 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.527999 29277 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.451237 29287 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.500519 29175 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:52.530321 29277 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:52.503479 29264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.023s	sys 0.023s
I20260709 12:24:52.531185 29292 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:52.531601 29292 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:52.440323 29305 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:52.532358 29264 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:52.532294 29305 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.453450 29273 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:52.533556 29273 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:52.534801 29305 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.534492 29273 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.460616 29180 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:52.535765 29273 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.506587 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.404817 29064 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.536144 29261 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.507732 29187 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.506846 29062 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.529906 29175 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.536727 26789 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:52.513531 29311 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.459295 29306 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:52.532891 29264 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:52.507879 29299 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:52.505604 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.547003 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.505712 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:52.547896 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.543681 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.559250 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.560209 26196 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:52.560446 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.537590 29261 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:52.564605 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59015.8us
I20260709 12:24:52.539376 29274 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:52.539578 28951 consensus_queue.cc:1175] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 5 status { last_received { term: 5 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 5 index: 6 } }
I20260709 12:24:52.563153 29299 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:52.484120 29110 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.608330 29274 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.634087 29274 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:52.637181 26167 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.234s (SENT)
I20260709 12:24:52.637836 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:52.600718 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.640329 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.640996 26201 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:52.609650 29284 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.608148 29180 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.625469 29299 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.548985 29065 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:52.655319 29175 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:52.658906 29318 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:24:52.659580 29243 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:52.708053 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 254262us
I20260709 12:24:52.707118 26783 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
W20260709 12:24:52.679612 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 247694us
W20260709 12:24:52.658950 26785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 556) took 1891 ms (client timeout 2073 ms). Trace:
I20260709 12:24:52.711356 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:52.711637 26785 rpcz_store.cc:269] 0709 12:24:50.767139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.815399 (+ 48260us) service_pool.cc:224] Handling call
0709 12:24:52.658692 (+1843293us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54006bf8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:52.658884 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3271644416}
I20260709 12:24:52.709030 28380 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:52.611284 29316 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:52.725641 28380 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.718921 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:52.626708 29277 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:52.727509 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.728256 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.728948 26783 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:52.660939 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:52.729843 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:52.728317 26785 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:24:52.733721 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 96591us
I20260709 12:24:52.734611 28380 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:52.740490 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.741375 26045 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:52.743511 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:52.562728 29226 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.180s	user 0.009s	sys 0.004s
I20260709 12:24:52.748513 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.749140 26791 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:24:52.749655 26791 pending_rounds.cc:85] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 5
I20260709 12:24:52.749992 26791 pending_rounds.cc:107] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:24:52.750312 26791 raft_consensus.cc:2889] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.532474 29292 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.725299 28796 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:24:52.748736 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:52.763293 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:52.766636 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:52.769462 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:52.774663 26045 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:52.612517 29234 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.601135 29176 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:52.776803 26788 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:24:52.730885 29305 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.779278 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:52.787075 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:52.729437 29065 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.788270 29065 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.731341 29291 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:52.783788 26045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:52.611764 29185 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.785895 26196 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:52.772663 29190 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:52.780202 29194 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:52.706244 29180 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.797545 26045 raft_consensus.cc:606] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:52.811770 26045 raft_consensus.cc:614] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:52.795023 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:52.751647 29310 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:52.846414 26200 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:24:52.795873 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.745754 29156 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:24:52.730886 29300 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:24:52.818077 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 209894us
W20260709 12:24:52.838394 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.786538 29305 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.852783 26018 connection.cc:256] RPC call timeout handler was delayed by -1.3426882920002754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 12:24:52.858815 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1116432720000375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3174s for a response
I20260709 12:24:52.849058 26051 raft_consensus.cc:2955] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:52.777604 29267 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.797920 29229 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:52.764554 29311 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:52.799717 29277 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:52.746601 29264 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:52.915067 26050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2515) took 2103 ms (client timeout 1873 ms). Trace:
W20260709 12:24:53.163024 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.913764 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2524) took 1978 ms. Trace:
W20260709 12:24:53.163940 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.164871 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 369941us
I20260709 12:24:53.166554 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:52.968046 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.807561 29243 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:52.878082 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:52.881333 26049 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:52.912871 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1286) took 2295 ms. Trace:
I20260709 12:24:52.810703 29194 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:52.917524 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:53.163731 26054 rpcz_store.cc:276] 0709 12:24:50.935114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.960556 (+ 25442us) service_pool.cc:224] Handling call
0709 12:24:51.231191 (+270635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.290065 (+ 58874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.913557 (+1623492us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400020650. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:52.913702 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621632256}
I20260709 12:24:52.883663 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:52.936457 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2513) took 2132 ms (client timeout 2015 ms). Trace:
I20260709 12:24:52.882411 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:52.934909 26203 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:24:53.184427 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 405765us
I20260709 12:24:53.183831 26200 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:52.862313 29267 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:52.926586 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:53.178328 26194 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:24:53.191015 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 273568us
W20260709 12:24:53.162674 26050 rpcz_store.cc:269] 0709 12:24:50.811774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.811919 (+   145us) service_pool.cc:224] Handling call
0709 12:24:52.914814 (+2102895us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54002e2554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:52.914952 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582196480}
W20260709 12:24:53.063181 29272 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.239s	user 0.007s	sys 0.004s
I20260709 12:24:53.178300 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:53.178148 26199 rpcz_store.cc:276] 0709 12:24:50.617674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.617820 (+   146us) service_pool.cc:224] Handling call
0709 12:24:50.618239 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.049939 (+431700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.912498 (+1862559us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400336b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:52.912766 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981940480}
W20260709 12:24:53.183581 26125 rpcz_store.cc:269] 0709 12:24:50.804158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.873839 (+ 69681us) service_pool.cc:224] Handling call
0709 12:24:52.936128 (+2062289us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:52.936363 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3611419520}
W20260709 12:24:53.168685 28693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.004s	sys 0.000s
I20260709 12:24:52.918268 29189 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:53.200515 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.177013 28951 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.217418 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:53.201116 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.867884 29187 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.223588 29166 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:52.882629 29312 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.224954 29312 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.225423 29312 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.228984 29180 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:53.219856 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:52.921449 29062 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:53.232892 29062 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:53.244379 29273 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.708s	user 0.013s	sys 0.000s
W20260709 12:24:53.246826 29254 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.943s	user 0.007s	sys 0.000s
W20260709 12:24:53.249629 29269 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.002s	user 0.012s	sys 0.000s
I20260709 12:24:53.250232 29269 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.213305 29179 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:53.258597 29261 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.669s	user 0.004s	sys 0.005s
I20260709 12:24:53.262537 26052 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:53.266891 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:53.267714 26052 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:53.267439 29312 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.270177 29312 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:53.272858 29242 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.755s	user 0.008s	sys 0.002s
I20260709 12:24:53.273445 29242 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.222764 28951 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.275624 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.276309 28951 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.223888 29187 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.277310 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.278081 26047 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.225592 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:53.143860 29176 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.226917 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:53.279695 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.279925 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.280320 26054 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:53.281219 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.281361 26050 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:53.282673 26050 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:53.285462 26050 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:53.273774 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.275890 29242 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:52.916831 29319 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:24:53.306567 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128312us
I20260709 12:24:53.309842 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:53.188467 29299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.010s	sys 0.007s
I20260709 12:24:52.876681 29305 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.212478 28693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.000s
I20260709 12:24:53.300976 26054 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:53.176926 29228 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:53.186193 29194 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.215749 29189 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:52.852783 29323 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:53.303793 26783 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:53.191157 29264 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.305305 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:53.183079 29110 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.315822 26046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6389d613e36a42958d24b00099843b93"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:52.891764 29306 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.318606 29064 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.346151 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:53.333386 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.863282 29229 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:52.913630 29314 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.588s
I20260709 12:24:53.312601 29312 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:53.187769 29277 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.218396 29272 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:53.375193 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 322801us
I20260709 12:24:53.422035 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:53.423581 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:53.323710 29310 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.359253 26047 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:53.316521 29299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.011s	sys 0.007s
I20260709 12:24:53.386792 26203 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.319386 29179 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:53.195761 29175 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.151s
I20260709 12:24:53.302666 29267 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:24:53.413370 26200 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:24:53.348583 29242 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:53.427728 26786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.020s	sys 0.051s
I20260709 12:24:53.427829 26197 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:53.360112 29323 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:53.446115 26786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.020s	sys 0.051s
I20260709 12:24:53.385280 29228 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
W20260709 12:24:53.402215 29151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.015s	sys 0.037s
I20260709 12:24:53.410689 29314 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:53.457516 29269 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:53.447413 29323 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.468156 26203 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:53.457628 29310 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.487690 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 64195.6us
I20260709 12:24:53.438834 29312 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.432780 26783 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:53.495321 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:53.496834 26783 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.497933 29306 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:53.453707 26048 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.034s	sys 0.051s
I20260709 12:24:53.410315 29110 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:24:53.375124 29064 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.501194 29064 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.501748 29064 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:53.502089 29064 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:53.502992 29064 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:53.458949 29274 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.002s
I20260709 12:24:53.490847 26203 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:53.504590 29274 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.505586 26203 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:53.507417 29274 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.508870 26045 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:53.499147 26048 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.035s	sys 0.051s
W20260709 12:24:53.515565 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.491580 29272 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:53.499125 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1681) took 1023 ms. Trace:
I20260709 12:24:53.521011 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1316) took 1292 ms. Trace:
I20260709 12:24:53.519937 26198 rpcz_store.cc:276] 0709 12:24:52.475838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.482060 (+  6222us) service_pool.cc:224] Handling call
0709 12:24:52.482386 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.482746 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:52.733866 (+251120us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.112316 (+378450us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.167640 (+ 55324us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.167784 (+   144us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:53.167820 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.168031 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.498886 (+330855us) raft_consensus.cc:1685] finished
0709 12:24:53.498914 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.499058 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22765,"prepare.queue_time_us":393042,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":18459,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":40501,"thread_start_us":551698,"threads_started":2,"wal-append.queue_time_us":657247}
I20260709 12:24:53.521395 26049 rpcz_store.cc:276] 0709 12:24:52.229000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.261319 (+ 32319us) service_pool.cc:224] Handling call
0709 12:24:52.261718 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.181273 (+919555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.520768 (+339495us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:53.520952 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415772544}
I20260709 12:24:53.522290 29312 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.508159 29274 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:53.467145 29151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.016s	sys 0.038s
I20260709 12:24:53.467697 29180 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:53.554611 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:53.539876 26783 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:24:53.566057 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:53.567014 26783 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:53.544438 29342 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:24:53.578948 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:24:53.578207 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 391665us
I20260709 12:24:53.584415 29237 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:53.584939 29237 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:53.585804 29237 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.586834 29237 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.540910 29310 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.590209 29310 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.455034 29189 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.551882 29180 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:53.569332 29240 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:53.611651 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:53.613909 29310 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:53.615561 29310 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:53.616122 29310 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:53.619410 29310 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.622778 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:53.623251 29292 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.577895 29274 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:53.644021 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:53.649892 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.652341 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.612710 29180 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.659438 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.665023 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.666836 26120 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:53.638875 29189 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.625916 29240 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:53.729353 29240 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.730428 29240 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.701864 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.625698 29345 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:53.732110 26127 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:53.732358 29345 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.733204 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.733920 26127 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:53.734676 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:53.735554 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.735575 29345 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:53.638679 29305 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.640620 29274 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:53.736909 29274 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.738049 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:53.718681 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.708923 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:53.662514 29065 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.005s
W20260709 12:24:53.740628 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.740770 26124 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:53.741811 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:53.742573 26121 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:53.744529 26785 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.008s	sys 0.015s
W20260709 12:24:53.745076 26785 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.008s	sys 0.015s
I20260709 12:24:53.640995 29243 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:53.709088 26046 raft_consensus.cc:606] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:53.778524 26046 raft_consensus.cc:614] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:53.727138 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 401167us
I20260709 12:24:53.669960 26125 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:53.780686 26125 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:24:53.782728 26125 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:24:53.743067 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.012s	sys 0.041s
I20260709 12:24:53.751968 26787 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:53.783830 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.013s	sys 0.041s
W20260709 12:24:53.784745 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.755656 26123 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:53.764351 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.796275 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.797070 26124 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:53.646416 29194 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.809424 26129 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:53.846956 26045 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:53.846081 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.847617 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.850371 26047 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.848171 26053 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:53.854326 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:53.852562 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.009s	sys 0.017s
W20260709 12:24:53.856738 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.857106 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.009s	sys 0.017s
W20260709 12:24:53.858214 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.849510 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:53.861932 26045 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:53.863669 26194 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:53.864336 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:53.863772 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:53.865610 26197 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:53.865800 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:53.858101 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1322) took 1061 ms. Trace:
I20260709 12:24:53.867441 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:53.868278 26124 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:53.843796 29344 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
I20260709 12:24:53.869727 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:53.823571 26093 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.023s	sys 0.031s
I20260709 12:24:53.871527 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1767) took 3067 ms. Trace:
W20260709 12:24:53.879444 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.017s	sys 0.013s
W20260709 12:24:53.879527 26093 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.023s	sys 0.033s
W20260709 12:24:53.880396 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.017s	sys 0.013s
W20260709 12:24:53.882575 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 687159us
I20260709 12:24:53.881794 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:53.886415 26045 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:24:53.810284 29272 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.810422 29023 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.879882 26128 rpcz_store.cc:276] 0709 12:24:50.803897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.927476 (+123579us) service_pool.cc:224] Handling call
0709 12:24:50.927854 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.975079 (+ 47225us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:51.226930 (+251851us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b1800450b80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b7827958f 00007f5b712b8571 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd
0709 12:24:51.346387 (+119457us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.443250 (+1096863us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:52.443708 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.443921 (+   213us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:52.443958 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.871232 (+1427274us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b54002c9cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:53.871283 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.871327 (+    44us) raft_consensus.cc:1685] finished
0709 12:24:53.871341 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.871468 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22222,"prepare.queue_time_us":40588,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2590,"raft.queue_time_us":85360,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":55554,"spinlock_wait_cycles":97561728,"thread_start_us":1431637,"threads_started":3,"wal-append.queue_time_us":850626,"wal-append.run_cpu_time_us":3175,"wal-append.run_wall_time_us":1251473}
I20260709 12:24:53.810030 29302 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:53.745322 29065 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:53.867027 26051 rpcz_store.cc:276] 0709 12:24:52.796682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.796982 (+   300us) service_pool.cc:224] Handling call
0709 12:24:52.848707 (+ 51725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.848940 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:53.195058 (+346118us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.857659 (+662601us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.857731 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:53.857788 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.857880 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.858031 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":662255,"threads_started":1}
I20260709 12:24:53.885408 29277 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:53.924835 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.099957 26020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.711s (SENT) [suppressed 27 similar messages]
I20260709 12:24:54.100714 29229 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.928000 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.102291 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.103044 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 141652us
I20260709 12:24:54.021616 26783 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:54.103847 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:53.924372 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.105225 29179 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.105614 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 494234us
I20260709 12:24:54.105887 29228 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.106810 29228 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:53.930123 29194 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:53.961371 29246 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:54.108474 29246 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.107887 29228 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.104079 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:54.109546 29246 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:53.915582 26124 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:53.982645 29055 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.112172 29055 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.105495 29306 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:24:54.098294 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2540) took 1230 ms. Trace:
I20260709 12:24:54.114394 29306 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:54.105856 29179 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.113897 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:54.114903 29306 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.112440 29312 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:54.116413 29306 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.117231 29312 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.118546 29312 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:54.121861 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.013s	sys 0.016s
W20260709 12:24:54.123656 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.126624 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.013s	sys 0.017s
I20260709 12:24:54.129825 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:54.135386 29314 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.136405 29314 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.114662 26052 rpcz_store.cc:276] 0709 12:24:52.952409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.295016 (+342607us) service_pool.cc:224] Handling call
0709 12:24:53.295380 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.317674 (+ 22294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.317721 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.318371 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.708937 (+390566us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:53.708977 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.716355 (+  7378us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.927527 (+211172us) raft_consensus.cc:1685] finished
0709 12:24:53.927549 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.098218 (+170669us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27565,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":36404,"thread_start_us":287457,"threads_started":1,"wal-append.queue_time_us":523405}
I20260709 12:24:54.143316 29194 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:54.144196 29194 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:54.143258 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.125577 29179 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.110772 29023 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:54.181016 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.237543 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:54.237565 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.208608 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:54.276261 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 664589us
I20260709 12:24:54.282641 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:54.282210 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:54.284883 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.278788 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.285339 26198 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:54.285660 26125 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:24:54.232074 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.180871 26052 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [3.2-6.3]
I20260709 12:24:53.869518 28693 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.966886 26047 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:54.192878 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:54.272686 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:54.204998 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:54.301337 26194 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.300585 26053 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.022507 28973 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:54.304720 29323 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.762s	user 0.005s	sys 0.005s
I20260709 12:24:54.305198 29323 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.307668 29323 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:54.295221 29246 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.309080 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:54.309921 26203 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:54.236227 29272 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.310840 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.309818 29023 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.311551 26203 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:54.272473 29190 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.932348 29341 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:24:53.867347 29346 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:54.313534 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:54.312935 29176 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:54.312731 29190 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.315834 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.330770 26198 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:54.333144 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50976.8us
I20260709 12:24:53.878324 29355 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.328745 26203 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.235847 29277 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:54.334098 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:54.337627 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 233836us
W20260709 12:24:54.338737 26169 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.711s (SENT)
I20260709 12:24:54.276732 29306 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.340726 26124 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.340798 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.351727 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:54.342757 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 690) took 1145 ms. Trace:
I20260709 12:24:54.337715 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.353833 26198 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:54.344983 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.101351 29229 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.358224 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.326227 29345 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.314471 29358 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:54.361898 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.355505 26053 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:54.346532 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.355400 26785 rpcz_store.cc:276] 0709 12:24:53.197246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.197378 (+   132us) service_pool.cc:224] Handling call
0709 12:24:53.223193 (+ 25815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.223424 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:53.745619 (+522195us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.342078 (+596459us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:54.342147 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:54.342199 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.342272 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.342698 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":282539,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":144907,"thread_start_us":1117686,"threads_started":3}
I20260709 12:24:54.354656 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.235414 29302 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.362385 26203 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:54.339187 29176 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.355897 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.360129 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.335426 29179 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.336668 29305 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:54.360056 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.371968 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:54.373507 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.381210 26203 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.381685 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.378567 26194 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:54.373306 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:54.383714 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:54.383809 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:54.384639 26194 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:54.383966 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.372284 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.324934 28693 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:54.390031 29353 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:54.381824 29055 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:54.392879 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1317) took 1591 ms. Trace:
I20260709 12:24:54.424880 26200 rpcz_store.cc:276] 0709 12:24:52.801220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.448353 (+647133us) service_pool.cc:224] Handling call
0709 12:24:53.459822 (+ 11469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.464588 (+  4766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.464639 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.466912 (+  2273us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.860789 (+393877us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:53.860856 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.860947 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.392636 (+531689us) raft_consensus.cc:1685] finished
0709 12:24:54.392662 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.392805 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4414,"thread_start_us":370126,"threads_started":1,"wal-append.queue_time_us":762214}
I20260709 12:24:54.425962 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.395076 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 832) took 1948 ms. Trace:
I20260709 12:24:54.396590 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 347) took 1673 ms. Trace:
I20260709 12:24:54.427548 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.428186 26122 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:54.427716 26048 rpcz_store.cc:276] 0709 12:24:52.722943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.723562 (+   619us) service_pool.cc:224] Handling call
0709 12:24:52.724020 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.724547 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:53.510593 (+786046us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.869347 (+358754us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.871660 (+  2313us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.871810 (+   150us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:53.871848 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.871924 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.396347 (+524423us) raft_consensus.cc:1685] finished
0709 12:24:54.396370 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.396518 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":722,"prepare.queue_time_us":61375,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":72816,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":22606,"thread_start_us":1104533,"threads_started":2,"wal-append.queue_time_us":774134}
I20260709 12:24:54.429106 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:54.398132 29240 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.003s
I20260709 12:24:54.399008 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60878 (request call id 13) took 1083 ms. Trace:
I20260709 12:24:54.432170 26046 rpcz_store.cc:276] 0709 12:24:53.315418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.315608 (+   190us) service_pool.cc:224] Handling call
0709 12:24:54.398811 (+1083203us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b540022c254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:54.398935 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":801637248}
W20260709 12:24:54.399442 29189 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.663s	user 0.006s	sys 0.000s
W20260709 12:24:54.399752 29180 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.003s
I20260709 12:24:54.364161 29314 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.404002 29190 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.405442 29369 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.447077 29180 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.447885 29369 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.413856 29229 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:54.455152 29292 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.826s	user 0.008s	sys 0.005s
I20260709 12:24:54.456648 26195 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:54.457203 29273 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.457780 29273 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:54.392905 29362 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:54.417580 29062 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.422807 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.460816 26124 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11.
I20260709 12:24:54.461942 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:24:54.412123 29237 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.000s
I20260709 12:24:53.924881 29243 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.442983 26200 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:54.463900 29065 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:54.465025 26200 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:24:54.466545 26200 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:54.454061 29180 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.460762 29062 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.421196 29179 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.470635 29185 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.426923 26786 rpcz_store.cc:276] 0709 12:24:52.446923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.467280 (+ 20357us) service_pool.cc:224] Handling call
0709 12:24:52.467634 (+   354us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:52.737713 (+270079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:52.737761 (+    48us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:52.776343 (+ 38582us) log.cc:844] Serialized 78 byte log entry
0709 12:24:53.446940 (+670597us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:53.854455 (+407515us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.854706 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.854811 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.394843 (+540032us) raft_consensus.cc:1685] finished
0709 12:24:54.394863 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.395012 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59250,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":50203,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":10755,"thread_start_us":1252560,"threads_started":3,"wal-append.queue_time_us":817514}
I20260709 12:24:54.471522 26125 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.423197 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
W20260709 12:24:54.471460 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.473220 29272 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:54.373988 28814 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:54.474072 28814 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.477576 28693 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.432950 29356 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:54.478550 29356 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.479578 29356 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.470779 29179 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.485347 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:54.359486 29277 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.485644 26198 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:24:54.466940 26124 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:54.452793 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.499855 26124 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.502043 26124 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:54.509744 26122 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:54.513409 26785 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:24:54.516219 26125 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:54.519219 29369 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:54.524668 26124 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.539243 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.539639 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:54.542243 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.348006 29355 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:54.537556 29353 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:54.380671 29365 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 12:24:54.551775 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.606621 26200 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:24:54.563486 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.536656 29229 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.546464 28814 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:54.554373 29264 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.148s	user 0.011s	sys 0.002s
I20260709 12:24:54.565910 29302 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.575156 26120 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:54.579391 29308 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:54.583107 29373 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.584774 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.612819 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.614449 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.614841 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:54.572366 29274 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.835s	user 0.012s	sys 0.000s
W20260709 12:24:54.587329 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 253255us
W20260709 12:24:54.586400 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 234632us
I20260709 12:24:54.597347 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.617105 26122 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:54.583791 28973 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.568032 29176 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.610348 26198 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:54.585800 29358 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:54.620765 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:54.622416 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.623749 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:54.633157 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:54.638357 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.638796 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:54.640904 29369 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:54.641459 29369 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:54.641892 29369 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.644281 29062 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:54.644428 29180 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:54.625247 29373 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.646554 26169 leader_election.cc:400] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11.
I20260709 12:24:54.647089 26169 leader_election.cc:403] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:24:54.612097 29242 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.616371 29183 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.623227 29365 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:54.653767 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:54.659984 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.669876 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.646723 28693 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:54.669966 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 318296us
I20260709 12:24:54.651450 29062 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11.
I20260709 12:24:54.616088 29229 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.680445 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2537) took 1813 ms. Trace:
I20260709 12:24:54.622215 29274 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.622416 29308 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.633105 28973 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.677701 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:54.671561 29180 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:54.686162 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.678643 28693 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.659526 29272 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:54.690129 26054 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:54.662886 29185 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:24:54.689292 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:54.662727 29242 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.680924 26050 rpcz_store.cc:276] 0709 12:24:52.866571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.286185 (+419614us) service_pool.cc:224] Handling call
0709 12:24:53.286513 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:53.286943 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:53.287243 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.743934 (+456691us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:53.853740 (+109806us) log.cc:844] Serialized 52 byte log entry
0709 12:24:53.853933 (+   193us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:53.853981 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.854050 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.098083 (+244033us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b18004213c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1
0709 12:24:54.325744 (+227661us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18004213c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:24:54.680151 (+354407us) raft_consensus.cc:1685] finished
0709 12:24:54.680173 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.680382 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2892,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3616,"raft.queue_time_us":1649,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":12082,"spinlock_wait_cycles":17094272,"thread_start_us":456356,"threads_started":1,"wal-append.queue_time_us":1028866}
I20260709 12:24:54.681769 29277 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.659811 29183 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.694818 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.699276 26202 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.689687 29065 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:54.704269 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.689997 28693 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.708832 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:54.705400 26045 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:54.707049 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:54.702621 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.710618 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:54.710764 26046 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:54.711457 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.695542 29312 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.712610 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:54.713891 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:54.713989 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:54.714881 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:54.724951 26050 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:54.721167 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:54.719082 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.720853 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:54.726948 26122 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:54.725546 26045 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:54.728598 26046 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:54.728900 29229 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:54.729514 29229 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:54.701851 29358 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.734216 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.735834 26128 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:54.736891 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.737416 28973 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:54.737774 29229 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:54.738160 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.739391 29269 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:54.739675 26128 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.739789 29269 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:54.741269 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:54.742601 29237 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:54.742976 29237 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.743330 29237 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.746954 26125 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:54.747133 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:54.747725 28973 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:54.747954 29243 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:54.748632 29243 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:54.749760 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:54.749548 29243 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.750377 28693 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:54.751075 29379 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:54.753264 29240 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:54.738363 29229 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:54.727191 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.757526 26128 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:24:54.708511 29368 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 12:24:54.763125 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 379376us
I20260709 12:24:54.707891 29065 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.767342 29065 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.703938 29302 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.707995 29180 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.770772 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:54.708184 29179 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:24:54.778291 29385 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.782493 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:54.724792 29373 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:54.712584 29376 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:54.784868 29312 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:54.796857 29312 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.796913 29376 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.797844 29312 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.798772 29346 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:54.799068 29312 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.799286 29346 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.789489 29187 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.791608 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 696) took 1426 ms. Trace:
I20260709 12:24:54.792428 29264 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:54.794159 29388 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:54.801136 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1780) took 1135 ms. Trace:
I20260709 12:24:54.801296 29388 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:54.801560 26120 rpcz_store.cc:276] 0709 12:24:53.665567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.668104 (+  2537us) service_pool.cc:224] Handling call
0709 12:24:53.703158 (+ 35054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.610301 (+907143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.800885 (+190584us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:54.801074 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2004252032}
I20260709 12:24:54.801672 29269 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.802748 29269 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.802302 29388 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.788483 26126 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:54.803606 26788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 819) took 3728 ms (client timeout 2234 ms). Trace:
I20260709 12:24:54.761519 29246 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:54.804448 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1777) took 1140 ms. Trace:
W20260709 12:24:54.804065 26788 rpcz_store.cc:269] 0709 12:24:51.075265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.213317 (+138052us) service_pool.cc:224] Handling call
0709 12:24:54.803353 (+3590036us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:54.803514 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3404578560}
I20260709 12:24:54.804845 26126 rpcz_store.cc:276] 0709 12:24:53.664523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.664691 (+   168us) service_pool.cc:224] Handling call
0709 12:24:54.804289 (+1139598us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:54.804392 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581444992}
I20260709 12:24:54.803646 29388 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.807575 29064 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.799297 29376 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:54.800711 29229 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.800365 26783 rpcz_store.cc:276] 0709 12:24:53.365577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.624027 (+258450us) service_pool.cc:224] Handling call
0709 12:24:53.626426 (+  2399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.104435 (+478009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.791238 (+686803us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400433fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:54.791514 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908298112}
I20260709 12:24:54.813535 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:54.814451 26126 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:54.825096 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 104327us
W20260709 12:24:54.829579 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76236.8us
I20260709 12:24:54.838506 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:54.824923 26126 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:24:54.776793 29185 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:24:54.781962 29292 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:24:54.821879 29055 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.782570 29189 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:24:54.818516 28693 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:54.833349 29176 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.840978 29355 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.870433 26129 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:54.811510 29180 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.833809 29376 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:24:54.871243 29291 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:54.862742 29190 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:54.849694 29369 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.888327 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:54.879372 29189 consensus_queue.cc:1243] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:24:54.897759 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59299.7us
I20260709 12:24:54.897208 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:54.860148 29055 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.882128 29274 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.889586 29291 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:54.891695 29190 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:54.905049 29190 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.906298 29190 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.909412 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56074
I20260709 12:24:54.910285 26123 raft_consensus.cc:606] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Received request to transfer leadership
I20260709 12:24:54.889061 29308 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:54.918803 29308 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.919394 29308 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.919875 29308 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.925282 29376 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:54.926288 29055 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:54.922201 29180 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.927481 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.921821 29385 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:54.928431 29387 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:24:54.930433 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1314) took 2366 ms. Trace:
I20260709 12:24:54.891023 29240 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:24:54.932842 29240 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:54.933306 29312 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:54.933789 29312 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.934623 29369 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:54.951277 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1306) took 2454 ms (client timeout 1850 ms). Trace:
W20260709 12:24:54.951761 26195 rpcz_store.cc:269] 0709 12:24:52.497348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.560151 (+ 62803us) service_pool.cc:224] Handling call
0709 12:24:54.951037 (+2390886us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54002b5f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:54.951197 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4358369024}
I20260709 12:24:54.954111 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 692) took 1750 ms. Trace:
I20260709 12:24:54.954516 26787 rpcz_store.cc:276] 0709 12:24:53.203602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.217048 (+ 13446us) service_pool.cc:224] Handling call
0709 12:24:53.217407 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.784018 (+566611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.953878 (+1169860us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54000e2c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:54.954046 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228739328}
I20260709 12:24:54.930830 26201 rpcz_store.cc:276] 0709 12:24:52.563935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.402975 (+839040us) service_pool.cc:224] Handling call
0709 12:24:53.422105 (+ 19130us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:53.433247 (+ 11142us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:53.446697 (+ 13450us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.484362 (+1037665us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b1800419740. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:24:54.698201 (+213839us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:54.703414 (+  5213us) log.cc:844] Serialized 52 byte log entry
0709 12:24:54.703577 (+   163us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:54.703621 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.703694 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.930077 (+226383us) raft_consensus.cc:1685] finished
0709 12:24:54.930176 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.930333 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19573,"prepare.queue_time_us":13860,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":237,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":11714,"spinlock_wait_cycles":18133248,"thread_start_us":213584,"threads_started":1,"wal-append.queue_time_us":1351137}
W20260709 12:24:54.958776 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.901791 29306 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:54.934793 29312 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.935853 28748 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.891064 29393 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 12:24:54.968245 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59499.6us
I20260709 12:24:54.957778 29369 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:54.969379 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70135.7us
I20260709 12:24:54.970752 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:54.972093 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:54.965667 28748 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.970453 29312 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.951853 29237 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.964779 29306 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:54.961715 29323 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:24:54.984887 29055 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:54.984973 29306 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.985685 29055 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.988034 29189 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:54.988713 29189 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:54.989562 29055 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:54.989957 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:54.973778 28748 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:54.991676 29189 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:54.991784 29292 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:54.992285 29292 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:54.990088 29055 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.994227 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.999456 26126 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:54.996896 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.001058 26202 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:54.977227 29373 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.979398 29399 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.002524 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.003371 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.006538 26201 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:24:55.006772 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:55.007300 26782 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: [3.6-6.7]
I20260709 12:24:55.007841 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:54.975181 29369 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.010051 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.011518 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.012305 26785 raft_consensus.cc:2368] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:24:55.012547 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.013010 26126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56074
I20260709 12:24:55.013644 26200 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:55.015239 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.017169 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.018301 29369 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.019318 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.019933 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:54.984094 29393 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:55.020613 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:55.021198 29393 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:55.021623 29393 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:55.021970 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.013620 26126 raft_consensus.cc:606] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Received request to transfer leadership
I20260709 12:24:55.023422 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 448) took 1036 ms. Trace:
I20260709 12:24:55.023954 26127 rpcz_store.cc:276] 0709 12:24:53.986919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.993846 (+1006927us) service_pool.cc:224] Handling call
0709 12:24:55.023352 (+ 29506us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.004485 29055 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.025415 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:55.004555 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.009264 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.027282 26783 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:55.029862 26787 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:55.036020 26788 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:24:54.996591 29292 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.023463 29194 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:55.043337 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.032667 29229 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.004575 29346 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.047009 29229 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:55.053464 26121 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:24:55.029412 29065 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.055709 29371 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.075393 29269 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.075981 29269 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.075903 29371 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:55.078238 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.047492 29292 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:55.093163 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.059813 29229 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:55.094223 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.072742 29055 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:55.103456 29110 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.113546 29110 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.115800 29110 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.117218 29346 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:55.118417 29065 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.109714 29237 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:55.124163 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.126245 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:55.126609 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:55.127245 26126 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:55.133562 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.134588 26787 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:55.056006 29393 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.121405 29246 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:55.138013 29246 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.138015 29388 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:55.122929 29269 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.118902 29308 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:55.136859 29406 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:55.150202 26126 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
W20260709 12:24:55.155812 29407 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:55.157493 29308 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.153680 29065 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:24:55.162199 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.164757 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.189006 26198 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.166608 26200 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:55.167217 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.168833 26196 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:24:55.154707 29399 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.170948 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:55.186966 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.165513 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.195433 26203 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:55.188808 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.197297 29399 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.199317 29179 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:55.199671 29399 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.199844 29179 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:55.159987 29183 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.195047 29246 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.202484 29401 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:55.202932 29401 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:55.059860 29291 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.204741 29291 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.164121 29229 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.205889 29166 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:55.206331 29166 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:55.207585 29229 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.208413 29065 consensus_queue.cc:1243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:55.208976 29065 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:55.209349 29065 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:55.209827 29373 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:55.210232 29065 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.211433 29065 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.212065 26790 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 12:24:55.212123 26200 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:24:55.207201 29166 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.204006 29246 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:55.216926 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 190534us
I20260709 12:24:55.218564 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.220181 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.221144 26018 leader_election.cc:400] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:24:55.222162 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.223026 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:55.224974 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.225832 26195 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:24:55.225970 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.226819 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.226940 26200 raft_consensus.cc:2368] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7.
I20260709 12:24:55.223546 29166 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:55.232437 26202 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:55.233486 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.232437 29261 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:24:55.235440 29261 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.224781 29183 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.236459 29274 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:55.203948 29401 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.237169 29274 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:55.229508 29267 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:55.241433 26198 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:55.242113 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.228935 29269 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.245114 29274 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.246441 29274 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.249146 26203 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:55.236423 29369 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.229240 29346 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:55.250725 29346 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.251255 29346 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:55.233515 29414 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:24:55.200906 29179 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.258790 29179 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.235862 29261 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.240455 29246 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.250686 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.260203 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 449) took 1263 ms. Trace:
I20260709 12:24:55.274631 26129 rpcz_store.cc:276] 0709 12:24:53.996984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.317829 (+320845us) service_pool.cc:224] Handling call
0709 12:24:55.259964 (+942135us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54001f77d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:55.260102 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102559488}
I20260709 12:24:55.268805 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:55.268415 29246 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:55.278272 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.269193 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:55.280593 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.280798 26127 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:55.269536 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:55.281562 26124 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:55.273049 26203 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:55.289812 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:55.261233 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.283409 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:55.294806 26049 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:55.287631 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.300791 26787 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:55.303198 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:55.296031 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.306808 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:55.307700 26124 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.283732 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:55.315537 25939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 178466us
I20260709 12:24:55.315903 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.316962 29190 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:55.318200 26791 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:24:55.319341 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.268905 29261 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:55.270601 29409 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:24:55.321646 29246 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:55.322409 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.324667 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.285310 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.327952 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.328179 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.380532 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.331915 26049 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:24:55.368196 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52983.2us
I20260709 12:24:55.335294 26120 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:55.332094 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:55.330659 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:55.381410 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.380532 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:55.280668 29292 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.269810 29416 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:55.387728 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 93643.7us
I20260709 12:24:55.433784 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.434032 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:55.432174 26049 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:55.387888 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.434998 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.387907 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.394107 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.436995 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:55.390565 26785 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:55.432380 29395 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:24:55.390372 26123 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:55.383240 26786 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:55.397459 26054 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:55.401916 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:55.288975 29399 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 12:24:55.302518 29267 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:24:55.406781 29242 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.309376 29373 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 12:24:55.478839 29373 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:55.479260 29373 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:55.380880 29419 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:55.380775 29305 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.503456 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.407868 26124 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:55.380995 29190 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:55.408113 29292 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.410092 29176 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.410342 29371 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.412484 29273 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:55.410844 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81079.6us
I20260709 12:24:55.511929 29355 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:24:55.526013 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:55.527580 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.528091 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.422254 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.528319 29062 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:55.425665 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.438613 26790 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:55.529707 26203 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:55.440213 26787 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
W20260709 12:24:55.440693 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 102390us
I20260709 12:24:55.533416 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.535419 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:24:55.452080 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.463094 29393 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.538285 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.538539 29411 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.538954 26202 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.539280 29411 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:55.539681 29411 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.476734 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.540030 29411 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.481793 26126 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:55.542685 26785 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:55.481887 26783 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:24:55.482421 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:55.544654 26783 pending_rounds.cc:85] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 5
I20260709 12:24:55.544911 26783 pending_rounds.cc:107] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:24:55.545097 26200 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:55.483757 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.487430 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:55.545993 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62298.5us
W20260709 12:24:55.489282 26121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1785) took 1745 ms (client timeout 1915 ms). Trace:
I20260709 12:24:55.546844 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:55.546811 26121 rpcz_store.cc:269] 0709 12:24:53.744411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.744533 (+   122us) service_pool.cc:224] Handling call
0709 12:24:55.489049 (+1744516us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002ca6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:55.489199 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729744128}
W20260709 12:24:55.547942 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.491863 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:55.548264 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.548839 26194 raft_consensus.cc:2376] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11. Re-sending same reply.
I20260709 12:24:55.545140 26783 raft_consensus.cc:2889] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.272693 29229 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.169s
I20260709 12:24:55.273864 29403 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.185s
I20260709 12:24:55.327138 29302 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 12:24:55.432463 29194 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:24:55.444522 28973 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:55.441910 29237 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:55.516227 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.455260 29166 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:55.512380 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65849.5us
I20260709 12:24:55.510319 29345 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:55.528900 29062 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:55.550944 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63591.4us
I20260709 12:24:55.511096 28748 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:55.572176 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.573745 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.573829 26202 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:55.528379 29272 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:55.555419 26054 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:55.559509 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:55.562461 26196 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:55.522534 29401 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.585079 29369 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.585304 29371 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.567237 26787 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:55.578461 29246 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.581533 29234 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:24:55.542107 29176 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:55.581861 29062 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.582324 29269 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.550860 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.575201 28748 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:55.558571 28973 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.552122 29175 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.557651 29267 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.556564 29355 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.557824 29310 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:55.595759 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44995.8us
I20260709 12:24:55.622035 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.480108 29373 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.510097 29416 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.569847 29345 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:55.557968 29308 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 12:24:55.589877 29228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.270s	user 0.022s	sys 0.031s
I20260709 12:24:55.625950 26094 leader_election.cc:400] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7.
W20260709 12:24:55.624468 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.601927 29369 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:55.625489 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:55.634754 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.595520 29176 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.638574 29292 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:55.639328 29373 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:55.576990 29166 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.597926 29272 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:55.634442 26094 leader_election.cc:403] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:55.596165 29234 consensus_queue.cc:1243] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:55.652199 26791 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:55.635268 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.592676 29240 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.626672 29183 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.636729 28748 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.599403 28973 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.645211 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.649761 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.678862 26050 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:55.650473 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.679646 26046 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.651224 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:55.653313 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.628854 29062 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:55.633982 29228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.023s	sys 0.032s
I20260709 12:24:55.648568 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.654920 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.685649 26049 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
W20260709 12:24:55.659350 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.598851 29064 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:55.651250 29369 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.621325 29190 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.676458 29345 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:55.680627 26045 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:24:55.680644 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.628738 29385 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:24:55.682811 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 93666.4us
I20260709 12:24:55.680644 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.599116 29269 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.677801 29183 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.736768 29183 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.737121 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.738307 26052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:24:55.738790 29269 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.739645 29305 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.684604 26054 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:55.594014 29384 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:24:55.686784 26050 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:24:55.743422 26050 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 6
I20260709 12:24:55.743708 26050 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:55.743943 26050 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.738847 26052 raft_consensus.cc:606] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.745672 26052 raft_consensus.cc:614] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:55.689590 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.693511 26047 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:24:55.694324 26051 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:55.696148 26197 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
W20260709 12:24:55.697340 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.701213 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:55.748957 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47859.5us
I20260709 12:24:55.749872 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.658664 29240 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.705376 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.699788 28748 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.713814 29166 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:55.717142 29345 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.658169 29310 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:24:55.685762 29314 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:24:55.700549 29369 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:24:55.726583 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.725886 29272 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:55.685240 29229 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:55.692210 29356 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7.
I20260709 12:24:55.704205 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:55.750891 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:55.758019 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52284.1us
I20260709 12:24:55.767315 29240 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.769747 29237 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.786633 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:55.806118 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.786801 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.806525 26790 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:55.717833 26046 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:55.791746 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.767683 29292 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.802177 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.774000 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.808378 26045 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:55.803902 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99790.2us
I20260709 12:24:55.813294 26200 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:55.700438 29419 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:24:55.818877 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.817032 26202 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:24:55.820994 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.821617 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:55.815372 29189 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:55.778855 29369 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.831211 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.772145 29179 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:24:55.799989 29229 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:55.846133 26046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
W20260709 12:24:55.844386 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.846794 26046 raft_consensus.cc:606] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.818305 28814 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:55.818437 29242 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.826790 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:55.832795 29243 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.836664 29356 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:55.858330 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 377960us
I20260709 12:24:55.871939 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.872848 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.873184 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.873793 26052 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:55.874284 26047 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:55.875051 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.878726 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.880255 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.881004 26045 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:55.805959 29420 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:24:55.845271 29189 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:55.856177 26046 raft_consensus.cc:614] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:55.836416 29065 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:55.859442 29242 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:55.863770 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 141380us
I20260709 12:24:55.884776 26047 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:24:55.829106 29385 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:55.905575 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.882869 26045 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:55.848595 29229 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.906894 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.903012 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.882577 29399 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.882807 29064 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:24:55.908649 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 106589us
I20260709 12:24:55.884915 29308 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.945555 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.946249 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.947448 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.944730 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:55.857071 29421 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:24:55.971798 26047 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:55.976202 26168 connection.cc:256] RPC call timeout handler was delayed by -1.4344023839994406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 12:24:55.977290 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.973554 26046 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:55.976491 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 145398us
I20260709 12:24:55.981171 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.983244 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:55.906988 29176 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:55.983731 26168 connection.cc:256] RPC call timeout handler was delayed by -1.3281978019995222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 12:24:55.901136 28814 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:55.989313 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:55.909045 29385 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:56.001435 26046 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:56.002840 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.996760 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:55.902240 29314 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.908011 29242 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.004066 26045 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:56.014520 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:56.001078 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 214329us
I20260709 12:24:56.001911 29416 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.018929 26046 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:55.970582 29310 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.014871 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:55.978250 29237 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:56.025357 26124 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:56.014770 29064 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:56.015720 26126 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:56.016381 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:55.987146 29395 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:56.021950 29055 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.020303 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:56.006408 29356 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.017151 26123 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: [3.6-6.7]
I20260709 12:24:56.031446 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.018323 29308 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:56.034358 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:56.013489 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:24:56.005892 29385 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:56.012549 29399 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:55.996956 29393 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.038034 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:56.032275 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.060029 26124 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:56.041481 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.060493 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:56.061012 26122 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:56.061038 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:56.042178 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.042708 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.044147 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:56.062242 26045 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
W20260709 12:24:56.045181 29428 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:24:56.045300 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1323) took 1942 ms (client timeout 1692 ms). Trace:
I20260709 12:24:56.046315 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:56.063692 26050 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:55.996788 29269 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:24:56.046020 29064 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.051663 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.055857 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:55.978075 29411 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.065311 26126 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:56.038511 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:56.063308 26197 rpcz_store.cc:269] 0709 12:24:54.103185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.164439 (+1061254us) service_pool.cc:224] Handling call
0709 12:24:56.045122 (+880683us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400378554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:56.045232 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071246080}
I20260709 12:24:56.057376 26047 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:24:56.066854 26047 pending_rounds.cc:85] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 8
I20260709 12:24:56.067013 29269 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.066635 29411 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:56.067139 26047 pending_rounds.cc:107] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:24:56.068070 26047 raft_consensus.cc:2889] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:56.067898 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84736.6us
I20260709 12:24:56.065941 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.072505 29274 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:56.062078 26120 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:56.005692 29420 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:24:56.081251 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:56.091287 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.057106 29384 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:56.092003 26121 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:56.029479 29179 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:24:56.017637 29314 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.059283 29399 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:56.104631 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:56.102705 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.106348 26196 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 5.
I20260709 12:24:56.080511 29065 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.119464 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.120177 26194 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:56.036564 29237 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.111783 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:56.113245 26195 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:56.115617 26052 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:24:56.115914 29246 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:56.084807 29228 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:56.117352 26203 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:24:56.118003 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.118288 26200 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.077939 29442 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:56.121027 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:56.023280 29189 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.096201 29384 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:56.122397 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.184221 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.184736 26194 raft_consensus.cc:2410] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:56.185026 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:56.039716 29395 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:56.187645 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 66714.1us
I20260709 12:24:56.106508 29399 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:56.129510 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:56.047729 29055 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.176872 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:56.183442 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:56.056919 29356 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.133991 26201 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.190426 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.092628 28814 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:56.191881 29429 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
W20260709 12:24:56.196877 29435 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:24:56.200554 29384 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.205140 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.312927 26198 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:56.206285 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.313916 26194 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:24:56.289379 29401 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.001s
I20260709 12:24:56.131722 29246 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:56.209120 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:56.319490 26203 raft_consensus.cc:2376] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10. Re-sending same reply.
I20260709 12:24:56.209352 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.320473 26195 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:24:56.211637 29345 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:56.131484 29306 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.219414 29443 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:56.222020 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:56.324419 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 114712us
W20260709 12:24:56.223291 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.227965 29237 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:56.243319 29183 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.005s
I20260709 12:24:56.131645 29411 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
W20260709 12:24:56.258657 29346 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.007s	user 0.007s	sys 0.007s
I20260709 12:24:56.262908 29442 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.266024 29395 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:56.169919 29355 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.544s	user 0.000s	sys 0.011s
I20260709 12:24:56.278782 26197 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:56.323637 29443 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.117429 29314 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:56.207006 28973 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.515s	user 0.003s	sys 0.003s
I20260709 12:24:56.314880 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:56.288383 29373 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.648s	user 0.011s	sys 0.001s
W20260709 12:24:56.306358 29410 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
W20260709 12:24:56.306599 29441 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.066s	user 0.005s	sys 0.015s
I20260709 12:24:56.321727 26782 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:24:56.328866 26195 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:56.329725 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.320619 29242 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.331892 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.386418 26201 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:56.344101 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:56.389122 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45152.6us
I20260709 12:24:56.390308 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:56.346694 29449 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.356552 29183 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.360760 26199 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:24:56.279927 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 198745us
I20260709 12:24:56.373216 26121 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:56.376394 26197 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:24:56.380044 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.331472 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.372634 29355 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.333973 29345 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.357343 29454 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:56.374980 29443 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.374699 29176 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:56.421924 29384 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.365633 29346 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.430768 29455 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:24:56.433310 29419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.013s	sys 0.014s
I20260709 12:24:56.379477 28973 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:56.430967 29243 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.004s
I20260709 12:24:56.424058 29302 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:56.572866 26121 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:56.549253 29401 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.591935 29401 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.373171 29246 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.589533 26048 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:24:56.566054 29369 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.001s
W20260709 12:24:56.570178 29419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.013s	sys 0.015s
I20260709 12:24:56.424990 29228 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.149s
I20260709 12:24:56.552073 29314 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.566589 29455 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:56.610924 29373 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.626039 29356 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:56.595063 29243 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.626317 29373 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:56.618934 29062 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.922s	user 0.012s	sys 0.000s
I20260709 12:24:56.641132 29373 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:56.616561 29267 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.991s	user 0.012s	sys 0.001s
W20260709 12:24:56.626786 29458 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:24:56.643071 26123 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:56.647724 26123 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:56.649448 29453 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:56.653831 29384 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:56.656265 29346 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:56.617853 29369 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:24:56.661882 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 271906us
I20260709 12:24:56.664827 26203 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.611148 29302 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:56.672205 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.617370 29376 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:24:56.676787 29183 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:56.679082 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.679440 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.679776 29242 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.628711 29314 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.680076 29459 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.680449 29176 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.681111 29459 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.681173 29176 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.682296 29314 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.682689 29401 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.683445 29459 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:56.684203 28973 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:56.684310 29308 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.005s
I20260709 12:24:56.684823 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:56.684628 29237 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:56.671802 29438 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.350s	user 0.002s	sys 0.000s
I20260709 12:24:56.686097 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.686488 29345 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.684432 29459 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:56.685293 29308 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.687976 26791 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:56.689004 29454 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.689836 29454 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:56.690260 29454 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:56.690721 29454 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:56.657711 29419 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:24:56.693333 29229 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.716s	user 0.006s	sys 0.003s
I20260709 12:24:56.692824 29419 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.693779 29229 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.696646 29234 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:56.687882 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.697366 29234 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:56.698097 29240 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.880s	user 0.011s	sys 0.000s
I20260709 12:24:56.698230 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.698557 29240 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.699560 29234 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:56.700783 29229 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.663878 29267 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:56.672977 29269 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.000s
I20260709 12:24:56.703094 29269 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.703766 26051 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.703434 26126 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 12:24:56.737387 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62486.3us
I20260709 12:24:56.662005 29246 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.755905 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:56.685370 29456 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:56.766239 28748 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:56.767042 28748 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:56.767102 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:56.768255 29228 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:56.769166 29228 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:56.770082 29228 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.757712 29356 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:24:56.772719 29055 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:56.773293 29055 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.773929 29055 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:56.774288 29055 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.774700 29055 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.776568 29369 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.781468 29240 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:56.784250 29385 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.000s
I20260709 12:24:56.784770 29385 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:56.786242 26049 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:24:56.790931 29384 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:56.791363 29356 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:56.793292 29369 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.793885 29369 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.793320 28973 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:56.797281 28973 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:56.797473 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56074
I20260709 12:24:56.798209 26129 raft_consensus.cc:606] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Received request to transfer leadership
I20260709 12:24:56.798167 28973 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.799485 28973 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.803596 26195 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:56.805688 26054 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:56.806471 26053 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:56.807432 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:56.808609 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:56.808871 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:56.809362 26054 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:56.808027 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.810052 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:56.675498 29189 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:56.811841 26194 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.814985 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:56.819845 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:56.820580 26053 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.818663 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:24:56.779731 29454 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.819120 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:56.821385 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:56.806699 29449 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.700191 29234 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.768253 28748 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.841399 26194 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:56.811659 29385 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.826807 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:56.830665 26052 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 5
I20260709 12:24:56.932140 26052 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:56.932518 26052 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:56.857556 29454 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:56.780206 29443 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.859089 29234 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.851806 29302 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:56.810761 29369 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:56.937352 29234 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:56.913956 29445 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.674s	user 0.003s	sys 0.007s
I20260709 12:24:56.848168 29189 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:56.915331 29399 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.002s
I20260709 12:24:56.930795 29385 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:56.896876 29446 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.659s	user 0.011s	sys 0.001s
W20260709 12:24:56.987673 26197 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.007s	sys 0.033s
W20260709 12:24:56.989279 26197 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.007s	sys 0.033s
I20260709 12:24:56.992406 26053 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:56.977155 29246 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:56.984822 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 286643us
I20260709 12:24:56.998615 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:57.005008 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:57.007995 29234 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.004169 29469 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.007831 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:57.011310 29469 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.014747 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:57.018319 29465 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:24:57.024660 29443 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.018697 29419 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.019197 29189 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:56.986874 29399 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.029655 29399 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:57.023203 29055 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.001441 29246 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.068742 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:56.989219 29385 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.080576 26199 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:57.076895 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:57.064787 29469 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:57.079746 29183 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:57.065176 29269 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:24:57.065050 29234 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:24:57.072319 29369 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:24:57.110414 29346 raft_consensus.cc:2764] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:57.104429 29469 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.149111 29469 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.072081 29345 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:24:57.135826 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2564) took 1081 ms. Trace:
I20260709 12:24:57.118774 29385 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.153105 26121 rpcz_store.cc:276] 0709 12:24:56.053961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.702339 (+648378us) service_pool.cc:224] Handling call
0709 12:24:56.704691 (+  2352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.704897 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:56.764421 (+ 59524us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.135283 (+370862us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:57.135367 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:57.135432 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.135529 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.135736 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59885,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":29,"raft.queue_time_us":6947,"thread_start_us":362912,"threads_started":1}
I20260709 12:24:57.160784 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.161830 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:57.163236 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:57.165380 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.166209 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:57.167263 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:57.168771 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.164055 26202 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:24:57.170990 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.172734 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.173533 26202 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:24:57.177572 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.180112 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:57.183503 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:57.188283 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:57.199101 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:57.158737 29385 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:57.182618 29455 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.204687 29055 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:57.206027 26783 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:57.185520 29314 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.209342 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:57.216813 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:57.217541 26786 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:57.219563 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:57.219884 29314 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:57.221100 29471 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:57.222445 29314 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.188481 29345 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:57.222582 29476 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 12:24:57.209827 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136951us
I20260709 12:24:57.234253 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 721) took 1673 ms. Trace:
I20260709 12:24:57.235078 26790 rpcz_store.cc:276] 0709 12:24:55.561130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.561836 (+   706us) service_pool.cc:224] Handling call
0709 12:24:55.806200 (+244364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.807019 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.233949 (+1426930us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400680450. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:57.234170 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561137536}
I20260709 12:24:57.246853 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:57.222999 29176 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:57.228223 29345 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:57.218288 29455 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.242475 29246 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.258033 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:57.261828 29228 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.284741 29228 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.286407 26786 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:24:57.287187 29228 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.287818 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:57.260186 29272 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:57.257094 28748 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:24:57.297982 29228 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:57.262457 29384 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:24:57.319516 29228 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:57.313088 28748 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:57.320504 28748 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:57.320943 29228 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.325240 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 465) took 1273 ms. Trace:
I20260709 12:24:57.325776 26125 rpcz_store.cc:276] 0709 12:24:56.051784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.051897 (+   113us) service_pool.cc:224] Handling call
0709 12:24:56.052146 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.265849 (+213703us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:56.266151 (+   302us) log.cc:844] Serialized 14 byte log entry
0709 12:24:56.426117 (+159966us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:56.426187 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:56.426257 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.324907 (+898650us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:57.324968 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.325157 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55410,"prepare.queue_time_us":506,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3122,"raft.queue_time_us":55103,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":12194,"spinlock_wait_cycles":490559872,"thread_start_us":106447,"threads_started":1,"wal-append.queue_time_us":377051}
I20260709 12:24:57.318485 29455 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.321568 29442 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:57.317422 29440 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.084s	user 0.014s	sys 0.005s
I20260709 12:24:57.302600 29269 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.348413 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:57.348927 26787 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:57.349637 26786 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:57.344669 29183 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:57.368184 29455 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:57.369268 29455 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:57.370388 29455 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:57.369246 29183 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.300032 29272 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.342928 29399 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:57.297014 29345 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:57.382215 29464 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.767s	user 0.003s	sys 0.004s
I20260709 12:24:57.366882 29269 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.321573 28748 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.385556 29399 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.400561 29314 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.400095 29345 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.401374 29455 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.421324 28748 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:57.413569 29314 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:57.431056 29482 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:24:57.432207 29469 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.433087 29455 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.434810 29469 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:57.436851 29269 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.438426 29399 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.440707 29314 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:57.391310 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 282632us
I20260709 12:24:57.475943 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 362) took 1254 ms. Trace:
I20260709 12:24:57.479475 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1354) took 1103 ms. Trace:
I20260709 12:24:57.480185 26197 rpcz_store.cc:276] 0709 12:24:56.375606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.424148 (+ 48542us) service_pool.cc:224] Handling call
0709 12:24:56.424658 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.990423 (+565765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.990462 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.999917 (+  9455us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.256748 (+256831us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:57.256782 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.257422 (+   640us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.478524 (+221102us) raft_consensus.cc:1685] finished
0709 12:24:57.478585 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.479044 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2269,"raft.queue_time_us":128670,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":21012,"thread_start_us":819463,"threads_started":2,"wal-append.queue_time_us":384896}
I20260709 12:24:57.489185 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 503) took 1117 ms. Trace:
I20260709 12:24:57.489779 26201 rpcz_store.cc:276] 0709 12:24:56.372064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.387250 (+ 15186us) service_pool.cc:224] Handling call
0709 12:24:56.387668 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.388082 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.388149 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.932043 (+543894us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:56.943507 (+ 11464us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.181434 (+237927us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:57.181697 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.181791 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.488461 (+306670us) raft_consensus.cc:1685] finished
0709 12:24:57.488503 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.488947 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11254,"raft.queue_time_us":10716,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":26728,"spinlock_wait_cycles":13385088,"thread_start_us":236095,"threads_started":1,"wal-append.queue_time_us":349434}
W20260709 12:24:57.495067 26049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2556) took 1623 ms (client timeout 1503 ms). Trace:
I20260709 12:24:57.497359 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 355) took 2062 ms. Trace:
I20260709 12:24:57.497239 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:57.497944 26048 rpcz_store.cc:276] 0709 12:24:55.434619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.434745 (+   126us) service_pool.cc:224] Handling call
0709 12:24:57.496904 (+2062159us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:57.497181 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653070080}
W20260709 12:24:57.495720 26049 rpcz_store.cc:269] 0709 12:24:55.871575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.871712 (+   137us) service_pool.cc:224] Handling call
0709 12:24:57.494527 (+1622815us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:57.494819 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100418432}
I20260709 12:24:57.476727 26047 rpcz_store.cc:276] 0709 12:24:56.221617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.221774 (+   157us) service_pool.cc:224] Handling call
0709 12:24:56.222260 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.222678 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.222718 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:57.237527 (+1014809us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b2c000014c8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:24:57.237963 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.474812 (+236849us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:57.474960 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.475140 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.475224 (+    84us) raft_consensus.cc:1685] finished
0709 12:24:57.475247 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.475711 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":451,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":167474944,"thread_start_us":212139,"threads_started":1,"wal-append.queue_time_us":185011}
I20260709 12:24:57.502314 29269 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.513206 29183 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.512986 29269 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.514526 29183 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:57.528434 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.529927 29489 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.533502 29490 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.534516 29490 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.536888 29314 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:57.538110 29314 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.542042 29314 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.545782 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 363) took 1317 ms. Trace:
I20260709 12:24:57.549299 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:57.549544 29183 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.546494 26051 rpcz_store.cc:276] 0709 12:24:56.228098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.228280 (+   182us) service_pool.cc:224] Handling call
0709 12:24:56.228589 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.704147 (+475558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.545091 (+840944us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:57.545546 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092324608}
I20260709 12:24:57.556173 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 848) took 1437 ms. Trace:
I20260709 12:24:57.561673 29314 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:57.551743 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:57.557873 26791 rpcz_store.cc:276] 0709 12:24:56.119005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.119761 (+   756us) service_pool.cc:224] Handling call
0709 12:24:56.120031 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.688460 (+568429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.555463 (+867003us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400650254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:24:57.555904 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305701632}
I20260709 12:24:57.568594 29506 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.569859 29506 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.564539 25952 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 5, term changed from 4 to 5, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129), VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130), VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131), VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 5 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:57.572494 29469 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.573437 29469 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.573943 29490 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.575712 29269 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.576951 29469 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:57.582690 29183 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.582815 29399 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:57.583753 29399 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.586881 29399 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.587831 29314 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.592689 28748 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.593241 29314 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.593581 28748 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.594663 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.583776 29183 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.600018 29323 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:57.600680 29323 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:57.601104 29506 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:57.602046 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.612902 29376 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:57.613247 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 116530us
I20260709 12:24:57.602427 29399 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:24:57.616863 29399 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:57.617291 29399 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:57.602226 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:57.618945 29399 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.619359 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:57.614699 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:57.601347 28748 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.602329 29323 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.620679 29503 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:57.621284 29503 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.621287 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.621948 29503 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.621831 29323 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:57.623536 29385 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:57.625289 29388 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:57.625777 29388 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:57.604030 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:57.606518 29346 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:57.610020 29506 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:57.610327 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:57.610819 26094 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.012s	sys 0.027s
I20260709 12:24:57.611050 29492 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.598870 29183 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.619845 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:57.632320 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:57.635314 26094 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.013s	sys 0.027s
I20260709 12:24:57.639048 29228 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.639159 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:57.640305 26054 raft_consensus.cc:2376] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10. Re-sending same reply.
I20260709 12:24:57.641192 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:57.639998 29228 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.644268 29228 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.645519 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:57.645931 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:57.582719 29419 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:57.646781 29419 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.648054 26785 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:57.647883 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:57.649712 29419 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.650141 26054 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:57.651269 26203 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.650913 29506 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.651751 26201 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:24:57.654294 29506 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.654498 28748 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:57.655515 28748 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:57.656879 26197 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:57.657497 29272 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:57.657845 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:57.659147 29356 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:57.660270 29356 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:57.660636 29356 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:57.632915 29346 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:57.661787 29356 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.633169 29469 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:57.664477 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.663448 29356 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:57.664688 26782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 836) took 2374 ms (client timeout 1823 ms). Trace:
I20260709 12:24:57.663730 29469 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.649731 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:57.665148 26782 rpcz_store.cc:269] 0709 12:24:55.289765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.327622 (+ 37857us) service_pool.cc:224] Handling call
0709 12:24:57.664198 (+2336576us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b540069e6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:24:57.664467 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282599680}
I20260709 12:24:57.673250 29492 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.675761 29492 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.672389 29399 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.627329 29388 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.658252 28748 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:57.663367 29346 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:57.682919 29506 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:57.689096 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 856705us
W20260709 12:24:57.693950 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74734.7us
I20260709 12:24:57.696964 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:57.697556 29477 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.489s	user 0.003s	sys 0.000s
W20260709 12:24:57.697953 29481 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 12:24:57.698221 26120 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:57.699244 29376 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:57.700575 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.702670 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:57.703354 26128 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:57.704259 29242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.011s	sys 0.000s
W20260709 12:24:57.700426 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 86095.6us
W20260709 12:24:57.704555 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.727048 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73744us
I20260709 12:24:57.770954 28693 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:57.718854 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:57.695922 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:57.741771 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:57.705619 25952 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:57.699985 29376 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.687554 29183 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:57.931030 28748 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.933022 29194 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:57.946590 26045 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:57.933527 29272 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:57.947037 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:57.947691 29511 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:57.778955 29292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.423s	user 0.015s	sys 0.017s
W20260709 12:24:57.931488 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.937893 26051 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:57.940737 26054 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:57.939448 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
W20260709 12:24:57.829818 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.941908 26053 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:57.950824 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:57.940387 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:57.937741 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:57.939265 26050 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:57.979995 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:57.957896 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.951622 26128 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
W20260709 12:24:57.987195 29455 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.020s
I20260709 12:24:57.987416 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:57.987092 29194 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.988430 26125 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:57.988440 28748 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.940547 29403 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:57.993597 28748 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.992049 29194 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:57.995747 26048 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:24:57.997244 29492 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.998397 29346 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.992672 29403 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:57.999359 29511 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.001032 29403 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.002251 29511 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.003377 29272 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.004678 29455 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.005076 29183 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.005414 29455 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.005946 29272 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.007808 29455 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:58.008023 29345 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.020s
I20260709 12:24:58.008872 26047 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:58.010114 29388 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.012696 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:24:58.001400 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 305319us
I20260709 12:24:57.935377 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:57.982512 29292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.723s	user 0.020s	sys 0.019s
I20260709 12:24:58.031368 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.033345 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:24:57.936780 29506 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:58.032917 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.032518 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:58.014696 28748 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.008044 29183 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.055351 26127 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:58.012475 29305 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:24:58.062601 26120 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:58.061921 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:58.063459 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:58.069157 26045 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 5.
I20260709 12:24:58.054280 29272 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.048131 29345 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:58.074260 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.085631 26785 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:58.072845 29511 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:58.090040 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 110349us
I20260709 12:24:57.984835 28693 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.089641 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:24:58.096786 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43649.4us
I20260709 12:24:58.097361 26050 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:58.094863 26782 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:58.099913 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 162484us
I20260709 12:24:58.079624 29292 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.110324 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.094659 29511 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.114945 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.120846 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.121021 29419 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:58.121157 29508 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:24:58.121817 26121 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:24:58.122853 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.123486 29455 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.124063 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.124569 26786 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:24:58.125092 26786 pending_rounds.cc:85] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 8
I20260709 12:24:58.125422 26786 pending_rounds.cc:107] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:24:58.125734 26786 raft_consensus.cc:2889] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:58.127112 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.127954 29505 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:58.128444 29504 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.128741 29505 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.129117 29504 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.129561 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.130424 26203 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:58.131237 29505 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.131501 29504 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.132418 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.134605 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.108695 29183 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:58.106032 29469 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:58.141197 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.159929 26045 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:58.160480 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 225410us
I20260709 12:24:58.120489 29511 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.094587 28748 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.167938 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.172506 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.106504 29506 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.176338 26203 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:58.178231 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.162148 29323 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.007s
I20260709 12:24:58.148954 29183 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:58.161707 29503 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.006s
I20260709 12:24:58.134748 29455 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.198167 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:58.196046 26785 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:24:58.175236 29492 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:58.201941 29513 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.258s	user 0.000s	sys 0.003s
I20260709 12:24:58.222693 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.204522 29183 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.228742 26786 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:24:58.222121 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.229835 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.242333 26200 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:58.243808 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:58.252224 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.252755 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.253114 26194 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:58.253664 26196 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:58.254406 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.255493 26129 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:58.256433 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.256645 29376 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:58.257265 29376 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.257767 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 143875us
I20260709 12:24:58.257908 29376 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.259249 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.260468 29504 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:58.260751 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.261650 26201 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:58.257345 26126 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:58.261662 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.265048 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.274356 28693 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.276748 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.278400 26787 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:58.218391 29345 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.279851 29228 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:58.280594 29228 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.281562 28748 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.282826 29228 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.285832 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:58.286242 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:24:58.286633 26046 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:58.287079 26053 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.287487 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:58.288350 26046 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:58.290007 29455 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:58.290767 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 156033us
I20260709 12:24:58.291710 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:58.298548 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.300896 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:24:58.239444 29356 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.005s
I20260709 12:24:58.318428 26121 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:58.209764 29503 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.293005 26052 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:58.333935 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.337360 26126 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.338096 26124 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:24:58.338336 26782 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:24:58.345813 26168 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.503s (SENT)
I20260709 12:24:58.340389 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.340955 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:24:58.351575 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.356580 26045 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:58.347898 26786 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:58.363871 26050 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7.
I20260709 12:24:58.367494 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:58.330163 29376 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.328193 29504 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:58.374814 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.377799 26120 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 12:24:58.378750 29514 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 12:24:58.384462 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 231097us
I20260709 12:24:58.386222 26126 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
W20260709 12:24:58.388863 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 97308us
W20260709 12:24:58.389833 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.390939 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.345163 29314 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.400702 29314 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.401124 29314 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.405292 29376 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.406068 29376 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.408824 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.412391 28693 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.412986 28693 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.417071 29509 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.424180 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:58.354455 29323 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.424695 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.425457 29323 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.426647 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 469) took 2371 ms. Trace:
I20260709 12:24:58.354562 29356 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.431521 29517 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:58.432026 29356 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.433447 29517 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:58.434418 29356 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.436615 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:58.438755 29455 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:58.439714 29455 consensus_queue.cc:1243] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:58.439323 25952 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 3 to 7, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:58.441449 29455 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:58.427289 26123 rpcz_store.cc:276] 0709 12:24:56.052800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.652372 (+599572us) service_pool.cc:224] Handling call
0709 12:24:56.661253 (+  8881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.661612 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.661641 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.668924 (+  7283us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.720203 (+1051279us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b1800340d20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:24:57.869538 (+149335us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:57.869629 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.869759 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.384314 (+514555us) raft_consensus.cc:1685] finished
0709 12:24:58.384340 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.423968 (+ 39628us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58695,"raft.queue_time_us":472,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10775,"spinlock_wait_cycles":69324672,"thread_start_us":148904,"threads_started":1,"wal-append.queue_time_us":1421487}
I20260709 12:24:58.443107 29455 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.443866 29455 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.445457 29305 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.445951 29305 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:24:58.482297 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 149522us
I20260709 12:24:58.441298 25952 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 5, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:24:58.484351 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:58.504653 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113914us
W20260709 12:24:58.505784 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 388852us
I20260709 12:24:58.507300 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:58.513186 26787 pending_rounds.cc:85] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
I20260709 12:24:58.513515 26787 pending_rounds.cc:107] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.7
W20260709 12:24:58.514137 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.517872 26051 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:58.370393 29345 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:58.513757 26787 raft_consensus.cc:2889] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:58.512981 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.519239 29345 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.500348 29509 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.504024 29346 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:58.575383 29346 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.576172 29469 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:58.577692 29346 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.577809 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:58.580842 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.581950 29517 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:58.581996 29506 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.582521 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:58.583448 29517 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.583644 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.583937 29517 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.556079 29292 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.584415 29517 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.586395 26053 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:58.586501 29511 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.527915 26045 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:58.532215 26783 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:24:58.537786 26782 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:58.548629 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.552140 29399 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:58.553109 28693 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:58.523223 29504 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:58.556865 26790 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:58.559098 26123 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:58.560214 26120 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:58.560600 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.561667 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.519881 25952 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 4 to 6, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 6 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:24:58.563678 29323 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.564828 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.564977 26124 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:58.568398 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.569731 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:58.571820 29272 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:58.573076 29503 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.573493 29356 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:58.573989 29194 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:58.574561 25950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 293) took 17806 ms (client timeout 15000 ms). Trace:
I20260709 12:24:58.447139 29305 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.590171 26049 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:58.595022 26122 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:24:58.596112 26127 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.005s
I20260709 12:24:58.597342 26054 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:58.597249 26125 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:58.588526 26783 pending_rounds.cc:85] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
I20260709 12:24:58.610582 26126 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:58.611953 26791 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:24:58.630637 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61040us
I20260709 12:24:58.630336 26127 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:24:58.615687 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 67246.1us
I20260709 12:24:58.631167 26783 pending_rounds.cc:107] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:58.633136 26783 raft_consensus.cc:2889] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:58.604699 29504 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.635776 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:58.630846 25952 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 5 to 6, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 6 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.641451 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:58.639762 29242 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.644618 29522 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:58.645109 26785 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:24:58.655282 29356 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:58.651540 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.596850 29509 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.651556 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.598277 29399 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:58.612609 28693 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:58.668733 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.614758 29376 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.672271 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.667749 25952 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.674363 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.656229 29504 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.682042 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.667276 29242 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.684135 26121 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:24:58.680182 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:58.686959 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:58.688683 26122 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:58.690507 26124 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.694820 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:58.677331 29272 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.700987 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:58.703701 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62343.3us
I20260709 12:24:58.703992 26127 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:58.702849 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.659948 29314 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:58.709981 29511 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.710803 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.718315 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.718148 26127 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:58.699826 29356 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:58.682092 29228 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:24:58.706110 29305 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.736690 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.706238 29376 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:58.734649 26125 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:24:58.738049 26783 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:58.739679 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:58.725596 29323 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:58.752396 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.743595 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:58.777634 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 518455us
I20260709 12:24:58.775346 26120 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
W20260709 12:24:58.779322 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 95329.2us
I20260709 12:24:58.759351 29272 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:58.783334 29272 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:58.790583 26125 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:58.758662 29323 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.793521 26124 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:24:58.794682 25952 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 1 to 4, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 4 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.797930 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.784293 29272 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.804221 29272 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.804721 29356 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.808971 25952 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.812094 29183 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:58.812773 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80533.4us
I20260709 12:24:58.813278 29455 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:58.814509 29517 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.817396 26783 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:58.817682 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.818559 29509 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:58.819753 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.820212 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.822937 26786 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:58.826087 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:58.775359 29518 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:58.835853 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.802772 29323 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:58.614151 25950 rpcz_store.cc:269] 0709 12:24:40.980842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.001414 (+ 20572us) service_pool.cc:224] Handling call
0709 12:24:58.574257 (+17572843us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:49.248427 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:49.249198 (+   771us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:49.249266 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:24:49.249282 (+    16us) tablet.cc:662] Decoding operations
0709 12:24:49.325153 (+ 75871us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:49.325262 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:49.325311 (+    49us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:24:49.394499 (+ 69188us) tablet.cc:701] Row locks acquired
0709 12:24:49.394547 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:24:49.394783 (+   236us) write_op.cc:270] Start()
0709 12:24:49.394953 (+   170us) write_op.cc:276] Timestamp: P: 1783599889394691 usec, L: 0
0709 12:24:49.394979 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:24:49.519411 (+124432us) log.cc:844] Serialized 9826 byte log entry
0709 12:24:49.592015 (+ 72604us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a0e28 00007f5b77ddb581 00007f5b77ddad58 00007f5b77ddf9f1 00007f5b77ddf9a9 00007f5b77ddf939 00007f5b77ddf901 00007f5b77ddeb2d 0000000000388634 0000000000388479
0709 12:24:50.333241 (+741226us) op_driver.cc:464] REPLICATION: finished
0709 12:24:50.890968 (+557727us) write_op.cc:301] APPLY: starting
0709 12:24:50.927061 (+ 36093us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:51.083954 (+156893us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:51.106840 (+ 22886us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:52.220701 (+1113861us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:52.221330 (+   629us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 12:24:52.221403 (+    73us) write_op.cc:312] APPLY: finished
0709 12:24:52.222151 (+   748us) log.cc:844] Serialized 164 byte log entry
0709 12:24:52.437242 (+215091us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:52.495423 (+ 58181us) write_op.cc:454] Released schema lock
0709 12:24:52.544560 (+ 49137us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:52.658831 (+114271us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:52.658859 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14085,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":418553,"apply.run_cpu_time_us":17869,"apply.run_wall_time_us":1793460,"num_ops":24,"prepare.queue_time_us":252209,"prepare.run_cpu_time_us":20681,"prepare.run_wall_time_us":343875,"raft.queue_time_us":510724,"raft.run_cpu_time_us":19942,"raft.run_wall_time_us":431216,"replication_time_us":895258,"spinlock_wait_cycles":57063680,"thread_start_us":737012,"threads_started":5,"wal-append.queue_time_us":314002,"wal-append.run_cpu_time_us":85885,"wal-append.run_wall_time_us":2857249}]]}
I20260709 12:24:58.851418 26194 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:24:58.853186 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181679us
I20260709 12:24:58.854007 26129 raft_consensus.cc:2368] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:24:58.758131 29399 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:58.858235 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 280512us
W20260709 12:24:58.845204 26756 connection.cc:256] RPC call timeout handler was delayed by -1.0346348670000225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.311s for a response
W20260709 12:24:58.888624 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44028.5us
I20260709 12:24:58.901235 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.902108 26791 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:24:58.903702 29228 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:58.904206 29228 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:58.905267 29228 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:58.906672 29228 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.909415 29517 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.898366 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:58.864468 29388 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:58.919080 26198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.018s
W20260709 12:24:58.919489 26198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.015s	sys 0.019s
I20260709 12:24:58.916884 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:58.923954 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 243829us
I20260709 12:24:58.864617 29509 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.914506 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.889950 29345 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:58.932608 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.895577 29469 consensus_queue.cc:1243] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:58.932096 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.934312 26783 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:58.945559 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.953714 26194 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:24:58.953116 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.959530 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:58.960253 26199 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:58.979547 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.981423 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:58.982143 26791 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 5.
I20260709 12:24:58.983103 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:58.981519 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:58.932296 29242 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.927778 29388 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:58.954468 29314 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:58.993175 26791 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:24:58.991575 29532 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:24:58.989312 28693 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:59.001291 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:59.004626 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 139329us
W20260709 12:24:59.010226 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77937.8us
I20260709 12:24:58.992739 29183 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:59.016834 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 113431us
I20260709 12:24:58.981714 29509 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.024s
W20260709 12:24:59.022871 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 177590us
I20260709 12:24:59.004626 29511 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:59.025409 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:59.029968 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:58.992475 29535 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:59.027514 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.026834 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.047299 26788 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:24:59.019644 29345 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.993423 29534 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:24:59.053483 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:59.051018 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.055660 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:59.057391 26785 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.047878 29455 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:59.058701 29511 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:59.075748 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.076170 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:59.076911 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.077458 29183 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.077580 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.077966 29183 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:24:59.078389 29183 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.078863 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.079635 26789 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:24:59.080343 29525 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.080842 29525 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.080602 29539 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:24:59.081246 29525 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.082015 29527 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.081943 29511 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.082753 29527 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:59.084580 29292 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:59.084978 29292 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:59.085892 29292 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.087318 29292 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.068372 29503 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:59.089046 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.090238 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.085675 29546 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:59.088877 29526 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:59.092725 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:59.093124 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:24:59.101078 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.091033 26789 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:24:59.102260 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:59.105908 29388 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.105592 26129 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:24:59.116761 26791 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:24:59.116319 28693 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:24:59.118388 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.122738 26784 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:24:59.125387 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.143724 26785 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
W20260709 12:24:59.121490 29531 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:24:59.097179 29546 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.096964 29526 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.157191 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:24:59.163805 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71381.8us
W20260709 12:24:59.172173 29529 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:24:59.172722 29535 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.120150 29504 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.178426 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1733) took 1048 ms. Trace:
I20260709 12:24:59.180307 29504 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:59.179104 26198 rpcz_store.cc:276] 0709 12:24:58.129475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.131860 (+  2385us) service_pool.cc:224] Handling call
0709 12:24:58.132301 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.132769 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.132801 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.133367 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.919834 (+786467us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:58.920163 (+   329us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.045661 (+125498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.045741 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.178072 (+132331us) raft_consensus.cc:1685] finished
0709 12:24:59.178100 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.178302 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1575,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":166198,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":34226,"thread_start_us":911180,"threads_started":2,"wal-append.queue_time_us":710208}
I20260709 12:24:59.182395 29551 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:59.183046 29551 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:59.184480 29539 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:59.186636 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:59.187003 29469 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:59.187674 29469 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:59.189729 29536 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.189067 29469 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.190459 29536 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:59.191125 29469 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.135074 29506 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:59.139863 29503 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:59.146188 29505 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:59.165506 29194 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.007s	sys 0.005s
I20260709 12:24:59.164898 29546 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:59.203697 26756 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 3.110s (SENT) [suppressed 30 similar messages]
I20260709 12:24:59.220769 26199 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:24:59.192596 29526 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.192600 29538 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 12:24:59.226050 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 91138.8us
I20260709 12:24:59.202553 29506 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.210254 29503 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:59.206552 29505 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:59.241962 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 139761us
I20260709 12:24:59.218291 29504 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:59.246917 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:59.221436 29539 raft_consensus.cc:2764] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:24:59.230192 29528 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 12:24:59.256745 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 245324us
I20260709 12:24:59.238626 29506 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:59.260751 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 365564us
I20260709 12:24:59.247098 29503 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:59.249490 29505 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.276746 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.230773 29534 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:24:59.288296 29523 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:59.289078 29523 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.289590 29523 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:59.289788 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:59.296878 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50747.1us
I20260709 12:24:59.204129 29551 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.292380 29519 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:24:59.173722 29535 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.303658 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:59.308871 29544 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 12:24:59.309176 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.191640 29536 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.313120 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.313175 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:24:59.315304 26000 auto_rebalancer.cc:893] Removing replace marker failed: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:24:59.325359 26000 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 69c5f2ff7a584f80b9a347f6adb3c2f8 already in progress
I20260709 12:24:59.298138 29272 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:59.316129 29561 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:59.334776 29561 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.335227 29561 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.336699 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1741) took 1085 ms. Trace:
I20260709 12:24:59.337112 26203 rpcz_store.cc:276] 0709 12:24:58.251139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.265845 (+ 14706us) service_pool.cc:224] Handling call
0709 12:24:58.266258 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:58.266848 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.266881 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:58.267498 (+   617us) log.cc:844] Serialized 52 byte log entry
0709 12:24:58.734948 (+467450us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:58.735013 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.735109 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.336415 (+601306us) raft_consensus.cc:1685] finished
0709 12:24:59.336437 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.336618 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":466999,"threads_started":1,"wal-append.queue_time_us":617571}
I20260709 12:24:59.341360 29399 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:59.341447 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62604.2us
I20260709 12:24:59.343362 29563 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:59.325726 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 69c5f2ff7a584f80b9a347f6adb3c2f8 already in progress
W20260709 12:24:59.326584 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 76458.2us
I20260709 12:24:59.296940 29543 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:24:59.319384 29551 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:59.366662 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65814.4us
I20260709 12:24:59.083900 29527 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:59.371397 29292 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:59.373521 29561 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:59.374751 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62506.5us
I20260709 12:24:59.375824 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.376667 29194 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:59.376852 26169 leader_election.cc:400] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:24:59.377239 26169 leader_election.cc:403] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:59.377135 29194 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:59.377710 29194 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.379753 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.380554 29557 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:59.381129 29539 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:59.381659 29539 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:59.382562 29539 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.383481 29509 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.384513 29563 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:59.385546 29563 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:59.385924 29563 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:24:59.386087 29194 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.386821 29563 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.337637 29323 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.394212 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:59.381281 29557 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:24:59.397112 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94364.3us
W20260709 12:24:59.397456 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 193792us
I20260709 12:24:59.333174 29537 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.333541 29272 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:24:59.402493 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.406909 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.414537 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.416388 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:59.364478 29547 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.417568 26125 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:24:59.418666 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 115148us
I20260709 12:24:59.375847 29561 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:59.424220 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44541.1us
I20260709 12:24:59.413569 29563 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.445011 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:24:59.447091 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.400830 29538 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:24:59.419874 29547 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.448561 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.427433 29534 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.426688 29504 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:59.460199 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 43957us
I20260709 12:24:59.445137 29356 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.442569 29194 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:59.468147 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.480348 29559 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.454430 29547 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.459259 29561 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:59.463251 29508 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:59.484570 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.484345 26126 pending_rounds.cc:85] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 2
I20260709 12:24:59.488893 26195 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:59.492892 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.493175 26126 pending_rounds.cc:107] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:24:59.493471 26126 raft_consensus.cc:2889] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:59.483157 29559 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.504994 29559 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.505240 29505 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.506219 29323 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:59.507055 29567 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:59.507622 29567 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:59.462823 29272 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.508917 29272 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:59.511070 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:59.511631 26788 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.487021 29561 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.512652 29561 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.513636 29509 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:24:59.473904 29523 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.500550 29568 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:59.486016 29526 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:24:59.504451 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78228.9us
I20260709 12:24:59.541980 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:24:59.485441 29557 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:59.519565 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58591.2us
I20260709 12:24:59.551283 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:59.489336 29565 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:24:59.469715 29292 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:59.563968 29505 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:24:59.567480 29228 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.660s	user 0.013s	sys 0.001s
W20260709 12:24:59.519472 29545 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:24:59.576452 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91954.8us
I20260709 12:24:59.577626 26782 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:24:59.557603 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:59.583657 26789 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:59.501061 29571 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:59.548926 29508 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:59.561007 29345 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:59.570653 29527 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:59.596844 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:24:59.590670 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:24:59.572935 29538 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:59.600162 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:59.602977 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.604806 26782 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.605069 26120 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:59.566903 29504 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:24:59.606894 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.608728 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:59.609833 26786 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.516142 29503 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.603408 29570 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.597697 29194 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.555817 29568 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.604274 29272 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:59.585857 29509 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:24:59.603721 29535 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:59.577090 29525 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.630463 29525 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:59.560474 29557 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.634128 29557 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:24:59.634428 26786 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:59.614168 29563 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:59.635865 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.617431 29570 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:59.618927 29543 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.625075 29469 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.620303 29571 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.676734 29571 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:24:59.677881 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 102568us
I20260709 12:24:59.626718 29527 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:24:59.676842 29572 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:24:59.687668 26120 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:24:59.651575 29563 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.650797 29505 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:59.690018 26782 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:24:59.692335 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91704us
I20260709 12:24:59.693723 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:59.681694 29183 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.002s
I20260709 12:24:59.625252 29376 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:24:59.707600 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:59.664844 29570 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.688433 26788 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
W20260709 12:24:59.708998 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 117695us
I20260709 12:24:59.660647 29345 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.704766 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:59.681270 29469 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:59.707055 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100248us
I20260709 12:24:59.771616 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:24:59.771907 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:24:59.740094 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.772923 26122 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:24:59.713886 29505 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.706020 29563 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.749140 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:59.705309 29323 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:24:59.752555 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:24:59.753831 29578 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:59.609268 29567 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.753589 29527 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.653522 29574 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:24:59.761940 29503 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:24:59.764875 29376 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:59.775547 29526 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:24:59.777719 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73034.8us
I20260709 12:24:59.822991 29323 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:59.823478 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:24:59.823709 29323 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:24:59.779716 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:24:59.824936 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45337us
I20260709 12:24:59.825798 26168 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7.
I20260709 12:24:59.826061 26168 leader_election.cc:403] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:59.826607 29588 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:59.827361 29588 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.778672 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:59.828953 29535 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:59.829149 29523 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:59.792412 29388 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:24:59.774605 29559 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.791294 29537 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.624574 29508 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.797099 29582 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.853842 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.793443 29194 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.742105 29183 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.797992 28693 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.798269 26788 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:24:59.780763 29469 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.863902 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:24:59.819989 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.794616 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.789464 29571 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:59.870678 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:59.830186 29376 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:24:59.797322 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.871124 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:59.852842 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:24:59.865624 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:59.874913 26125 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.864853 26198 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:24:59.872846 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.793733 29345 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:24:59.877349 26199 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:59.880523 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.875226 26195 raft_consensus.cc:2376] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6. Re-sending same reply.
I20260709 12:24:59.899765 29591 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:59.858649 29194 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:24:59.911001 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.859856 29537 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:24:59.922137 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 110775us
I20260709 12:24:59.853991 29523 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:24:59.856545 29551 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:24:59.924067 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.883127 29571 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:24:59.930164 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:24:59.877017 29505 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:24:59.870831 28693 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.882440 29388 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.859133 29582 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:00.109170 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:24:59.945688 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:00.122490 26120 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.956077 26125 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:00.112257 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:00.125070 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:00.125849 26125 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.932130 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:00.003641 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:24:59.968940 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 145532us
I20260709 12:24:59.998318 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:00.134732 29518 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:24:59.975423 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 143264us
I20260709 12:25:00.159507 26127 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:00.137624 29570 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:00.133076 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 202996us
I20260709 12:25:00.043378 29568 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:00.111027 29592 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:24:59.983310 29585 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:25:00.161082 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:00.172852 26196 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:25:00.003758 29579 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:25:00.044202 29228 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.178984 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:24:59.918431 29194 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.168087 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:24:59.918771 29591 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:00.092392 29546 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.000s
I20260709 12:25:00.128362 29551 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:00.193519 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:00.194247 26200 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:00.189275 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:00.041060 29242 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:00.001993 29584 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:00.177627 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:00.098572 29538 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:24:59.948119 29323 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:00.178399 29575 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.179391 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:24:59.922350 29504 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:00.181018 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:00.140709 29511 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:00.183393 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:00.187374 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:00.188597 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:00.114980 29537 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:24:59.966512 29376 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:01.160991 26094 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.005s	sys 0.016s
I20260709 12:25:01.281301 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:01.282133 26196 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:00.495329 26120 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:00.723757 26050 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:00.577756 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:00.309623 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.285807 26051 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:01.284081 26194 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.459152 26200 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:25:01.097788 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.004s	sys 0.018s
I20260709 12:25:00.290904 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.289199 26049 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.289870 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1373) took 1091 ms. Trace:
I20260709 12:25:01.290316 26049 rpcz_store.cc:276] 0709 12:25:00.198002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.198149 (+   147us) service_pool.cc:224] Handling call
0709 12:25:01.289784 (+1091635us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.212080 26125 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:00.224756 26054 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:01.281325 26094 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.008s	sys 0.017s
I20260709 12:25:00.210248 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:01.293635 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 858403us
I20260709 12:25:01.294028 26124 raft_consensus.cc:2376] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8. Re-sending same reply.
I20260709 12:25:01.283412 26199 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:25:01.291036 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2591) took 1899 ms. Trace:
I20260709 12:25:01.074508 26047 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:01.286877 26202 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:25:01.299010 26126 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.508342 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.299866 26053 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:01.079838 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.300643 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1372) took 1114 ms. Trace:
I20260709 12:25:01.304580 26198 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:01.306705 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1831) took 1422 ms (client timeout 1771 ms). Trace:
I20260709 12:25:00.197587 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:00.176579 29292 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:01.307451 26124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1822) took 1467 ms (client timeout 1874 ms). Trace:
I20260709 12:25:01.309518 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1746) took 1136 ms. Trace:
I20260709 12:25:01.307914 26127 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:01.310149 26198 rpcz_store.cc:276] 0709 12:25:00.175391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.182331 (+  6940us) service_pool.cc:224] Handling call
0709 12:25:01.305280 (+1122949us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.311396 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1747) took 1111 ms. Trace:
W20260709 12:25:01.311780 26120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1829) took 1443 ms (client timeout 1779 ms). Trace:
I20260709 12:25:01.301384 26052 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:01.307590 26125 rpcz_store.cc:269] 0709 12:24:59.869593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.124592 (+254999us) service_pool.cc:224] Handling call
0709 12:25:01.291857 (+1167265us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:01.308980 26124 rpcz_store.cc:269] 0709 12:24:59.826931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.166719 (+339788us) service_pool.cc:224] Handling call
0709 12:25:01.294762 (+1128043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.315243 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1745) took 1129 ms. Trace:
I20260709 12:25:01.315322 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1749) took 1139 ms. Trace:
I20260709 12:25:01.314592 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1374) took 1150 ms. Trace:
I20260709 12:25:01.316357 26197 rpcz_store.cc:276] 0709 12:25:00.176388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.176975 (+   587us) service_pool.cc:224] Handling call
0709 12:25:01.315206 (+1138231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.312040 26200 rpcz_store.cc:276] 0709 12:25:00.175761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.175925 (+   164us) service_pool.cc:224] Handling call
0709 12:25:01.287227 (+1111302us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:01.312438 26120 rpcz_store.cc:269] 0709 12:24:59.857340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.108882 (+251542us) service_pool.cc:224] Handling call
0709 12:25:01.300792 (+1191910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.306708 29538 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.320025 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.320760 26046 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:01.321436 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 487) took 1799 ms. Trace:
I20260709 12:25:01.316051 26202 rpcz_store.cc:276] 0709 12:25:00.168718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.168817 (+    99us) service_pool.cc:224] Handling call
0709 12:25:01.297792 (+1128975us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.474823 29561 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.950s	user 0.009s	sys 0.003s
I20260709 12:25:01.317232 26199 rpcz_store.cc:276] 0709 12:25:00.164179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.164454 (+   275us) service_pool.cc:224] Handling call
0709 12:25:01.314526 (+1150072us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.440894 29600 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:00.463287 29563 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.001s
W20260709 12:25:00.290819 29506 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.998s	user 0.011s	sys 0.001s
I20260709 12:25:01.348093 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:01.304513 26054 rpcz_store.cc:276] 0709 12:25:00.186273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.186474 (+   201us) service_pool.cc:224] Handling call
0709 12:25:00.224566 (+ 38092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.300390 (+1075824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.300584 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.323560 29534 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.860s	user 0.002s	sys 0.008s
W20260709 12:25:00.521155 29547 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.026s	user 0.012s	sys 0.000s
W20260709 12:25:01.063155 29469 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.180s	user 0.013s	sys 0.000s
I20260709 12:25:01.297168 26129 rpcz_store.cc:276] 0709 12:24:59.391697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.391891 (+   194us) service_pool.cc:224] Handling call
0709 12:24:59.404803 (+ 12912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.445553 (+ 40750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.445586 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.460158 (+ 14572us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.613860 (+153702us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:59.615070 (+  1210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.615156 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.209852 (+594696us) raft_consensus.cc:1685] finished
0709 12:25:00.209870 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.290959 (+1081089us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5652,"raft.queue_time_us":24861,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8672,"spinlock_wait_cycles":2432,"thread_start_us":127901,"threads_started":1,"wal-append.queue_time_us":181517,"wal-append.run_cpu_time_us":15286,"wal-append.run_wall_time_us":1561702}
I20260709 12:25:01.322036 26128 rpcz_store.cc:276] 0709 12:24:59.521645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.548609 (+ 26964us) service_pool.cc:224] Handling call
0709 12:24:59.571379 (+ 22770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.585236 (+ 13857us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:59.601931 (+ 16695us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.708029 (+106098us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.738860 (+ 30831us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.739415 (+   555us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:59.739454 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.739520 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.321228 (+1581708us) raft_consensus.cc:1685] finished
0709 12:25:01.321253 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.321374 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11801,"prepare.queue_time_us":95501,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":5868,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":61758,"thread_start_us":84274,"threads_started":1,"wal-append.queue_time_us":227203}
I20260709 12:25:01.355154 26126 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:01.355270 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:00.180605 29559 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:00.208166 29570 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.016552 29557 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.224390 29595 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.333432 29242 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:01.335469 29194 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.136s	user 0.011s	sys 0.000s
I20260709 12:25:01.336918 29582 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.338380 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.338824 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.323643 29292 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.340963 26124 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:25:01.372680 26124 pending_rounds.cc:85] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 4
I20260709 12:25:01.373028 26124 pending_rounds.cc:107] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:25:01.373328 26124 raft_consensus.cc:2889] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:01.374217 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1824) took 1546 ms. Trace:
I20260709 12:25:01.374622 26124 rpcz_store.cc:276] 0709 12:24:59.827506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.325973 (+1498467us) service_pool.cc:224] Handling call
0709 12:25:01.340642 (+ 14669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.373882 (+ 33240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.374141 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.340749 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.341568 26196 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:25:01.375851 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1751) took 1197 ms. Trace:
I20260709 12:25:00.330061 29345 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:01.376190 26196 rpcz_store.cc:276] 0709 12:25:00.178909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.181715 (+  2806us) service_pool.cc:224] Handling call
0709 12:25:01.375797 (+1194082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.421186 29455 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.211298 29183 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:00.405666 29399 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.354969 29547 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.341715 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:00.314394 29588 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7.
I20260709 12:25:00.391608 29517 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.223260 29606 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.223855 29535 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.344552 26194 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:00.222996 29583 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:00.329704 28693 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:00.471607 29604 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.350972 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:01.351791 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.353101 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1753) took 1149 ms. Trace:
I20260709 12:25:01.395512 26199 rpcz_store.cc:276] 0709 12:25:00.199961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.347711 (+1147750us) service_pool.cc:224] Handling call
0709 12:25:01.348969 (+  1258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.189872 29228 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.021766 29596 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.355576 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:01.355769 26126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1830) took 1494 ms (client timeout 1915 ms). Trace:
I20260709 12:25:00.199702 29546 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:01.351490 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.009s	sys 0.019s
I20260709 12:25:01.356036 26052 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.362461 26127 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:25:01.421499 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.009s	sys 0.019s
I20260709 12:25:01.421865 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 496) took 1633 ms. Trace:
W20260709 12:25:01.421960 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 959349us
I20260709 12:25:01.296087 29388 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.422335 26127 rpcz_store.cc:276] 0709 12:24:59.788728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.174953 (+386225us) service_pool.cc:224] Handling call
0709 12:25:01.421793 (+1246840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.424369 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.391779 29613 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.425253 26052 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.366385 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:01.322769 29504 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.367681 26053 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.368043 26051 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:25:01.385263 29455 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.332456 29561 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:00.451272 29571 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.396350 29583 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.309365 29511 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.368382 29194 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.392841 29606 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.398376 29604 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.382050 29595 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:00.333518 29591 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.383844 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.384660 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:00.210512 29568 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.349963 29563 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.367633 29559 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.350768 29506 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.331888 29575 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.357501 29469 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.347314 29600 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.471431 26048 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:25:01.360559 29557 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.404786 29596 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:01.404510 26126 rpcz_store.cc:269] 0709 12:24:59.861718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.168949 (+307231us) service_pool.cc:224] Handling call
0709 12:25:01.355709 (+1186760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.406011 29616 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.397023 29535 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:01.423022 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:01.532182 26199 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.426566 26054 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.357534 29534 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:01.429699 26123 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:01.428511 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:01.388636 29399 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.434914 26201 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:01.425661 29582 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.448437 26045 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.438743 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.451475 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.446621 29595 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:01.391855 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1748) took 1215 ms. Trace:
I20260709 12:25:01.438261 29614 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:01.426311 29588 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:01.444856 29574 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.433415 29292 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.450994 29559 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.542251 26196 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:01.444823 29194 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:01.542223 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113752us
I20260709 12:25:01.443230 29583 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.445148 29606 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.451867 29547 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.434026 29242 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:01.439383 29612 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.545831 29615 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.547362 26197 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:01.548069 26203 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:01.390467 29517 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.446421 29604 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.448079 29561 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.547961 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:01.470769 29575 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:01.479584 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 95024.4us
I20260709 12:25:01.457620 29511 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:01.448827 29568 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.553211 26194 rpcz_store.cc:276] 0709 12:25:00.176072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.176180 (+   108us) service_pool.cc:224] Handling call
0709 12:25:01.391777 (+1215597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.431344 29613 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.450687 29563 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.578862 29574 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.548269 29506 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.541603 29616 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.806759 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1750) took 1628 ms. Trace:
I20260709 12:25:01.586819 26054 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:01.807256 26203 rpcz_store.cc:276] 0709 12:25:00.178592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.451182 (+1272590us) service_pool.cc:224] Handling call
0709 12:25:01.806699 (+355517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.791800 26199 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:01.548786 29527 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:01.808835 26199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1385) took 1296 ms (client timeout 1693 ms). Trace:
I20260709 12:25:01.807904 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:01.812090 26199 rpcz_store.cc:269] 0709 12:25:00.512478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.396916 (+884438us) service_pool.cc:224] Handling call
0709 12:25:01.808789 (+411873us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.809401 29631 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.812397 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:01.813509 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.581962 29292 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.815979 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:01.821565 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:01.824254 26054 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:01.560633 29534 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:01.827256 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.816474 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:01.828066 26053 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.821002 26199 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:01.831251 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1379) took 1321 ms. Trace:
I20260709 12:25:01.832549 26199 rpcz_store.cc:276] 0709 12:25:00.509870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.813293 (+1303423us) service_pool.cc:224] Handling call
0709 12:25:01.831166 (+ 17873us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.841136 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:01.841862 26053 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:01.847496 26046 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:01.848505 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.582154 29535 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.557569 29582 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.818015 29604 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.567584 29614 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.865233 29604 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.868589 29557 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.869081 29568 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:01.633800 29588 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.823686 29612 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.847324 29527 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:01.871727 29588 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.872546 29600 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.872689 29612 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.859903 29629 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:01.854063 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1378) took 1344 ms. Trace:
I20260709 12:25:01.823057 29583 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.689306 29228 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.583097 29504 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.815192 29388 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:01.847527 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.692s (SENT)
I20260709 12:25:01.857353 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:01.861060 26053 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:01.854460 29563 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.874169 26199 rpcz_store.cc:276] 0709 12:25:00.509384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.847563 (+1338179us) service_pool.cc:224] Handling call
0709 12:25:01.853993 (+  6430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.895741 26194 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:01.898211 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:01.898969 26199 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.826284 29575 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.872571 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:01.902041 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1394) took 1386 ms. Trace:
I20260709 12:25:01.902467 26203 rpcz_store.cc:276] 0709 12:25:00.515636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.846844 (+1331208us) service_pool.cc:224] Handling call
0709 12:25:01.901975 (+ 55131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.846920 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.823199 29511 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.878580 26054 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:01.829200 29631 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.900068 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.898237 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1388) took 1384 ms. Trace:
W20260709 12:25:01.903570 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46308.7us
I20260709 12:25:01.903962 26052 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.905715 26053 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.904865 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:01.905989 26194 rpcz_store.cc:276] 0709 12:25:00.513585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.804981 (+1291396us) service_pool.cc:224] Handling call
0709 12:25:01.898174 (+ 93193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.907541 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:01.832077 29292 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.906399 26050 raft_consensus.cc:2376] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6. Re-sending same reply.
I20260709 12:25:01.909301 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.910239 26054 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.934726 26199 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:01.935305 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1387) took 1422 ms. Trace:
I20260709 12:25:01.935604 26199 rpcz_store.cc:276] 0709 12:25:00.513203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.897823 (+1384620us) service_pool.cc:224] Handling call
0709 12:25:01.935247 (+ 37424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.899952 29469 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.911252 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.944005 26203 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:25:01.944582 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1386) took 1431 ms. Trace:
I20260709 12:25:01.944756 29506 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.944952 26203 rpcz_store.cc:276] 0709 12:25:00.512833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.903253 (+1390420us) service_pool.cc:224] Handling call
0709 12:25:01.944526 (+ 41273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:01.932497 29621 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:25:01.945317 29506 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.945981 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.907039 29608 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:01.946801 29626 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.947791 29506 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.872098 29527 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.946656 26203 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:01.936650 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.937282 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.953655 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:01.937022 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:01.902246 29388 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.940025 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.936831 29612 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.899344 29625 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:01.941254 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 114469us
I20260709 12:25:01.820286 29615 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.873795 29629 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:01.910928 29574 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.953467 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 530) took 2100 ms. Trace:
I20260709 12:25:01.985674 26201 rpcz_store.cc:276] 0709 12:24:59.853410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.853571 (+   161us) service_pool.cc:224] Handling call
0709 12:25:01.953162 (+2099591us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:01.953360 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3587149568}
I20260709 12:25:01.987960 26201 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:25:01.988443 26201 pending_rounds.cc:85] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 7
I20260709 12:25:01.988687 26201 pending_rounds.cc:107] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:25:01.944613 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:01.947176 29631 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:01.989771 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45251.9us
I20260709 12:25:01.963884 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1752) took 1784 ms. Trace:
I20260709 12:25:01.963828 26053 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:01.939318 29559 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.964291 26199 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:01.995911 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1381) took 1485 ms. Trace:
I20260709 12:25:01.996399 26199 rpcz_store.cc:276] 0709 12:25:00.510927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.936170 (+1425243us) service_pool.cc:224] Handling call
0709 12:25:01.995849 (+ 59679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.901335 29591 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:01.947943 29608 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:01.930779 29399 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.887506 29583 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.975874 29469 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:01.983078 26054 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:01.984549 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:01.989850 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1382) took 1478 ms (client timeout 1660 ms). Trace:
I20260709 12:25:01.936597 29634 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.948403 29606 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:01.991019 26203 rpcz_store.cc:276] 0709 12:25:00.179191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.945716 (+1766525us) service_pool.cc:224] Handling call
0709 12:25:01.963819 (+ 18103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.994915 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:01.948467 29603 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.988948 26201 raft_consensus.cc:2889] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:01.989760 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:02.009394 26049 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:01.905148 29563 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.000109 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 872) took 2217 ms. Trace:
I20260709 12:25:01.979040 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.011613 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:02.011776 26786 rpcz_store.cc:276] 0709 12:24:59.778754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.778868 (+   114us) service_pool.cc:224] Handling call
0709 12:24:59.796965 (+ 18097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.812331 (+ 15366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.812363 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.812814 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.151015 (+338201us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:00.151079 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.151174 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.988562 (+1837388us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b5400116d20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431adbe 00007f5b74325930 00007f5b743258d8 00007f5b74325858 00007f5b74325813 00007f5b74324a9c 00007f5b7a2102f6 00007f5b7a20aaa8 00007f5b743617cb 00007f5b743642c0
0709 12:25:01.990698 (+  2136us) raft_consensus.cc:1685] finished
0709 12:25:01.990718 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.996100 (+  5382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24056,"raft.queue_time_us":38902,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":23729,"spinlock_wait_cycles":40254208,"thread_start_us":250112,"threads_started":1,"wal-append.queue_time_us":277010}
I20260709 12:25:01.997000 29506 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.908865 29511 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:02.004897 26196 rpcz_store.cc:269] 0709 12:25:00.511347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.383548 (+872201us) service_pool.cc:224] Handling call
0709 12:25:01.989628 (+606080us) spinlock_profiling.cc:243] Waited 95 ms on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:01.989766 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218571648}
I20260709 12:25:01.907042 29575 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:01.990499 29625 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.006634 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 871) took 2228 ms. Trace:
I20260709 12:25:02.007732 29583 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:01.994522 29228 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.022650 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1376) took 1602 ms. Trace:
I20260709 12:25:02.024168 26782 rpcz_store.cc:276] 0709 12:24:59.778472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.779424 (+   952us) service_pool.cc:224] Handling call
0709 12:24:59.779775 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:59.834128 (+ 54353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.834159 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:59.849297 (+ 15138us) log.cc:844] Serialized 52 byte log entry
0709 12:24:59.907112 (+ 57815us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:59.912227 (+  5115us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.919409 (+  7182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.919487 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.950928 (+2031441us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b18002e91e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:02.006355 (+ 55427us) raft_consensus.cc:1685] finished
0709 12:25:02.006450 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.006576 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86361,"prepare.queue_time_us":159,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":63498,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":35721,"spinlock_wait_cycles":8140032,"thread_start_us":57336,"threads_started":1,"wal-append.queue_time_us":120713}
I20260709 12:25:02.028443 26201 rpcz_store.cc:276] 0709 12:25:00.420854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.987054 (+1566200us) service_pool.cc:224] Handling call
0709 12:25:01.987489 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.022401 (+ 34912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.022584 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.022724 26203 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:02.025061 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.013967 29606 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.001650 29596 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:01.982677 29612 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.029983 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1377) took 1526 ms. Trace:
I20260709 12:25:01.947954 29626 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.036135 26203 rpcz_store.cc:276] 0709 12:25:00.503617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.009199 (+1505582us) service_pool.cc:224] Handling call
0709 12:25:02.022430 (+ 13231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.029702 (+  7272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.029929 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.022248 29571 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:02.039311 29228 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.049685 29228 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.050364 29228 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:02.033193 29517 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.011366 29469 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:02.105266 29633 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:02.096616 29616 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.014094 29634 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.098667 29623 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.109424 29623 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.110386 29615 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.080518 29608 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.112125 29623 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.115691 29612 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.117190 29629 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.167169 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1383) took 1655 ms. Trace:
I20260709 12:25:02.167862 26201 rpcz_store.cc:276] 0709 12:25:00.511727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.040689 (+1528962us) service_pool.cc:224] Handling call
0709 12:25:02.052160 (+ 11471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.052558 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.052589 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.153932 (+101343us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.166663 (+ 12731us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:02.166711 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.166806 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.166864 (+    58us) raft_consensus.cc:1685] finished
0709 12:25:02.166882 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.167045 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3763,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":9825,"thread_start_us":12209,"threads_started":1,"wal-append.queue_time_us":10219}
I20260709 12:25:02.170786 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1384) took 1658 ms. Trace:
I20260709 12:25:02.171267 26201 rpcz_store.cc:276] 0709 12:25:00.512091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.169806 (+1657715us) service_pool.cc:224] Handling call
0709 12:25:02.170211 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.170417 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.170447 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.170501 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.170542 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.170599 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.170729 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.177134 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1389) took 1663 ms. Trace:
I20260709 12:25:02.177632 26201 rpcz_store.cc:276] 0709 12:25:00.513982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.173189 (+1659207us) service_pool.cc:224] Handling call
0709 12:25:02.173523 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.173783 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:02.175059 (+  1276us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.176710 (+  1651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.176764 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:02.176809 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.176888 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.177035 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1347,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1437,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":1634,"threads_started":2,"wal-append.queue_time_us":1559}
I20260709 12:25:02.180126 26201 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:02.180903 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1390) took 1666 ms. Trace:
I20260709 12:25:02.181558 26201 rpcz_store.cc:276] 0709 12:25:00.514314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.179538 (+1665224us) service_pool.cc:224] Handling call
0709 12:25:02.179891 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.180663 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.180847 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.186204 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1391) took 1671 ms. Trace:
I20260709 12:25:02.186802 26201 rpcz_store.cc:276] 0709 12:25:00.514681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.182393 (+1667712us) service_pool.cc:224] Handling call
0709 12:25:02.183472 (+  1079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.183747 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:02.184098 (+   351us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.185759 (+  1661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.185819 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:02.185876 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.185978 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.186136 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":337,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":31,"raft.queue_time_us":1501,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":1183,"threads_started":1,"wal-append.queue_time_us":1487}
I20260709 12:25:02.193701 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1380) took 1683 ms. Trace:
I20260709 12:25:02.193966 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1392) took 1679 ms. Trace:
I20260709 12:25:02.194159 26196 rpcz_store.cc:276] 0709 12:25:00.510458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.026617 (+1516159us) service_pool.cc:224] Handling call
0709 12:25:02.026988 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.027616 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.027649 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.039577 (+ 11928us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.193324 (+153747us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:02.193366 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.193431 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.193474 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:02.193484 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.193644 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2390,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":3848,"thread_start_us":141613,"threads_started":1,"wal-append.queue_time_us":88619}
I20260709 12:25:02.196686 26196 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:02.197422 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1393) took 1682 ms. Trace:
I20260709 12:25:02.197779 26196 rpcz_store.cc:276] 0709 12:25:00.515298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.196037 (+1680739us) service_pool.cc:224] Handling call
0709 12:25:02.196480 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.197156 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.197364 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.199370 26196 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:02.199956 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1395) took 1684 ms. Trace:
I20260709 12:25:02.200330 26196 rpcz_store.cc:276] 0709 12:25:00.516006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.198615 (+1682609us) service_pool.cc:224] Handling call
0709 12:25:02.198987 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.199711 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.199901 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.194525 26201 rpcz_store.cc:276] 0709 12:25:00.514969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.192984 (+1678015us) service_pool.cc:224] Handling call
0709 12:25:02.193386 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.193595 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.193629 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.193679 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:02.193719 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.193779 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.193906 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.307365 29469 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.361325 29571 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:02.361969 29571 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.370307 29538 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.371722 29571 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.378855 29571 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.395376 29292 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:02.411535 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1744) took 2243 ms (client timeout 1626 ms). Trace:
W20260709 12:25:02.430143 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1375) took 2219 ms (client timeout 1559 ms). Trace:
W20260709 12:25:02.415560 26195 rpcz_store.cc:269] 0709 12:25:00.168407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.168738 (+   331us) service_pool.cc:224] Handling call
0709 12:25:02.411252 (+2242514us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400397954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:02.411412 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799199616}
W20260709 12:25:02.430696 26197 rpcz_store.cc:269] 0709 12:25:00.211071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.346125 (+1135054us) service_pool.cc:224] Handling call
0709 12:25:02.429918 (+1083793us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400399c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:02.430053 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321942272}
I20260709 12:25:02.021386 29603 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.446348 29603 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.456727 29614 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.464416 29603 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.465165 29603 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.467597 29603 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.475916 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1823) took 2648 ms. Trace:
I20260709 12:25:02.482649 26121 rpcz_store.cc:276] 0709 12:24:59.827216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.197196 (+369980us) service_pool.cc:224] Handling call
0709 12:25:00.197562 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.390189 (+1192627us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:01.802506 (+412317us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.901137 (+ 98631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:01.901213 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:01.901365 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.475514 (+574149us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:02.475586 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.475799 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":426471,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":14963,"raft.queue_time_us":122,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":94,"spinlock_wait_cycles":2727319936,"thread_start_us":435876,"threads_started":2,"wal-append.queue_time_us":123102}
I20260709 12:25:02.492086 29642 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.492743 29642 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.493629 29228 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:02.494252 29228 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.494992 29642 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.495556 29642 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:02.496040 29642 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.497269 29228 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.498320 29642 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.509363 29455 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.532532 29642 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:02.534950 29506 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:02.537343 26019 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.011s
W20260709 12:25:02.538316 26019 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.001s	sys 0.012s
I20260709 12:25:02.538079 29642 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:02.538669 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 605246us
I20260709 12:25:02.540380 29642 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.554929 29228 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.555815 29228 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.556855 29623 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.557657 29623 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.559345 29228 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.561451 29642 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:02.562134 29642 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.562628 29228 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.563372 29228 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.564674 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.565655 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.569355 29642 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.570083 29642 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.569408 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.572448 29623 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.570920 29642 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:02.572407 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1828) took 2734 ms (client timeout 1689 ms). Trace:
W20260709 12:25:02.580302 26123 rpcz_store.cc:269] 0709 12:24:59.838067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.909280 (+ 71213us) service_pool.cc:224] Handling call
0709 12:25:02.572161 (+2662881us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400287654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:02.572290 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2995244288}
I20260709 12:25:02.572341 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.586680 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.588613 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.589481 26127 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:02.590642 29604 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:02.589520 26121 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:02.590065 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.592461 26124 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:02.590380 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.593415 29642 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.593752 26129 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 6: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:02.596876 26121 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:02.601738 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.602869 29606 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.605114 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.606168 26123 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:02.613575 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.594637 29228 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.616274 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.617192 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.619263 26121 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:02.620214 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.617486 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.621893 26782 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:02.622298 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.623202 26121 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:02.623761 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.624439 26782 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:02.625047 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.626046 26785 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:02.623792 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1936572460003845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 12:25:02.627420 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.629251 26052 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:02.621016 29571 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.630736 29571 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:02.631557 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 641861us
I20260709 12:25:02.632633 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.633037 29571 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.628787 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.633980 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.635398 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.636163 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:02.636418 26791 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:25:02.634502 26121 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.633862 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.638036 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.638362 26788 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:02.638160 29642 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:02.638885 29642 consensus_queue.cc:1243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:02.639274 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.639688 26053 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:02.640398 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.641263 29228 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:02.641921 26050 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:02.642141 29228 consensus_queue.cc:1243] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:02.620950 29610 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.642804 29623 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:02.625345 29603 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.643198 29623 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:02.642908 29228 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:02.636518 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:02.647428 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.009s	sys 0.000s
W20260709 12:25:02.651090 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.011s	sys 0.000s
I20260709 12:25:02.643272 29610 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.647408 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.647176 29623 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.643838 29603 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.663941 26053 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:02.657912 29568 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.645486 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.665711 29623 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.665805 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.666694 26787 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:02.669399 29642 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:02.670408 29642 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.670845 29642 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.648813 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.680881 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:02.660475 26791 pending_rounds.cc:85] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
I20260709 12:25:02.681734 26791 pending_rounds.cc:107] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:25:02.682106 26791 raft_consensus.cc:2889] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:02.652667 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1395) took 1263 ms. Trace:
I20260709 12:25:02.662925 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.665140 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.666010 26786 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:25:02.653244 26123 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:02.685250 26129 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:02.681102 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:02.684816 26051 rpcz_store.cc:276] 0709 12:25:01.389145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.429782 (+ 40637us) service_pool.cc:224] Handling call
0709 12:25:01.458596 (+ 28814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.458809 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:02.003415 (+544606us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.651665 (+648250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.651761 (+    96us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:02.651846 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.652106 (+   260us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.652579 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":384012,"prepare.queue_time_us":493864,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":2475,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8351,"thread_start_us":751219,"threads_started":2,"wal-append.queue_time_us":143433}
I20260709 12:25:02.687201 26050 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.688072 26049 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:02.689108 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.689710 26049 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:02.661170 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:02.685084 26047 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:02.693077 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:02.655280 26121 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:02.694473 26049 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.696579 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:02.696424 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.696353 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:02.698458 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.698791 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.699752 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.700431 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.699378 26124 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:02.697516 29455 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:02.707329 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.702808 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:02.701426 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:02.708901 26047 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.710467 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.713220 29455 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:02.713729 29455 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.714120 29455 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.715534 29517 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:02.719353 29228 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:02.713920 29603 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.720280 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.711020 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:02.723937 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.724889 26129 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:02.729022 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.730893 26046 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:02.737058 26047 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:02.716012 29517 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:02.747185 26049 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:02.749547 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:02.749979 29536 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:02.772150 29662 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:02.774250 29536 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:02.773787 29228 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:02.774881 29563 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.774487 29662 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.776193 29662 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.769496 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.779609 26049 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:25:02.784559 29661 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:02.782501 29575 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:02.742983 29455 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.799928 29228 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.801205 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.808701 26122 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:02.808444 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.743228 29642 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:02.753700 29517 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:02.809314 26049 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
W20260709 12:25:02.811640 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 115169us
I20260709 12:25:02.815354 26121 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7.
I20260709 12:25:02.838076 29642 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.869753 29536 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:02.873221 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:02.873947 26052 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:02.878058 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:02.878420 26050 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:02.878732 26791 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:02.882606 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128619us
I20260709 12:25:02.882697 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:02.888942 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.889361 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.889595 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.890098 26125 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:02.889858 26127 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:02.890395 26120 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:02.891952 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.892899 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.839628 29662 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:02.894791 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.894878 26120 raft_consensus.cc:2376] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7. Re-sending same reply.
I20260709 12:25:02.895481 26122 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:02.895840 26020 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:02.893684 26127 raft_consensus.cc:2376] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8. Re-sending same reply.
I20260709 12:25:02.897660 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.897913 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:02.899094 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.898941 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:02.879482 29455 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:02.900821 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.901226 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.906646 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.902309 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:02.904620 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.908190 26786 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.906744 26127 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:02.907426 26125 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:02.905634 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.902408 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:02.909911 26120 raft_consensus.cc:2376] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8. Re-sending same reply.
I20260709 12:25:02.913614 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:02.914726 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.917398 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.917635 26129 raft_consensus.cc:2376] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7. Re-sending same reply.
I20260709 12:25:02.918114 26122 raft_consensus.cc:2376] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9. Re-sending same reply.
I20260709 12:25:02.920562 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:02.920720 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.917323 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:02.922590 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:02.925688 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 49932.5us
I20260709 12:25:02.925931 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:02.887387 29527 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.927722 26127 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:02.929050 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:02.894130 29623 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:25:02.929950 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:02.930787 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:02.931348 26125 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:02.903247 29505 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:02.932922 26128 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:02.919881 29561 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.931501 29606 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:03.021214 29606 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:02.929765 29603 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:03.021724 29506 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:03.021999 29603 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:03.022428 29604 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.022927 29603 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.914830 29614 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:25:03.024206 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:03.024211 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 127357us
I20260709 12:25:02.930812 29292 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:03.025797 29667 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:02.931828 29527 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.026734 29527 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.021803 29606 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.989809 29662 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.030727 29658 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:03.032500 29662 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:03.034166 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.035748 26198 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 5.
I20260709 12:25:03.036427 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.039436 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.026108 29292 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.040272 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.041352 26197 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:03.040984 29292 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.015827 29517 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.040360 26198 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.043053 26124 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:03.043725 26786 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.039450 29614 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.046667 29614 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.047722 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.048293 26129 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:03.048569 26197 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:03.048904 29614 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.049958 29657 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.050508 29657 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.051199 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.052415 26197 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:03.053246 29657 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.055439 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.041471 29667 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:03.045351 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.056270 26197 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.056576 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.058883 26195 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:03.059270 29603 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.056429 29667 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:03.060456 29561 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.045857 26791 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:03.060878 29561 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.061237 29667 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.062150 29527 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.062762 26195 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:02.887497 29563 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:25:03.064541 29669 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:03.063074 29667 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:03.065089 29669 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:03.065567 29669 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.066349 29503 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:03.068858 29503 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:03.070989 29561 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.072322 29506 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.073460 29506 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.074198 29561 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.022905 29604 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.076074 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:03.079584 29659 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.103s	user 0.004s	sys 0.006s
W20260709 12:25:03.080317 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7261073610002313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.268s for a response
W20260709 12:25:03.080724 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6766073610001513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2735s for a response
W20260709 12:25:03.081018 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6478073610001047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2767s for a response
I20260709 12:25:03.073797 29603 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.086390 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.076730 26128 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:03.045135 29517 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.088079 26126 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:25:03.090693 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.104717 26052 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.091837 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.106186 26049 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:03.107532 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:03.092621 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9160 after lost signal to thread 24865
I20260709 12:25:03.108964 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.093505 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:03.109992 26199 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.110138 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.095091 29568 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:03.096814 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.117179 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.118007 26047 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:03.101922 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.108965 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.093197 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.112452 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.122110 26049 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:25:03.124552 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.125240 26198 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:03.125315 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:03.127529 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.127955 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.128841 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.135880 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.146755 26199 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.101778 29604 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.149561 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.154233 26047 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.157202 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.158493 26050 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:03.123832 29568 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.162926 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.164863 26053 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:03.165324 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 83899.5us
I20260709 12:25:03.114048 29669 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.152331 29603 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:25:03.148839 29606 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.185092 29606 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.186092 26054 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:03.173656 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5236073609999039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2905s for a response
I20260709 12:25:03.177388 26047 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:03.179689 29613 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.156059 29561 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:03.166483 29527 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.224898 26120 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:03.199827 26052 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:03.230443 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.213057 26053 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.219300 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.213274 26202 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:03.225255 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.220556 26054 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:03.246130 26121 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:03.246964 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:03.235087 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1371) took 3060 ms (client timeout 2905 ms). Trace:
W20260709 12:25:03.247979 26048 rpcz_store.cc:269] 0709 12:25:00.174532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.174688 (+   156us) service_pool.cc:224] Handling call
0709 12:25:03.234835 (+3060147us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:03.235013 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901495552}
I20260709 12:25:03.220952 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:03.249413 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.244565 26201 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:25:03.251161 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1933073609998246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3272s for a response
I20260709 12:25:03.219801 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.242075 29568 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.262746 26127 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:03.263337 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.264403 29568 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.264549 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.265507 26202 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:03.265343 29661 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:03.266541 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:03.266937 29661 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:03.222666 29604 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.267366 29661 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:03.268440 26200 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:03.269162 29634 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.269603 29604 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.269977 29661 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.271338 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.263875 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.272464 29665 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.272531 26127 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:03.273058 29665 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:03.273237 29626 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.273536 29665 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.273998 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.275123 26127 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:03.276183 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.277856 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.278596 26053 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:03.278972 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.278539 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.279693 26050 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:25:03.277863 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.280467 29582 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.280720 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.281622 26046 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:03.281786 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:03.282207 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.206482 26045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1374) took 2908 ms (client timeout 2320 ms). Trace:
I20260709 12:25:03.282532 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.395599 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.397251 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.397706 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.398241 26127 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.395628 26054 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:03.399408 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:03.278642 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0583328600000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 12:25:03.303347 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51858us
W20260709 12:25:03.400434 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0553232870001921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22930000000000003s for a response
W20260709 12:25:03.401149 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0364232870001615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23140000000000002s for a response
I20260709 12:25:03.402385 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.283144 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:03.395686 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98993.9us
W20260709 12:25:03.403244 26045 rpcz_store.cc:269] 0709 12:25:00.298242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.309236 (+ 10994us) service_pool.cc:224] Handling call
0709 12:25:03.206120 (+2896884us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54002d0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:03.206326 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373289728}
I20260709 12:25:03.407172 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.409997 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.410631 26054 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:03.410928 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.412360 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:03.387303 29553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.161s	user 0.009s	sys 0.038s
I20260709 12:25:03.233701 29388 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.415326 26051 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:03.419611 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.415548 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.415828 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.416070 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.421027 26049 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:03.415035 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.281632 29511 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.201606 29669 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.413378 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.417765 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.213438 29546 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.432137 29546 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:03.433030 29680 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:03.361349 29625 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.433517 29680 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.433919 29680 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.434928 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.436429 29614 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:03.438032 29614 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:03.435833 29292 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:25:03.439352 29292 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:03.439779 29292 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.440662 29292 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.439018 29661 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.420941 26054 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7. Re-sending same reply.
I20260709 12:25:03.442016 29292 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.417743 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.422706 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.444069 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.444859 29623 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:03.445271 29623 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.445649 29623 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:03.417588 29553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.194s	user 0.011s	sys 0.038s
I20260709 12:25:03.409090 29305 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:03.446369 29305 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:03.448457 29675 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:25:03.425148 29662 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:03.409381 29517 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:03.451797 29668 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.407s	user 0.004s	sys 0.000s
I20260709 12:25:03.420305 29624 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:03.414815 29598 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.453804 29598 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.457628 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.458482 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.458881 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.459758 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.460975 29455 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:03.461192 26129 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:03.461580 29455 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:03.462128 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.463160 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.462803 29455 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.463876 26129 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:03.465215 29517 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:03.465770 29517 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:03.465809 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.444303 29661 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.466867 26120 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:25:03.466782 29517 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.467839 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.476770 29680 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.477972 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:03.428908 26756 connection.cc:256] RPC call timeout handler was delayed by -2.8320586920003734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:25:03.478641 26756 connection.cc:256] RPC call timeout handler was delayed by -2.8320586920003734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:25:03.479012 26756 connection.cc:256] RPC call timeout handler was delayed by -2.8185586920003516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 12:25:03.422370 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.429564 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:03.479346 26756 connection.cc:256] RPC call timeout handler was delayed by -2.7420586920002279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:25:03.479882 26756 connection.cc:256] RPC call timeout handler was delayed by -2.7060586920001697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 12:25:03.480115 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49641.3us
I20260709 12:25:03.412703 29613 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.480641 29657 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.481182 29582 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.480644 26045 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:03.481292 29657 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.417212 29561 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.481786 29582 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.416622 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:03.484149 26756 connection.cc:256] RPC call timeout handler was delayed by -2.0991938180000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 12:25:03.484357 26121 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:03.484622 26756 connection.cc:256] RPC call timeout handler was delayed by -2.0919938179999917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:25:03.484877 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.485550 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:03.485690 26756 connection.cc:256] RPC call timeout handler was delayed by -2.0035935660002906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 12:25:03.486189 26756 connection.cc:256] RPC call timeout handler was delayed by -1.8736982370000987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4202s for a response
W20260709 12:25:03.486688 26756 connection.cc:256] RPC call timeout handler was delayed by -1.8276586920001137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2852s for a response
W20260709 12:25:03.487006 26756 connection.cc:256] RPC call timeout handler was delayed by -1.4140935660002469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
I20260709 12:25:03.487600 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.448544 29305 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.456171 29598 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.426409 29604 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:25:03.470008 29527 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:03.425724 26127 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:03.501305 26121 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:03.473412 29624 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:03.428303 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.503654 29624 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:03.489444 26120 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 6: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:03.484022 29582 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.504041 29624 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.504012 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.483842 29657 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.497283 26125 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:03.471359 29571 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:03.483017 29561 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.532325 29680 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:03.503996 26787 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:03.536782 29658 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:03.505786 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.539593 26120 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.532115 29571 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:03.541263 29571 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.542492 29571 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:03.489907 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55066.1us
I20260709 12:25:03.553252 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:03.554476 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.505901 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.557516 26127 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:03.558971 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.559423 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.560603 26127 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.561623 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.505932 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.506546 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.506219 29305 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:03.507766 29624 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.511766 29604 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:03.508605 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:03.512530 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800270aa0 after lost signal to thread 25986
I20260709 12:25:03.501657 29525 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:03.504168 29228 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.514423 26053 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:03.515316 29598 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:03.515486 29323 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:03.479337 29665 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.455786 29623 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.507786 29527 consensus_queue.cc:1243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:03.583729 29657 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:03.510545 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.490532 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.566483 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.555558 29292 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:03.582069 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800234060 after lost signal to thread 26078
W20260709 12:25:03.596994 25983 debug-util.cc:398] Leaking SignalData structure 0x7b080005d480 after lost signal to thread 26079
W20260709 12:25:03.598212 25983 debug-util.cc:398] Leaking SignalData structure 0x7b080035d0e0 after lost signal to thread 26153
I20260709 12:25:03.588335 26787 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEARNER]: Advancing to term 10
I20260709 12:25:03.590044 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:03.585153 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78688.5us
I20260709 12:25:03.599368 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.600419 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.600744 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.601384 26121 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:03.601524 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.583811 29525 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.529744 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:03.603986 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 93520.1us
I20260709 12:25:03.604573 26125 raft_consensus.cc:2376] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8. Re-sending same reply.
I20260709 12:25:03.579447 29613 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.579967 29546 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:03.584654 29604 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.584484 29272 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:03.608785 25983 debug-util.cc:398] Leaking SignalData structure 0x7b080023bac0 after lost signal to thread 26227
W20260709 12:25:03.596503 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 106045us
I20260709 12:25:03.590899 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.602303 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.605372 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.609966 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:03.606038 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:03.606487 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.591734 29669 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:03.609176 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800258320 after lost signal to thread 26228
W20260709 12:25:03.614713 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd0a0 after lost signal to thread 26815
I20260709 12:25:03.614879 26128 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:03.591068 29571 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.616890 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.619127 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.619454 26127 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
W20260709 12:25:03.622820 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08003c67a0 after lost signal to thread 29183
I20260709 12:25:03.624528 29661 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:03.625224 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800194080 after lost signal to thread 29570
I20260709 12:25:03.596479 29624 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.602686 29292 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.629385 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:03.630079 29292 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.630872 26125 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:03.595554 29657 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.626418 26120 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:03.607631 29527 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:03.604386 29658 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:03.625792 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.634090 26125 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:03.607564 29525 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.607574 29575 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:03.636857 26120 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:03.586963 29614 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:25:03.612574 29603 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.621372 29305 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.612923 29563 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.641260 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.639858 29519 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.618222 29606 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:03.627382 29517 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:03.633426 29669 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.637599 29657 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:03.618275 29613 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.638691 29509 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:03.659029 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.659899 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.669323 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.650555 29455 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:03.705778 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 99802.4us
I20260709 12:25:03.663915 26129 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:03.656842 29305 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:03.618690 29582 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.679335 29376 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.657938 29603 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.712780 29603 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.688982 29604 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.714679 29604 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.717015 29665 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:03.719622 29598 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:03.661255 29614 consensus_queue.cc:1243] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:03.706686 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.669504 29537 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:03.708333 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48538.8us
I20260709 12:25:03.651410 29527 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.699960 29606 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.689549 29517 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.733203 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.733858 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:03.738646 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.720750 29305 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.742769 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:03.770079 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 111736us
I20260709 12:25:03.732784 29509 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.741150 29537 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:03.741317 29517 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.741321 29606 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.746021 29598 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:03.770848 29669 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.772845 29527 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.732506 29665 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:03.793499 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49733.8us
I20260709 12:25:03.797748 29610 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:03.809645 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66947.7us
I20260709 12:25:03.795172 29345 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:03.746261 29693 raft_consensus.cc:2764] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:03.795099 29563 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:03.794541 29625 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:03.797904 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.820760 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.820937 26787 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:03.830344 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:03.821943 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:03.816426 29598 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:03.827512 29563 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.794706 29692 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:03.842351 29692 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.842775 29692 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.842756 29563 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.844913 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:03.845357 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:03.845027 29509 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.823321 29345 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.831790 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.833315 29551 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:03.834461 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.849058 26052 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:03.837208 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:03.838156 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:03.820319 29575 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:03.838165 29662 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:03.829169 29625 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.856415 29626 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:03.838420 29537 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:03.841638 29598 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.851861 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.849177 26048 raft_consensus.cc:2376] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10. Re-sending same reply.
I20260709 12:25:03.862042 26049 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:03.863364 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.863577 29626 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:03.864204 26045 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:03.864837 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.865093 29614 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:03.866045 29614 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:03.866072 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.867316 26046 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:03.853442 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.804177 29292 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.854866 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.856149 29625 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.856901 26202 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:03.857537 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:03.851027 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.858944 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.857172 29693 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.856105 29662 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:03.849282 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:03.868670 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:03.868497 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:03.860374 29272 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:25:03.931417 29519 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:03.846974 29345 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:03.934983 29688 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:25:03.848424 29658 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:03.939926 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78764.6us
I20260709 12:25:03.940763 29575 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:03.947132 29690 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.238s	user 0.006s	sys 0.000s
I20260709 12:25:03.932250 29292 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.922988 26054 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:03.937475 26127 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:03.853060 29551 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:03.854936 29610 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:03.866434 29614 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:03.849705 29563 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:03.989742 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 134896us
W20260709 12:25:04.016816 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148396us
W20260709 12:25:04.019464 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 170253us
I20260709 12:25:04.044817 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.023097 29504 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:04.027014 29605 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:03.992084 26199 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:04.024361 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:04.032167 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.864744 29626 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.056835 26054 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:04.032418 26127 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.023375 29658 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.032980 26129 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:25:03.981256 29613 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:03.988668 29665 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.972934 29680 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:04.024078 29518 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.023047 29345 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.019667 29693 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:04.054702 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.052776 29605 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.089654 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.059135 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.035256 29242 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.063261 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.038774 29551 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.076442 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:03.973613 29606 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.093081 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:04.091378 29699 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:04.078353 29582 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:04.072278 29658 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.065227 29292 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.102954 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.080345 29323 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:04.082857 29518 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.061993 29511 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:25:04.058069 29603 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
W20260709 12:25:04.116381 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.111505 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.095750 29242 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.166973 29272 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:04.175107 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63742.2us
W20260709 12:25:04.175478 29702 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:04.100035 28693 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.176613 28693 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.175274 29272 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:04.177309 28693 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.179690 29519 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:04.180163 26123 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:04.095763 29613 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:04.093909 29605 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:04.194546 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91667.6us
I20260709 12:25:04.092780 29680 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:04.111261 29536 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:04.085474 29665 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.206029 26124 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:04.207711 26198 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:04.209897 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.196424 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:04.093021 29693 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.214597 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.114820 29323 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:04.216717 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.170223 29623 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:04.183753 29518 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.227138 26203 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:04.225003 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:04.228927 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 115437us
I20260709 12:25:04.193853 29455 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:04.231084 26195 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:04.231775 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.210592 29519 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 12:25:04.216851 29657 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.003s
I20260709 12:25:04.233546 26201 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:04.233762 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.205092 29228 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.237989 26194 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.224740 29661 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.249121 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.249389 29661 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:04.249830 29661 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:04.240800 29698 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:25:04.255260 29662 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:04.230301 29323 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:04.256008 29662 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:04.256733 29606 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.256701 29662 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:04.258551 29665 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.243199 29517 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.247920 29657 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.261875 29305 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.263449 29657 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.263899 29305 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:04.261512 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.264669 26122 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:04.248100 29525 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.004s
I20260709 12:25:04.266052 29517 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:04.234721 29692 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.267402 29228 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.268580 26127 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.271586 26129 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:04.296496 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 237480us
I20260709 12:25:04.273190 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:04.283778 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.752s (SENT) [suppressed 51 similar messages]
I20260709 12:25:04.297607 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.300050 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.300834 26201 raft_consensus.cc:2376] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7. Re-sending same reply.
I20260709 12:25:04.301007 26195 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:25:04.301540 26195 pending_rounds.cc:85] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 6
I20260709 12:25:04.301848 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.301987 26195 pending_rounds.cc:107] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
I20260709 12:25:04.302958 26195 raft_consensus.cc:2917] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 12:25:04.303604 26195 raft_consensus.cc:2889] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.300177 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.304760 29598 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.307113 29598 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.301631 29661 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.310606 29661 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:04.301575 29376 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.001s
I20260709 12:25:04.316552 26049 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:25:04.317077 26049 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 6
I20260709 12:25:04.317301 26049 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:25:04.317550 26049 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.317819 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.318109 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.318832 26194 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.259652 29680 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.322131 26203 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.321944 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.323124 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.313079 29640 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.325433 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.329067 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.330986 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.331748 26129 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:04.338177 29228 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.324877 29376 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:25:04.339020 28693 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.339304 29518 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.339932 29626 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:04.344019 26201 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.345758 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.347429 26120 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:04.293669 29525 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.348271 26194 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:04.322692 29305 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.276149 29605 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.349980 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.351222 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:04.355522 26125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.008s	sys 0.000s
W20260709 12:25:04.356122 26125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.008s	sys 0.000s
I20260709 12:25:04.325224 29710 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.361071 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.365837 26120 raft_consensus.cc:1217] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Deduplicated request from leader. Original: 6.5->[8.6-9.7]   Dedup: 8.6->[9.7-9.7]
I20260709 12:25:04.366685 26120 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [8.6-9.7]
I20260709 12:25:04.370059 26120 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:04.371199 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2627) took 1074 ms. Trace:
I20260709 12:25:04.371789 26125 rpcz_store.cc:276] 0709 12:25:03.296750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.641155 (+344405us) service_pool.cc:224] Handling call
0709 12:25:03.654580 (+ 13425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:03.654991 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.655027 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:03.838360 (+183333us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:25:03.839005 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.356624 (+517619us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:04.356700 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.356830 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.356892 (+    62us) raft_consensus.cc:1685] finished
0709 12:25:04.356910 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.371123 (+ 14213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6708,"raft.queue_time_us":1201,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":14402,"spinlock_wait_cycles":3369728,"thread_start_us":517108,"threads_started":1,"wal-append.queue_time_us":397749}
I20260709 12:25:04.373606 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.376614 28693 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:04.335135 29640 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.352988 29305 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:04.365468 29625 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.381142 26196 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:04.382431 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.383880 29692 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:04.387108 29525 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:04.388129 26194 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:25:04.413131 26194 pending_rounds.cc:85] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 6
I20260709 12:25:04.413496 26194 pending_rounds.cc:107] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:25:04.413838 26194 raft_consensus.cc:2889] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.390131 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.415154 26120 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.393616 26202 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.393705 29624 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:04.400025 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:04.421904 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.407414 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.350858 29518 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:04.412348 29242 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:04.427728 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.433077 26787 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:04.433809 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.434655 26791 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.389623 29709 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:04.411602 29715 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:25:04.402868 29706 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.457041 29625 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.459309 29693 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.461730 29693 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:04.462766 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1867) took 1164 ms. Trace:
I20260709 12:25:04.463838 29605 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:04.462852 29551 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.465721 26200 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.466195 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.466794 29551 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:04.463402 26124 rpcz_store.cc:276] 0709 12:25:03.298743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.679621 (+380878us) service_pool.cc:224] Handling call
0709 12:25:03.679968 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.206929 (+526961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.462371 (+255442us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400369cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:04.462678 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155899648}
I20260709 12:25:04.469517 26124 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:04.471056 26124 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:04.414956 29305 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:04.445843 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52757.1us
I20260709 12:25:04.446416 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:04.447496 29714 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 12:25:04.448378 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.451274 26791 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:04.449903 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.452672 26195 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:04.454798 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.455396 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.455178 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.445461 29563 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:04.436342 29525 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:04.484619 26196 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.484027 26784 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:04.472834 29658 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.440222 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.476073 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.471243 29323 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.479815 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.446792 26120 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:04.488919 26788 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:04.489591 26790 raft_consensus.cc:2368] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
W20260709 12:25:04.481370 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 261215us
I20260709 12:25:04.487355 26202 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.453729 29624 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.486598 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.491770 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.493520 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.494582 26783 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:04.495697 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.496376 26788 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:04.497073 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.499645 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.416220 29626 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:04.504505 29228 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.515270 29710 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.519680 29228 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:04.520767 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.523866 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.524883 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.525997 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:04.528358 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:04.529472 29705 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.361s	user 0.003s	sys 0.005s
I20260709 12:25:04.529377 26195 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:04.532883 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.534598 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.535298 26785 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
W20260709 12:25:04.535881 26019 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.626s (SENT)
I20260709 12:25:04.536996 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1876) took 1080 ms. Trace:
I20260709 12:25:04.494500 29624 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.537987 29624 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.537411 26121 rpcz_store.cc:276] 0709 12:25:03.456555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.941829 (+485274us) service_pool.cc:224] Handling call
0709 12:25:03.942225 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.325193 (+382968us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:04.325228 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.325737 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.425327 (+ 99590us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:04.425361 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.425417 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.536771 (+111354us) raft_consensus.cc:1685] finished
0709 12:25:04.536790 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.536941 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46836,"raft.queue_time_us":94965,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":7345,"thread_start_us":391513,"threads_started":2,"wal-append.queue_time_us":130945}
I20260709 12:25:04.539373 29571 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:04.539497 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.485386 29563 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:04.540439 26785 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:04.522017 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.540858 29568 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:04.541141 26788 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:04.523584 29640 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.541491 29563 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.543381 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.543428 29563 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.544201 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:04.546018 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.546756 26785 raft_consensus.cc:2368] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:04.546613 29323 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.492811 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56074
I20260709 12:25:04.541393 29568 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:04.547873 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.573838 26120 raft_consensus.cc:606] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Received request to transfer leadership
I20260709 12:25:04.574193 26788 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:04.500169 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.575443 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.556623 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.558115 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.483490 29551 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:25:04.560541 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:04.580916 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80888.3us
I20260709 12:25:04.560678 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.581771 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:04.562075 29693 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:04.563733 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.582623 26787 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:04.583475 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.583947 26787 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.553412 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.564519 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.561647 29605 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:25:04.551455 29658 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:04.521261 29504 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:04.592813 29546 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:04.597146 29504 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.597761 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.598802 26782 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:04.552618 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2612) took 1336 ms (client timeout 1548 ms). Trace:
W20260709 12:25:04.600353 26123 rpcz_store.cc:269] 0709 12:25:03.216648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.220621 (+  3973us) service_pool.cc:224] Handling call
0709 12:25:04.552414 (+1331793us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:04.552558 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138976512}
I20260709 12:25:04.571305 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:04.599849 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.602002 26782 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:04.553282 29511 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:04.540072 29571 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:04.537272 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.626098 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.601441 26127 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:04.574373 26120 raft_consensus.cc:614] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:04.559499 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.627415 26786 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:04.581884 26791 raft_consensus.cc:2376] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10. Re-sending same reply.
I20260709 12:25:04.560770 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.585572 26785 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.588809 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.629717 26788 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:04.593990 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:04.603348 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.606246 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 509) took 1141 ms. Trace:
I20260709 12:25:04.603161 26787 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:04.622246 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.622898 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.573280 26129 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:04.633683 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.637085 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:04.639303 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.632365 26122 rpcz_store.cc:276] 0709 12:25:03.465274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.945135 (+479861us) service_pool.cc:224] Handling call
0709 12:25:03.945609 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.284329 (+338720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.605915 (+321586us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54001da954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:04.606161 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733244672}
I20260709 12:25:04.640386 26194 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.644055 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:04.645256 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51798.5us
I20260709 12:25:04.646993 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.648051 26787 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:04.654290 26198 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:04.591516 29662 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.667804 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:04.669641 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.670284 26048 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:04.670819 26202 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:04.678961 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:04.686216 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.687281 26049 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.688998 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.593397 29713 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:04.699213 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.712118 26048 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:04.702500 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56338.4us
I20260709 12:25:04.705830 26785 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:04.709824 26051 pending_rounds.cc:85] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 1
I20260709 12:25:04.714238 26051 pending_rounds.cc:107] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:25:04.714433 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.714638 26051 raft_consensus.cc:2889] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.625900 29511 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:04.716416 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.716924 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.696261 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.623517 29546 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:04.729712 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.722088 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.730417 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.730834 26050 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.731462 26045 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:04.732666 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53788.8us
I20260709 12:25:04.732992 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.732681 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.736121 26046 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.736256 26053 raft_consensus.cc:2376] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10. Re-sending same reply.
I20260709 12:25:04.648568 28693 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.742106 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.749996 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:04.741262 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.620903 29710 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 12:25:04.742641 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.747448 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.604137 29640 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
W20260709 12:25:04.757051 26018 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.732s (SENT)
I20260709 12:25:04.784471 26785 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.753499 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.787220 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.783314 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:04.767869 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.792838 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.711880 29713 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:04.715633 29455 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.800155 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.807121 26199 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:04.801476 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:04.804627 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:04.808665 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.809556 26197 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:04.810915 26047 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:04.804174 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.822435 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.820647 26049 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:04.741866 29527 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:04.825094 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.712042 29683 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:25:04.839443 26045 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.825834 26053 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:04.712401 29506 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 12:25:04.748270 29667 raft_consensus.cc:2764] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:25:04.865756 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.845558 26121 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:04.845383 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:04.870812 26122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:04.813066 29713 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:04.785794 29604 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:04.849268 26787 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:04.850188 29525 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:04.924077 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 514) took 1421 ms. Trace:
I20260709 12:25:04.853132 26050 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
W20260709 12:25:04.875209 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 133435us
I20260709 12:25:04.846261 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.926884 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:04.882442 26120 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.884440 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:04.930241 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:04.919278 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 114775us
I20260709 12:25:04.731040 29661 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:25:04.720479 29345 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:25:04.926550 26128 rpcz_store.cc:276] 0709 12:25:03.502223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.109645 (+607422us) service_pool.cc:224] Handling call
0709 12:25:04.197614 (+ 87969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.197896 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:04.198249 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.610086 (+411837us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b18003ac720. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:25:04.923577 (+313491us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.923658 (+    81us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:04.923727 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.923822 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.923987 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76677,"prepare.queue_time_us":139,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3129,"spinlock_wait_cycles":13787904,"thread_start_us":236518,"threads_started":1}
I20260709 12:25:04.935289 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.935451 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.928969 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.943292 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:04.935557 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.945051 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.941030 26783 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:04.936085 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:04.868535 29509 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:04.949527 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:04.955166 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:04.788295 28693 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:04.956856 26787 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.957110 26198 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:25:04.957653 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:04.939173 29711 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:25:04.964449 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:04.966303 26783 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:04.918519 29604 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:04.945343 29376 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:04.976064 29640 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.918780 29242 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:25:04.954679 29601 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:05.008667 26194 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:04.931506 29506 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:25:05.026659 29680 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:05.027756 29519 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:05.029477 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:05.099565 26053 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:05.032848 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:05.101059 26049 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:05.034704 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:04.962294 29561 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:04.960860 29669 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:25:05.062669 29601 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:05.062080 26048 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:05.065641 29568 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:05.067432 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:05.117089 26783 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:05.042193 29640 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.075918 29614 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.088886 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:05.092854 29720 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:25:05.043166 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:05.199770 26198 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:05.054639 29723 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:25:05.205529 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.114157 26122 raft_consensus.cc:606] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:05.103541 29680 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:04.938858 29546 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:05.238199 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 484767us
W20260709 12:25:05.130932 29624 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.004s
I20260709 12:25:05.132544 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:05.104600 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:05.143718 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:05.240442 26048 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:05.244858 26784 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
W20260709 12:25:05.246755 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 316582us
I20260709 12:25:05.247838 26049 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:05.164490 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:05.248118 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:05.074262 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.206303 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:05.206377 26791 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:05.208021 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.207803 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:05.098568 29519 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:05.236778 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.098677 29606 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 12:25:05.266283 26048 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:05.153604 29640 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:05.267076 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59138us
I20260709 12:25:05.267701 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.268344 26049 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:05.269402 26050 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:05.251757 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.255645 26202 raft_consensus.cc:2368] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9.
I20260709 12:25:05.129459 29662 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.942651 29511 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.256085 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:05.260492 26018 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.950s (SENT)
I20260709 12:25:05.274005 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.275056 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:05.152645 29571 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:05.221390 29292 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.276868 26048 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:05.285445 26201 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:05.286450 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.276921 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:05.291926 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.270440 26167 leader_election.cc:400] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:05.293246 26167 leader_election.cc:403] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:05.237057 29614 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.305080 26050 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:05.307448 26049 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:25:05.308055 26049 pending_rounds.cc:85] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 6
I20260709 12:25:05.309636 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:05.308687 26049 pending_rounds.cc:107] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:25:05.299053 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.286458 29613 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.321130 26195 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:05.324416 26046 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.000s
I20260709 12:25:05.335629 26046 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
W20260709 12:25:05.326769 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.273097 29658 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.336557 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:05.337179 26782 raft_consensus.cc:2376] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10. Re-sending same reply.
W20260709 12:25:05.337553 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89510.8us
W20260709 12:25:05.294057 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53649.9us
I20260709 12:25:05.336251 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:05.314152 26124 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:05.289580 29693 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:05.289302 29601 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:05.339244 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:05.339618 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:05.334239 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 369837us
I20260709 12:25:05.128983 29604 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.334657 26049 raft_consensus.cc:2889] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:05.306983 29605 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.350929 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:05.352480 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:05.347530 29242 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:05.258828 29624 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.358357 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:05.370757 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 443903us
I20260709 12:25:05.373826 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:05.330168 29568 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:05.372344 29732 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:05.306749 29692 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:25:05.388134 29730 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.406098 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.407357 26782 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:05.408650 29518 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:05.409595 29626 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:05.410006 29626 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:05.410971 29626 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.308451 29292 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.334209 29683 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:25:05.537995 26120 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:05.543077 26201 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:05.552927 29721 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
W20260709 12:25:05.523454 29642 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.009s	sys 0.020s
I20260709 12:25:05.412889 29376 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:05.413597 29604 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:05.579552 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.017s	sys 0.033s
W20260709 12:25:05.665041 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.017s	sys 0.033s
I20260709 12:25:05.363643 29693 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:05.634758 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.022s	sys 0.030s
W20260709 12:25:05.536374 29723 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:25:05.587024 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:05.673400 26120 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:05.365487 29658 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:05.666109 29737 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
W20260709 12:25:05.667999 29734 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:25:05.372179 29571 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.179s
I20260709 12:25:05.666385 26129 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:05.668629 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.022s	sys 0.030s
W20260709 12:25:05.590265 29726 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.548s	user 0.000s	sys 0.002s
W20260709 12:25:05.634521 29504 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.016s	sys 0.012s
W20260709 12:25:05.723589 29504 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.016s	sys 0.013s
I20260709 12:25:05.724116 29746 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:05.726913 29455 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.000s
I20260709 12:25:05.727378 29455 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.729490 29292 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:05.730633 29693 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:05.730963 29735 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:05.551805 29626 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:05.609205 26045 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.758s	user 0.011s	sys 0.000s
I20260709 12:25:05.755798 26045 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
W20260709 12:25:05.644171 26790 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.695s	user 0.007s	sys 0.003s
I20260709 12:25:05.346009 29680 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:05.667268 29517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.001s	sys 0.030s
W20260709 12:25:05.665364 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 314485us
I20260709 12:25:05.680697 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.768919 26782 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:05.768852 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:05.665083 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 312687us
W20260709 12:25:05.665840 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.708285 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 372049us
I20260709 12:25:05.771250 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:05.671298 29727 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 12:25:05.753439 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2633) took 1087 ms. Trace:
I20260709 12:25:05.754254 29624 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:05.757277 26790 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:25:05.762427 26045 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:25:05.767311 26783 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.000s
I20260709 12:25:05.708460 29662 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:05.772915 26051 rpcz_store.cc:276] 0709 12:25:04.666271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.708987 (+ 42716us) service_pool.cc:224] Handling call
0709 12:25:04.709507 (+   520us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:04.742263 (+ 32756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.742294 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:04.743015 (+   721us) log.cc:844] Serialized 78 byte log entry
0709 12:25:05.206814 (+463799us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:05.266499 (+ 59685us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.325430 (+ 58931us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.325937 (+   507us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.332532 (+  6595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.332640 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.753213 (+420573us) raft_consensus.cc:1685] finished
0709 12:25:05.753232 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.753375 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126068,"prepare.queue_time_us":69090,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":79898,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":36786,"spinlock_wait_cycles":178991744,"thread_start_us":366991,"threads_started":1,"wal-append.queue_time_us":613300}
I20260709 12:25:05.803416 26783 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:05.806919 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 619) took 1241 ms. Trace:
I20260709 12:25:05.804203 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.775535 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:05.777485 29680 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:05.768175 29517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.002s	sys 0.030s
I20260709 12:25:05.780117 26049 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:25:05.780483 29305 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.296s	user 0.013s	sys 0.001s
I20260709 12:25:05.782089 29537 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:05.785902 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2634) took 1119 ms. Trace:
I20260709 12:25:05.773128 29667 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.791154 26129 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:05.791718 29661 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:05.792778 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:05.793331 29683 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:05.794512 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2632) took 1128 ms. Trace:
I20260709 12:25:05.797006 29658 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:05.798383 26198 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.000s
I20260709 12:25:05.803073 29740 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.809551 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 617) took 1244 ms. Trace:
I20260709 12:25:05.813527 26054 rpcz_store.cc:276] 0709 12:25:04.666618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.710210 (+ 43592us) service_pool.cc:224] Handling call
0709 12:25:04.710583 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.713427 (+  2844us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:04.918878 (+205451us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.362370 (+443492us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.362543 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.362599 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.362658 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.785105 (+422447us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18003d7f80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419
0709 12:25:05.785649 (+   544us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b5400227754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:05.785691 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.785837 (+   146us) inbound_call.cc:177] Queueing success response
0709 12:25:05.793069 (+  7232us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b18003d7f80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
Metrics: {"mutex_wait_us":25421,"prepare.queue_time_us":346310,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":99,"raft.queue_time_us":44393,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":243,"spinlock_wait_cycles":843364096,"thread_start_us":643065,"threads_started":2}
I20260709 12:25:05.790259 29745 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:05.769322 29323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.008s	sys 0.020s
W20260709 12:25:05.831027 29323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.008s	sys 0.020s
I20260709 12:25:05.773638 29603 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:05.818668 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:05.831952 29603 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.832556 26788 rpcz_store.cc:276] 0709 12:25:04.565437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.774091 (+1208654us) service_pool.cc:224] Handling call
0709 12:25:05.809468 (+ 35377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.833345 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:05.827109 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:05.807319 26783 rpcz_store.cc:276] 0709 12:25:04.565983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.067061 (+501078us) service_pool.cc:224] Handling call
0709 12:25:05.806846 (+739785us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.772727 29345 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:25:05.836009 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 610) took 1272 ms. Trace:
I20260709 12:25:05.830794 29745 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.832307 29752 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:05.828599 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.813310 29537 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:05.836474 26788 rpcz_store.cc:276] 0709 12:25:04.563589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.834507 (+1270918us) service_pool.cc:224] Handling call
0709 12:25:05.835321 (+   814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.835589 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.835646 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.835709 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.835739 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.835828 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.835934 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:05.838871 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46159.3us
I20260709 12:25:05.901249 29537 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.902400 29537 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.902999 29736 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:05.903606 29736 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:05.903962 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.904467 29613 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:05.904655 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.905020 29613 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:05.906013 29613 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.906982 29755 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:05.823679 26047 rpcz_store.cc:276] 0709 12:25:04.665946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.671018 (+  5072us) service_pool.cc:224] Handling call
0709 12:25:04.721543 (+ 50525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.811363 (+ 89820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.794262 (+982899us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b540008e3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:05.794451 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204876544}
W20260709 12:25:05.834936 29563 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.265s	user 0.002s	sys 0.012s
I20260709 12:25:05.909127 29563 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:05.909606 29563 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.909874 26047 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:05.910071 29563 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.910573 29292 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.835984 26783 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEARNER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:05.911355 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 612) took 1347 ms. Trace:
I20260709 12:25:05.911087 29292 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.911727 26783 rpcz_store.cc:276] 0709 12:25:04.564067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.835310 (+1271243us) service_pool.cc:224] Handling call
0709 12:25:05.835672 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.911120 (+ 75448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.911301 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.913285 26783 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:05.913551 29561 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:05.913915 29561 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:05.851706 29749 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:05.915109 29749 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:05.914958 29561 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.915529 29749 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:05.916603 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:05.916435 29749 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.917754 29749 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:05.918140 26789 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 6. Current term is 7. Ops: [4.5-6.7]
I20260709 12:25:05.919303 29736 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:05.920899 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 149766us
I20260709 12:25:05.920095 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 873) took 1494 ms. Trace:
I20260709 12:25:05.921470 29511 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:05.910104 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.921653 29228 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:05.816510 29305 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.922235 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:05.922303 29228 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.922829 29731 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.923326 29731 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.923840 26786 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:05.924908 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1788) took 1265 ms. Trace:
I20260709 12:25:05.921483 26789 rpcz_store.cc:276] 0709 12:25:04.426080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.426279 (+   199us) service_pool.cc:224] Handling call
0709 12:25:04.426771 (+   492us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:05.918540 (+1491769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.919829 (+  1289us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54003c5854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:05.920038 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3429047040}
I20260709 12:25:05.925231 29509 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:05.925300 26194 rpcz_store.cc:276] 0709 12:25:04.659996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.704867 (+ 44871us) service_pool.cc:224] Handling call
0709 12:25:04.705261 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.053626 (+348365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.924673 (+871047us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400301e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:05.924853 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656803072}
I20260709 12:25:05.925945 29509 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:05.926306 26789 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:05.926899 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 625) took 1359 ms. Trace:
I20260709 12:25:05.858506 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 615) took 1293 ms. Trace:
I20260709 12:25:05.859509 29739 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.927521 29757 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.927836 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:05.842747 29551 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:05.823637 29661 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:05.865873 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1883) took 1041 ms. Trace:
I20260709 12:25:05.943081 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:05.855301 29519 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.866811 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 629) took 1298 ms. Trace:
I20260709 12:25:05.868556 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 407) took 1104 ms. Trace:
I20260709 12:25:05.943378 26123 rpcz_store.cc:276] 0709 12:25:04.824369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.826099 (+  1730us) service_pool.cc:224] Handling call
0709 12:25:04.862055 (+ 35956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.936606 (+ 74551us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:04.936640 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.937130 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.419599 (+482469us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:05.419692 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.419807 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.865646 (+445839us) raft_consensus.cc:1685] finished
0709 12:25:05.865670 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.865811 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74124,"raft.queue_time_us":97610,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":17290,"thread_start_us":414038,"threads_started":1,"wal-append.queue_time_us":837426}
I20260709 12:25:05.869006 29603 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:05.874948 29345 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.823086 29669 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:05.890336 29746 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:05.826596 29667 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.822916 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:05.926698 29624 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:05.840703 29740 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.966821 29624 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:05.927338 26791 rpcz_store.cc:276] 0709 12:25:04.564897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.857594 (+1292697us) service_pool.cc:224] Handling call
0709 12:25:05.857915 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.858138 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.858166 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.858221 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.858256 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.858314 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.858442 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.928248 29757 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.928301 29571 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:05.929539 26789 rpcz_store.cc:276] 0709 12:25:04.567522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.925770 (+1358248us) service_pool.cc:224] Handling call
0709 12:25:05.926119 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.926679 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.926847 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.860878 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58316 (request call id 15) took 1037 ms. Trace:
I20260709 12:25:05.968762 26122 rpcz_store.cc:276] 0709 12:25:04.823733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.825780 (+  2047us) service_pool.cc:224] Handling call
0709 12:25:05.860693 (+1034913us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:05.860786 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557990144}
I20260709 12:25:05.913923 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 624) took 1346 ms. Trace:
I20260709 12:25:05.969769 26783 rpcz_store.cc:276] 0709 12:25:04.567268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.912784 (+1345516us) service_pool.cc:224] Handling call
0709 12:25:05.913087 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.913698 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.913872 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.971804 26783 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:05.937230 26124 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:05.942229 26788 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:05.931124 29561 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:05.949100 26000 auto_rebalancer.cc:893] Removing replace marker failed: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:25:05.974886 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 626) took 1407 ms. Trace:
W20260709 12:25:05.974923 26000 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:25:05.975543 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:25:05.975427 26788 rpcz_store.cc:276] 0709 12:25:04.567755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.934893 (+1367138us) service_pool.cc:224] Handling call
0709 12:25:05.941918 (+  7025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.974573 (+ 32655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.974810 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.948920 29323 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:05.949476 29693 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:05.990116 29323 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.990569 29693 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:05.990695 29323 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:05.951905 26790 rpcz_store.cc:276] 0709 12:25:04.568537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.591061 (+ 22524us) service_pool.cc:224] Handling call
0709 12:25:05.866608 (+1275547us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400715a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:05.866726 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709067008}
I20260709 12:25:05.955022 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:05.913347 29292 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:05.964835 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 634) took 1343 ms. Trace:
I20260709 12:25:05.984930 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:06.002316 26782 rpcz_store.cc:276] 0709 12:25:04.621730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.805530 (+1183800us) service_pool.cc:224] Handling call
0709 12:25:05.964747 (+159217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.986768 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:06.003854 26791 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:25:05.952980 29661 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:05.967480 29624 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:05.989449 26789 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
W20260709 12:25:05.955981 29640 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.006s
I20260709 12:25:06.003901 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:05.987275 29722 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.769s	user 0.008s	sys 0.035s
I20260709 12:25:05.944855 29604 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.005079 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 628) took 1405 ms. Trace:
I20260709 12:25:06.017094 26783 rpcz_store.cc:276] 0709 12:25:04.568280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.971114 (+1402834us) service_pool.cc:224] Handling call
0709 12:25:05.971546 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.973536 (+  1990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.973765 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.951588 29706 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:25:05.956662 29614 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.011304 26128 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:06.012878 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.051772 26122 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:06.053149 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.054255 26122 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:05.952574 26052 rpcz_store.cc:276] 0709 12:25:04.763841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.268857 (+505016us) service_pool.cc:224] Handling call
0709 12:25:05.303712 (+ 34855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:05.310802 (+  7090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.310834 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:05.365896 (+ 55062us) log.cc:844] Serialized 52 byte log entry
0709 12:25:05.803119 (+437223us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.822773 (+ 19654us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.823008 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.823088 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.868314 (+ 45226us) raft_consensus.cc:1685] finished
0709 12:25:05.868339 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.868496 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6572,"prepare.queue_time_us":33575,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":11070,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":17587,"thread_start_us":223655,"threads_started":1,"wal-append.queue_time_us":444047}
I20260709 12:25:05.961572 29669 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:06.015524 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:06.009059 26789 pending_rounds.cc:85] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
I20260709 12:25:06.066643 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:06.067313 26789 pending_rounds.cc:107] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
I20260709 12:25:06.067806 26789 raft_consensus.cc:2917] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 12:25:06.068413 26789 raft_consensus.cc:2889] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:06.069428 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 627) took 1501 ms. Trace:
I20260709 12:25:06.069746 26789 rpcz_store.cc:276] 0709 12:25:04.567990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.968793 (+1400803us) service_pool.cc:224] Handling call
0709 12:25:05.969083 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.069136 (+100053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.069367 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:06.007174 26791 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:06.072057 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 635) took 1419 ms. Trace:
I20260709 12:25:06.072605 26791 rpcz_store.cc:276] 0709 12:25:04.652787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.004664 (+1351877us) service_pool.cc:224] Handling call
0709 12:25:06.004953 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.071689 (+ 66736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.071967 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.989493 29667 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:05.959825 29228 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:05.973956 29305 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:06.038415 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 611) took 1474 ms (client timeout 1950 ms). Trace:
I20260709 12:25:05.926915 29509 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:05.993125 29740 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:06.018193 26782 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:06.090598 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 790) took 1510 ms. Trace:
I20260709 12:25:06.091157 26782 rpcz_store.cc:276] 0709 12:25:04.580116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.003413 (+1423297us) service_pool.cc:224] Handling call
0709 12:25:06.017876 (+ 14463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.090227 (+ 72351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.090504 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.962114 29746 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:06.018195 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:06.046396 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:06.009168 29623 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:06.086338 26786 rpcz_store.cc:269] 0709 12:25:04.563834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.788555 (+224721us) service_pool.cc:224] Handling call
0709 12:25:06.038064 (+1249509us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:06.038321 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2530988800}
I20260709 12:25:06.035724 29642 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:05.961298 29761 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:25:06.093130 26784 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:05.991653 29693 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.094569 26786 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:06.039309 26790 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:06.011377 29376 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:06.076211 29713 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.131s	user 0.013s	sys 0.001s
I20260709 12:25:06.017299 29640 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.100075 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 796) took 1263 ms. Trace:
I20260709 12:25:06.095188 29623 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.102622 29759 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:25:06.078186 29669 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:06.079849 29730 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.674s	user 0.012s	sys 0.000s
I20260709 12:25:06.045820 29604 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.087816 29242 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:06.128547 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:06.128654 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 295358us
I20260709 12:25:06.121279 26784 rpcz_store.cc:276] 0709 12:25:04.836696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.040216 (+1203520us) service_pool.cc:224] Handling call
0709 12:25:06.092875 (+ 52659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.099787 (+  6912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.100004 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:06.096755 29757 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:06.134397 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:06.134599 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 191605us
I20260709 12:25:06.123754 29642 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:06.114262 29731 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:06.075774 29614 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.154105 26787 raft_consensus.cc:2368] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8.
W20260709 12:25:06.112510 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.096433 29517 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:25:06.104199 28693 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:06.142876 29718 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:25:06.143586 29642 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.144850 29613 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:06.145201 29604 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:06.145383 29731 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:06.146513 29624 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:06.148267 29506 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:06.148792 29504 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:06.149871 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.114393 29745 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:06.065002 29739 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:25:06.155974 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.196924 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.159777 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2639) took 1565 ms. Trace:
I20260709 12:25:06.126780 29623 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:06.157047 29242 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:06.169047 29640 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:06.171051 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.150146 29730 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.157776 29710 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.123s
W20260709 12:25:06.193946 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.126636 29713 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.198102 26125 rpcz_store.cc:276] 0709 12:25:04.594625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.594791 (+   166us) service_pool.cc:224] Handling call
0709 12:25:04.595090 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.734212 (+139122us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:04.797097 (+ 62885us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.367857 (+570760us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.369732 (+  1875us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.369814 (+    82us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:05.369874 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.159459 (+789585us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540037c154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:06.159521 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.159701 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81431,"prepare.queue_time_us":65198,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":12680,"raft.queue_time_us":485449,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":10328,"spinlock_wait_cycles":319228800,"thread_start_us":485331,"threads_started":1}
I20260709 12:25:06.224959 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.226873 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.202025 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:06.235666 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 302417us
W20260709 12:25:06.238550 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.242570 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.252449 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 913252us
I20260709 12:25:06.293690 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:06.242307 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 300921us
I20260709 12:25:06.292303 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.162051 29740 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 12:25:06.195089 29603 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.481962 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:06.514549 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.515518 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:06.481886 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:06.516987 26125 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:25:06.483598 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 155631us
I20260709 12:25:06.406255 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 789) took 1826 ms. Trace:
I20260709 12:25:06.518330 26790 rpcz_store.cc:276] 0709 12:25:04.579728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.992303 (+1412575us) service_pool.cc:224] Handling call
0709 12:25:06.039001 (+ 46698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.105735 (+ 66734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.406076 (+300341us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b540063abd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:06.406197 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106538880}
W20260709 12:25:06.358775 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.403380 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 792) took 1820 ms. Trace:
W20260709 12:25:06.368420 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.297096 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.230055 29761 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.405498 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 555) took 1088 ms. Trace:
I20260709 12:25:06.524089 26201 rpcz_store.cc:276] 0709 12:25:05.317003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.320347 (+  3344us) service_pool.cc:224] Handling call
0709 12:25:06.405202 (+1084855us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002b5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:06.405405 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369111424}
I20260709 12:25:06.181897 29228 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 12:25:06.331434 26129 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:06.173254 29613 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.155481 29746 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:06.386740 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 184794us
I20260709 12:25:06.541692 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:06.395388 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.546268 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:06.546955 26202 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:06.505486 26120 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
W20260709 12:25:06.506875 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.548113 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:06.508064 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 546) took 1694 ms. Trace:
I20260709 12:25:06.302968 29692 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.521092 26124 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:06.468019 29776 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.520120 26787 rpcz_store.cc:276] 0709 12:25:04.582777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.267367 (+684590us) service_pool.cc:224] Handling call
0709 12:25:06.403141 (+1135774us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:06.403303 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035874688}
I20260709 12:25:06.458557 29624 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.530392 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 147651us
I20260709 12:25:06.548358 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:06.597043 29765 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 12:25:06.597210 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:06.598032 26054 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:06.598526 29778 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:25:06.598804 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:06.599428 26054 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:06.600003 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1870) took 3234 ms. Trace:
I20260709 12:25:06.468691 29242 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.548419 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:06.543565 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:06.485345 29751 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.341s
I20260709 12:25:06.600200 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:06.607851 26046 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:06.608726 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.610148 26194 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:06.624408 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.625803 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.626865 26124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "407574b4cda94da99be372d249d21fd5"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:06.627429 26124 raft_consensus.cc:606] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:06.610766 26047 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:25:06.607224 29763 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.439s	user 0.000s	sys 0.001s
I20260709 12:25:06.551378 26125 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:06.552850 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:06.600438 26126 rpcz_store.cc:276] 0709 12:25:03.365872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.796659 (+430787us) service_pool.cc:224] Handling call
0709 12:25:03.847277 (+ 50618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.847781 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.847813 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.852778 (+  4965us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.551716 (+698938us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b180031a200. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:04.783280 (+231564us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:04.783323 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.783393 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.986636 (+1203243us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:06.133204 (+146568us) raft_consensus.cc:1685] finished
0709 12:25:06.133227 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.599824 (+466597us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b54001f90d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:06.599942 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4236,"raft.queue_time_us":2048,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6803,"spinlock_wait_cycles":171347840,"thread_start_us":231383,"threads_started":1,"wal-append.queue_time_us":1123346}
I20260709 12:25:06.578684 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:06.579419 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:06.579618 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:06.638440 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 58947.5us
I20260709 12:25:06.639321 29730 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.639892 29730 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:06.640355 29730 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:06.642800 29736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.010s
W20260709 12:25:06.643234 29736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.002s	sys 0.010s
I20260709 12:25:06.397253 29518 raft_consensus.cc:2764] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:06.548904 26197 rpcz_store.cc:276] 0709 12:25:04.813862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.832408 (+ 18546us) service_pool.cc:224] Handling call
0709 12:25:04.863434 (+ 31026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.863851 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.863874 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.864610 (+   736us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.328488 (+463878us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:05.341430 (+ 12942us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.902137 (+560707us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b180041da00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:25:06.507406 (+605269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.507548 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.507637 (+    89us) raft_consensus.cc:1685] finished
0709 12:25:06.507756 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.507973 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25649,"prepare.queue_time_us":19336,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":574127,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":157,"spinlock_wait_cycles":10374016,"thread_start_us":812926,"threads_started":2,"wal-append.queue_time_us":984162}
I20260709 12:25:06.523437 29640 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.646683 29640 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.649142 29640 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:06.281201 29376 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.386395 29683 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:06.584580 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:06.589964 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74572.9us
I20260709 12:25:06.584666 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:06.665802 26199 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:06.584825 26196 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:06.592475 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:06.667223 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74821.4us
I20260709 12:25:06.591977 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:06.529639 29604 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.668041 29518 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.668082 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:06.667120 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:06.668649 29518 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.668676 29527 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:06.669991 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.671550 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.672478 29506 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.670043 29527 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:06.667595 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.675137 29527 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.675206 29518 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:06.680962 29527 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:06.681730 29527 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.680594 29683 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.685599 29527 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:06.687379 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:06.581652 29614 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:06.592448 29624 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.580828 29762 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:06.696810 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:06.718118 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 169774us
I20260709 12:25:06.720094 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:06.682411 26124 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:25:06.729842 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62816.4us
I20260709 12:25:06.731674 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:06.732100 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:06.601696 29770 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:25:06.743755 26198 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:25:06.752552 26198 pending_rounds.cc:85] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 4
I20260709 12:25:06.752858 26198 pending_rounds.cc:107] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:25:06.753111 26198 raft_consensus.cc:2889] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:06.756423 26053 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:06.757517 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:06.757551 26045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.016s	sys 0.031s
W20260709 12:25:06.758880 26045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.017s	sys 0.031s
I20260709 12:25:06.748026 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:06.765847 26053 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:06.606750 29680 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.000s
I20260709 12:25:06.774212 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:06.691959 29772 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.718423 29527 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:06.785351 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:06.719239 29762 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:06.742061 29603 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:06.792205 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.762728 29614 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:06.733155 29518 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.758042 29757 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:06.793212 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 250224us
I20260709 12:25:06.744154 29692 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:06.799141 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.775733 29345 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:06.812171 26124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:06.813208 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.592708 29776 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:06.823833 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 103817us
I20260709 12:25:06.787029 29527 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:06.796222 29762 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.820106 29757 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.891098 26124 raft_consensus.cc:606] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.854542 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:06.801705 29614 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.791316 29773 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:06.810649 29713 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:06.891619 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:06.827682 29749 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.909s	user 0.011s	sys 0.001s
I20260709 12:25:06.820780 29692 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:06.892829 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1887) took 2015 ms. Trace:
I20260709 12:25:06.819983 29345 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:06.818647 29662 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:06.894178 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 532) took 1047 ms. Trace:
W20260709 12:25:06.897825 29780 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:25:06.902184 29776 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:06.903168 29563 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.992s	user 0.010s	sys 0.000s
W20260709 12:25:06.903297 29623 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.508s	user 0.000s	sys 0.007s
W20260709 12:25:06.826891 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98455.6us
I20260709 12:25:06.850018 29730 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.827720 29680 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.830700 29683 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:06.938304 29779 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:06.826037 29518 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:06.850332 29527 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:06.793643 29772 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:06.958199 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2640) took 1701 ms. Trace:
I20260709 12:25:06.959923 26756 leader_election.cc:400] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9.
I20260709 12:25:06.961203 26124 raft_consensus.cc:614] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:06.894229 29757 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:06.950431 29511 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.007s	user 0.008s	sys 0.003s
I20260709 12:25:06.975185 26121 rpcz_store.cc:276] 0709 12:25:04.877575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.883019 (+  5444us) service_pool.cc:224] Handling call
0709 12:25:04.883485 (+   466us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:05.333210 (+449725us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:05.704643 (+371433us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.059279 (+354636us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.069536 (+ 10257us) log.cc:844] Serialized 52 byte log entry
0709 12:25:06.069690 (+   154us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:06.101218 (+ 31528us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.126196 (+ 24978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.126272 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.597718 (+471446us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:25:06.892598 (+294880us) raft_consensus.cc:1685] finished
0709 12:25:06.892626 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.892774 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155210,"prepare.queue_time_us":80652,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":131552,"raft.run_cpu_time_us":1969,"raft.run_wall_time_us":507017,"spinlock_wait_cycles":6002432,"thread_start_us":707561,"threads_started":2,"wal-append.queue_time_us":435019}
I20260709 12:25:06.939181 29614 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.002830 29731 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:06.980538 29537 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.075s	user 0.011s	sys 0.003s
I20260709 12:25:07.001420 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:06.972774 29773 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:06.997608 29781 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 12:25:06.984591 29345 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:06.979733 29692 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:06.993147 29613 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.109165 29773 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.112635 29613 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:07.113266 29613 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.115839 26785 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:07.043538 26127 rpcz_store.cc:276] 0709 12:25:05.257163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.257346 (+   183us) service_pool.cc:224] Handling call
0709 12:25:05.257720 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:05.761748 (+504028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.761798 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:05.829871 (+ 68073us) log.cc:844] Serialized 52 byte log entry
0709 12:25:06.330116 (+500245us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:06.330170 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.330261 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.957899 (+627638us) raft_consensus.cc:1685] finished
0709 12:25:06.957929 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.958119 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8624,"raft.queue_time_us":151850,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":955155,"threads_started":2,"wal-append.queue_time_us":455714}
I20260709 12:25:07.049054 26052 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:06.985365 26129 rpcz_store.cc:276] 0709 12:25:05.846551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.850532 (+  3981us) service_pool.cc:224] Handling call
0709 12:25:05.850883 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.529264 (+678381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.830719 (+301455us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540010f4d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:06.894114 (+ 63395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104560384}
I20260709 12:25:06.998549 29794 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.004707 29640 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.052857 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.053548 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1445) took 1112 ms. Trace:
I20260709 12:25:07.121477 26049 rpcz_store.cc:276] 0709 12:25:05.940946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.941681 (+   735us) service_pool.cc:224] Handling call
0709 12:25:05.941961 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:05.968307 (+ 26346us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:05.968347 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:05.968836 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.183270 (+214434us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:06.183330 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.183432 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.053246 (+869814us) raft_consensus.cc:1685] finished
0709 12:25:07.053275 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.053462 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25911,"raft.queue_time_us":44410,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":96,"thread_start_us":214088,"threads_started":1,"wal-append.queue_time_us":791295}
I20260709 12:25:07.006690 29776 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.124339 29776 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:07.019471 29624 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.124967 29776 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:07.125124 29624 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:07.125993 29776 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.126473 29305 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.126617 29624 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.021104 29736 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.127657 29736 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.127125 29305 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.128149 29736 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.130473 29736 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.130973 29736 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.993636 29662 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.133154 29662 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:07.132023 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 130540us
I20260709 12:25:07.119908 29640 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.137036 29640 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:07.105044 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.108796 29731 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.141970 29731 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.043637 26756 leader_election.cc:403] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.115716 29613 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.153000 29613 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.154016 29613 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:07.022080 29623 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.164814 26049 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:07.166087 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.166126 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.167140 26053 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:07.168798 26053 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:07.001842 29563 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:07.177500 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 281169us
I20260709 12:25:07.178412 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:07.184532 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.161381 29784 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.389s	user 0.000s	sys 0.001s
I20260709 12:25:07.189389 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.191071 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.181057 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.196060 26053 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:07.194519 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.197391 26197 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.157049 29751 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.161633 29511 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.162487 29794 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.211455 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.213861 26199 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.213887 26195 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:07.224799 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.225616 26053 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:07.045630 29779 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.189296 29640 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9.
I20260709 12:25:07.159835 29509 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:25:07.127168 29624 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.232097 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:07.244921 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.225452 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 788) took 2646 ms. Trace:
I20260709 12:25:07.239655 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.131402 29736 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.217280 29605 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.248034 26788 rpcz_store.cc:276] 0709 12:25:04.579303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.976705 (+1397402us) service_pool.cc:224] Handling call
0709 12:25:05.977093 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.977411 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:06.291043 (+313632us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.622734 (+331691us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.967037 (+344303us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.224112 (+257075us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.224165 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:07.224214 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.224310 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.225396 (+  1086us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":466729,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":137,"raft.queue_time_us":173642,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":19054,"thread_start_us":1210416,"threads_started":4,"wal-append.queue_time_us":509225}
I20260709 12:25:07.243122 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:07.217667 29731 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.227169 29706 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:07.239885 29792 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 12:25:07.250041 29551 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.287505 26201 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.200734 29762 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.291268 26199 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:07.221383 29511 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.294147 29795 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:07.216914 29751 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:07.308501 29791 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:25:07.308856 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1446) took 1365 ms. Trace:
W20260709 12:25:07.309667 29785 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 12:25:07.310639 29511 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:07.285159 29793 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
I20260709 12:25:07.309281 26051 rpcz_store.cc:276] 0709 12:25:05.943131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.944060 (+   929us) service_pool.cc:224] Handling call
0709 12:25:05.944502 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.002989 (+ 58487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.003021 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.009809 (+  6788us) log.cc:844] Serialized 52 byte log entry
0709 12:25:06.233790 (+223981us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:06.233838 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.404221 (+170383us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:06.406571 (+  2350us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.295979 (+889408us) raft_consensus.cc:1685] finished
0709 12:25:07.296005 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.308792 (+ 12787us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4107,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":1625,"spinlock_wait_cycles":133178112,"thread_start_us":139672,"threads_started":1,"wal-append.queue_time_us":543271}
W20260709 12:25:07.317903 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.294221 29662 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:07.281060 29787 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.436s	user 0.000s	sys 0.001s
I20260709 12:25:07.297310 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.312572 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1449) took 1137 ms. Trace:
I20260709 12:25:07.323809 26045 rpcz_store.cc:276] 0709 12:25:06.175266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.175430 (+   164us) service_pool.cc:224] Handling call
0709 12:25:06.175886 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.176447 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.176491 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.177109 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.759336 (+582227us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:06.759396 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.759567 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.312358 (+552791us) raft_consensus.cc:1685] finished
0709 12:25:07.312379 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.312513 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4447,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9190,"thread_start_us":581805,"threads_started":1,"wal-append.queue_time_us":866461}
I20260709 12:25:07.319512 29603 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.311214 29798 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:07.273761 29804 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:07.222081 29794 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:07.271759 29506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.017s	sys 0.047s
W20260709 12:25:07.331910 26166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.012s	sys 0.029s
I20260709 12:25:07.276441 29345 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:07.347060 26166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.012s	sys 0.030s
I20260709 12:25:07.289904 29551 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.361672 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:07.269363 29773 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.276918 29605 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:07.347460 29506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.018s	sys 0.048s
I20260709 12:25:07.320182 29731 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 12:25:07.371049 26785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 633) took 2801 ms (client timeout 2158 ms). Trace:
W20260709 12:25:07.366984 29783 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 12:25:07.326915 29816 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:25:07.384452 26785 rpcz_store.cc:269] 0709 12:25:04.569516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.935261 (+365745us) service_pool.cc:224] Handling call
0709 12:25:07.370874 (+2435613us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54004ffe54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:07.370982 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4854422528}
I20260709 12:25:07.397516 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.366780 29509 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.366998 29730 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.326638 29662 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.410351 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:07.334483 29603 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.367455 29506 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:07.398720 29518 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.396873 29640 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:07.347484 29804 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.433316 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.436868 29762 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.440824 29662 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.437968 29779 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.462090 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1892) took 1443 ms. Trace:
I20260709 12:25:07.439975 29603 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.412391 29623 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.407073 29812 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:07.443271 29807 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:25:07.448109 26018 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.181s (SENT)
I20260709 12:25:07.477936 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:07.466351 26128 rpcz_store.cc:276] 0709 12:25:06.018728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.065011 (+ 46283us) service_pool.cc:224] Handling call
0709 12:25:06.065369 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.065720 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.065747 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.066247 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.327030 (+260783us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:06.327095 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.327207 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.461839 (+1134632us) raft_consensus.cc:1685] finished
0709 12:25:07.461861 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.462000 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4854,"raft.queue_time_us":753,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12031,"thread_start_us":241532,"threads_started":1,"wal-append.queue_time_us":524594}
I20260709 12:25:07.469480 26201 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:07.442893 26197 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
W20260709 12:25:07.425493 29323 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.434s	user 0.017s	sys 0.000s
I20260709 12:25:07.443519 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.475798 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1893) took 1410 ms. Trace:
W20260709 12:25:07.448554 29789 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 12:25:07.480465 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:07.450736 29790 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.352s	user 0.000s	sys 0.002s
I20260709 12:25:07.440948 29800 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:07.415773 29692 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.448786 29809 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:07.505303 26197 raft_consensus.cc:2376] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8. Re-sending same reply.
I20260709 12:25:07.501948 26123 rpcz_store.cc:276] 0709 12:25:06.065074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.078699 (+ 13625us) service_pool.cc:224] Handling call
0709 12:25:06.100472 (+ 21773us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.106373 (+  5901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.106406 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.133966 (+ 27560us) log.cc:844] Serialized 52 byte log entry
0709 12:25:06.257510 (+123544us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:06.258101 (+   591us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.258651 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.258758 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.475471 (+1216713us) raft_consensus.cc:1685] finished
0709 12:25:07.475499 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.475715 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5577,"prepare.queue_time_us":250,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":58,"raft.queue_time_us":7058,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":6159,"thread_start_us":60719,"threads_started":1,"wal-append.queue_time_us":328983}
W20260709 12:25:07.509404 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.011s
I20260709 12:25:07.417917 29509 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.451536 29730 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:07.509776 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.000s	sys 0.011s
W20260709 12:25:07.513345 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 621739us
I20260709 12:25:07.448550 29751 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.514827 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:07.521708 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.522603 26197 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
W20260709 12:25:07.519964 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 340248us
I20260709 12:25:07.529881 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.453611 29640 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.536300 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.537075 26197 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.539079 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:07.539873 26785 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:07.542567 26047 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:07.546852 26018 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.228s (SENT)
I20260709 12:25:07.547448 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:07.519718 29561 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.516s	user 0.013s	sys 0.002s
W20260709 12:25:07.552208 29766 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:25:07.552588 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.516613 29809 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.519526 29603 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.558208 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:07.558810 26785 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:25:07.534401 29661 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.517s	user 0.012s	sys 0.003s
I20260709 12:25:07.563012 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.535636 29806 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:07.506294 29323 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.562443 26783 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:25:07.564105 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.564606 26197 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:07.577836 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.578491 26197 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:07.567238 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:07.569207 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:07.558065 29749 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.571761 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 798) took 1644 ms. Trace:
I20260709 12:25:07.573055 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.545258 29506 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:07.558619 29669 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.425s	user 0.008s	sys 0.006s
I20260709 12:25:07.569218 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:07.590880 26788 raft_consensus.cc:2376] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9. Re-sending same reply.
I20260709 12:25:07.555398 29740 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.517450 29730 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.580287 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.595132 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.502637 29812 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.582934 26052 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:07.584381 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.560755 29809 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.593226 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.583377 26791 rpcz_store.cc:276] 0709 12:25:05.927679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.078866 (+151187us) service_pool.cc:224] Handling call
0709 12:25:06.079299 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.148429 (+ 69130us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:06.148461 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.148996 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.517547 (+368551us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:06.517624 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.517764 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.571405 (+1053641us) raft_consensus.cc:1685] finished
0709 12:25:07.571424 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.571700 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30195,"raft.queue_time_us":35527,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":368011,"threads_started":1,"wal-append.queue_time_us":642489}
I20260709 12:25:07.581450 29745 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.597594 29740 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.598162 29692 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.586400 29772 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.575572 29661 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:07.602237 26756 connection.cc:256] RPC call timeout handler was delayed by -1.2297823940002672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260709 12:25:07.605522 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:07.632753 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1766823940001814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
I20260709 12:25:07.580188 29345 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.579993 29794 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.562068 29614 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.637156 29614 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:07.598835 29818 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:25:07.593065 29511 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.599221 29810 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:25:07.638002 29511 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:07.639688 29777 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.111s	user 0.002s	sys 0.002s
W20260709 12:25:07.601307 29527 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.007s
I20260709 12:25:07.640583 29511 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.640525 29527 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.641507 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1803) took 1094 ms. Trace:
I20260709 12:25:07.642061 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1898) took 1527 ms. Trace:
I20260709 12:25:07.642830 29527 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.641880 26202 rpcz_store.cc:276] 0709 12:25:06.546609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.637191 (+ 90582us) service_pool.cc:224] Handling call
0709 12:25:06.637510 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.110160 (+472650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.110288 (+   128us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.112812 (+  2524us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.341631 (+228819us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:07.341842 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.341950 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.641304 (+299354us) raft_consensus.cc:1685] finished
0709 12:25:07.641320 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.641448 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100646,"raft.queue_time_us":131205,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":12108,"thread_start_us":612778,"threads_started":2,"wal-append.queue_time_us":389853}
I20260709 12:25:07.603526 29806 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:07.643826 29806 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.644356 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 538) took 1158 ms. Trace:
I20260709 12:25:07.644594 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.644765 26124 rpcz_store.cc:276] 0709 12:25:06.486173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.051990 (+565817us) service_pool.cc:224] Handling call
0709 12:25:07.108327 (+ 56337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.157677 (+ 49350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.157726 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.158418 (+   692us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.590378 (+431960us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:07.590422 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.590508 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.644118 (+ 53610us) raft_consensus.cc:1685] finished
0709 12:25:07.644142 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.644300 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":689,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14191,"thread_start_us":431485,"threads_started":1,"wal-append.queue_time_us":447636}
I20260709 12:25:07.646064 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.603574 29749 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:25:07.647286 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.647826 26199 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.648656 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1797) took 1103 ms. Trace:
I20260709 12:25:07.649034 29504 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:07.649458 29504 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:07.650240 29504 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.651201 29504 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:07.620716 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1791) took 1077 ms. Trace:
I20260709 12:25:07.653210 26198 rpcz_store.cc:276] 0709 12:25:06.542816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.563120 (+ 20304us) service_pool.cc:224] Handling call
0709 12:25:06.578798 (+ 15678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.753684 (+174886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.620452 (+866768us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54002166d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:07.620657 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378482944}
I20260709 12:25:07.654534 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.654667 29323 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.655133 26198 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:07.657063 29661 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.620949 29804 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.510825 29800 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.657626 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.659085 29812 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.659278 26194 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:07.661927 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:07.615913 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:07.631690 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52472.4us
I20260709 12:25:07.614220 29745 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.646121 29806 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.642520 26121 rpcz_store.cc:276] 0709 12:25:06.114917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.055278 (+940361us) service_pool.cc:224] Handling call
0709 12:25:07.091124 (+ 35846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.091405 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.091443 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.159303 (+ 67860us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.475895 (+316592us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:07.475949 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.476066 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.641862 (+165796us) raft_consensus.cc:1685] finished
0709 12:25:07.641884 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.642007 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":262836,"threads_started":1,"wal-append.queue_time_us":366043}
W20260709 12:25:07.633091 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1595823940001537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
I20260709 12:25:07.644038 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.677340 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.677841 26202 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.678004 26785 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:07.678153 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.679144 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:07.679837 26783 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:07.680723 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.681283 26785 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:07.682021 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.682521 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:07.682996 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.663383 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:07.665445 29640 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.585397 29751 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:25:07.668059 29745 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.659345 29800 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.669081 29624 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.696760 29815 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:07.697183 29815 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:07.698012 29815 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.699252 29815 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:07.701313 29829 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:25:07.702196 29736 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.702630 29740 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.669237 26198 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:07.669634 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 539) took 1183 ms. Trace:
I20260709 12:25:07.705585 26198 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:07.704644 26127 rpcz_store.cc:276] 0709 12:25:06.486453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.116811 (+630358us) service_pool.cc:224] Handling call
0709 12:25:07.117317 (+   506us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.117808 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:07.322078 (+204270us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.415841 (+ 93763us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.428662 (+ 12821us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.428919 (+   257us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:07.429288 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.429517 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.429621 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.669421 (+239800us) raft_consensus.cc:1685] finished
0709 12:25:07.669440 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.669579 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":201772,"prepare.run_cpu_time_us":342,"prepare.run_wall_time_us":9454,"raft.queue_time_us":2353,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":23295,"thread_start_us":284199,"threads_started":2,"wal-append.queue_time_us":179750}
I20260709 12:25:07.670477 29509 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:07.671483 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1901) took 1182 ms. Trace:
I20260709 12:25:07.706681 26129 rpcz_store.cc:276] 0709 12:25:06.489528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.119403 (+629875us) service_pool.cc:224] Handling call
0709 12:25:07.119768 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.133851 (+ 14083us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:07.283593 (+149742us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.399914 (+116321us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.400121 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.400169 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:07.400209 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.637184 (+236975us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b18003fb8c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:07.671273 (+ 34089us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b54001f68d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:07.671317 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.671430 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":253812,"prepare.run_cpu_time_us":4552,"prepare.run_wall_time_us":19949,"raft.queue_time_us":8082,"raft.run_wall_time_us":5676,"spinlock_wait_cycles":52267392,"thread_start_us":261145,"threads_started":2,"wal-append.queue_time_us":289522}
W20260709 12:25:07.672624 29757 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.001s
I20260709 12:25:07.708983 29757 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.684065 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.710130 26782 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:25:07.684279 26788 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:07.684803 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.623037 29730 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.711369 29794 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.710953 29809 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.713430 29809 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.686026 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.720911 26784 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:07.686316 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.722013 26786 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:07.687047 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.689538 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:07.690155 29613 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.599599 29669 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.692756 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:07.694656 29561 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.687955 29812 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.669771 29527 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:07.649068 26200 rpcz_store.cc:276] 0709 12:25:06.544797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.544925 (+   128us) service_pool.cc:224] Handling call
0709 12:25:06.584475 (+ 39550us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.891037 (+306562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.891071 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.937219 (+ 46148us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.104227 (+167008us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:07.104302 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.104404 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.648401 (+543997us) raft_consensus.cc:1685] finished
0709 12:25:07.648424 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.648570 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127221,"raft.queue_time_us":139021,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":27248,"spinlock_wait_cycles":38656,"thread_start_us":333843,"threads_started":2,"wal-append.queue_time_us":288459}
I20260709 12:25:07.711488 26790 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:07.685563 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:07.743492 26787 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:07.746598 29732 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:07.747007 29732 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:07.748720 26048 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:07.749382 29519 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.749902 29669 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:07.742806 29603 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:25:07.750936 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1453) took 1137 ms. Trace:
I20260709 12:25:07.751362 26048 rpcz_store.cc:276] 0709 12:25:06.613374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.635930 (+ 22556us) service_pool.cc:224] Handling call
0709 12:25:06.636451 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.749321 (+1112870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.750682 (+  1361us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:07.750873 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557458688}
I20260709 12:25:07.739246 29292 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.752676 29527 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:07.748679 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.753497 26200 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
W20260709 12:25:07.756800 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64127.5us
I20260709 12:25:07.757095 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:07.757275 29757 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.756136 26199 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:07.766343 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.753602 29812 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.739349 29730 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:07.774789 29640 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:07.762336 29376 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:07.778288 29761 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:07.742409 29613 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:07.815382 29613 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:07.753281 29740 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:25:07.784986 29527 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.747833 29732 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.799404 29746 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:07.809523 29506 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:07.821779 29812 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:07.829260 29693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.015s	sys 0.005s
W20260709 12:25:07.829834 29693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.015s	sys 0.005s
I20260709 12:25:07.738132 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:07.877516 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:07.877979 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 139872us
I20260709 12:25:07.875332 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.886883 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:07.887807 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.896634 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:07.897743 26120 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:07.919299 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1897) took 1809 ms. Trace:
I20260709 12:25:07.919834 26120 rpcz_store.cc:276] 0709 12:25:06.109713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.896289 (+1786576us) service_pool.cc:224] Handling call
0709 12:25:07.896996 (+   707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.918906 (+ 21910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.919174 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:07.922187 26122 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:07.786221 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:07.923559 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1454) took 1205 ms. Trace:
I20260709 12:25:07.924126 26047 rpcz_store.cc:276] 0709 12:25:06.718405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.728322 (+  9917us) service_pool.cc:224] Handling call
0709 12:25:07.542327 (+814005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.543098 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.922978 (+379880us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b540008e8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:07.923289 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1871154944}
W20260709 12:25:07.926301 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 140074us
I20260709 12:25:07.926648 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.926918 26125 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:07.929029 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1896) took 1829 ms. Trace:
I20260709 12:25:07.929528 26125 rpcz_store.cc:276] 0709 12:25:06.099805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.925612 (+1825807us) service_pool.cc:224] Handling call
0709 12:25:07.926326 (+   714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.928609 (+  2283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.928933 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:07.932215 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.807926 26199 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:07.940168 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.806905 26202 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:07.790943 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:07.956120 26019 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:07.956799 26019 leader_election.cc:403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.957253 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.958062 29837 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.936033 29504 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:07.961199 29809 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:07.939023 26789 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:07.938618 29571 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:07.941344 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.941996 29730 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.941884 29561 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.943145 29603 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.943542 29527 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.947876 29746 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.949936 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.782312 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:07.981062 26197 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:07.819965 29506 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:07.981464 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.982470 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:07.983762 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:07.984566 29815 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:07.984751 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:07.985287 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.985806 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:07.985672 29815 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:07.962809 29511 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.927934 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:07.968935 29730 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.988864 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:07.961807 29809 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:07.990425 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62858.3us
I20260709 12:25:07.995126 26198 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:07.996369 29519 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:07.996856 29519 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.997404 29519 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.997316 29746 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.001669 26052 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:08.002373 29527 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.005331 29536 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:08.006017 29536 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:08.006846 29527 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:07.995075 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 204151us
I20260709 12:25:08.008852 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:08.009092 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.009991 26049 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:08.011140 29613 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:08.011673 29613 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.005692 29640 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:08.012156 29613 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.015381 26198 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:07.992585 29730 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:08.019055 26046 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:07.996417 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1793) took 1452 ms. Trace:
I20260709 12:25:08.007843 29536 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.020278 29834 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:08.021181 29834 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:08.021306 29536 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.984495 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.012415 29511 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.022606 29834 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.015233 29603 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:25:08.026391 26051 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:08.012670 29812 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:08.029301 29305 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:08.030139 26049 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:08.030795 29551 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:08.020509 26195 rpcz_store.cc:276] 0709 12:25:06.543445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.543569 (+   124us) service_pool.cc:224] Handling call
0709 12:25:06.592395 (+ 48826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.214561 (+622166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.995719 (+781158us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:07.996196 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1428465408}
I20260709 12:25:08.014487 26200 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:07.993944 29809 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.037941 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.039000 26051 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:08.041890 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2647) took 1478 ms. Trace:
I20260709 12:25:08.042227 26199 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:08.045419 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:08.052896 26202 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:08.042379 26050 rpcz_store.cc:276] 0709 12:25:06.563163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.584947 (+ 21784us) service_pool.cc:224] Handling call
0709 12:25:07.913155 (+1328208us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:25:07.913614 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.914487 (+   873us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:07.914534 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.915585 (+  1051us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.921340 (+  5755us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:07.921391 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.921513 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.036461 (+114948us) raft_consensus.cc:1685] finished
0709 12:25:08.036478 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.041667 (+  5189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28773,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":324,"spinlock_wait_cycles":31469312,"thread_start_us":5177,"threads_started":1,"wal-append.queue_time_us":3743}
I20260709 12:25:08.068619 29604 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:08.072760 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:08.073289 29796 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.763s	user 0.003s	sys 0.000s
I20260709 12:25:08.073731 26052 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
W20260709 12:25:08.078896 29805 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.642s	user 0.000s	sys 0.001s
I20260709 12:25:08.081219 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1451) took 1468 ms. Trace:
I20260709 12:25:08.081715 26054 rpcz_store.cc:276] 0709 12:25:06.612885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.621323 (+  8438us) service_pool.cc:224] Handling call
0709 12:25:06.621649 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.625150 (+  3501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.625185 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.625707 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.854364 (+228657us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:06.854402 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.854470 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.072416 (+1217946us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:08.080710 (+  8294us) raft_consensus.cc:1685] finished
0709 12:25:08.080731 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.081032 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2928,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":13127936,"thread_start_us":228273,"threads_started":1,"wal-append.queue_time_us":684313}
I20260709 12:25:08.065850 29815 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:08.112670 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45780.9us
I20260709 12:25:08.121021 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:08.118750 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2651) took 1810 ms (client timeout 1687 ms). Trace:
W20260709 12:25:08.122797 26122 rpcz_store.cc:269] 0709 12:25:06.299496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.395025 (+ 95529us) service_pool.cc:224] Handling call
0709 12:25:08.101221 (+1706196us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001d15d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:08.110010 (+  8789us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3167717248}
I20260709 12:25:08.124536 29802 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:08.121193 29746 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.127257 29837 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:08.130484 29834 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:08.132970 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.134969 26125 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:08.137061 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:08.137429 29746 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:08.138388 29746 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:08.138944 29511 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:08.140753 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.142104 26120 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 6: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:08.143390 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.144661 29571 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:08.145251 29571 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.145650 29571 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:08.146739 26127 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:08.148736 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.149802 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.150997 26122 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:08.147915 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:08.152048 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.152153 26129 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:08.153143 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.153448 29755 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.154748 29769 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:08.155942 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.156163 29769 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:08.156697 29769 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:08.160828 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.161970 26121 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:08.162166 29824 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.728s	user 0.001s	sys 0.001s
I20260709 12:25:08.144569 29730 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.166489 29511 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:08.167256 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 640) took 2908 ms. Trace:
I20260709 12:25:08.147699 29506 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.167821 29511 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:08.167930 26789 rpcz_store.cc:276] 0709 12:25:05.257604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.070827 (+813223us) service_pool.cc:224] Handling call
0709 12:25:06.071271 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.961983 (+1890712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.163334 (+201351us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b5400433fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:08.166419 (+  3085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4293302528}
I20260709 12:25:08.154134 29755 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.167054 26053 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:08.172693 29815 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.173954 29746 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.237715 29506 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:08.246021 26052 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 10. Ops: []
I20260709 12:25:08.265038 29603 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:08.266347 29603 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:08.267423 29755 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.268618 29519 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.244556 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:08.247166 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71430.7us
I20260709 12:25:08.267167 29603 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:08.285826 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 162059us
I20260709 12:25:08.266397 29815 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.293092 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.294709 29794 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:08.295142 29815 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.295243 29794 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.296222 29769 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.298064 29769 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.295912 29794 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:08.298669 29519 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.286140 29809 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.301671 26198 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:08.306896 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:08.289763 29527 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:08.308517 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.314400 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:08.315392 26125 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:08.310613 29527 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:08.314471 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:08.318054 29527 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:08.320770 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:08.321120 29730 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.323102 29707 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.324069 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1805) took 1159 ms. Trace:
W20260709 12:25:08.338006 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.340420 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.337316 26121 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:08.312467 29809 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:08.350667 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60548.1us
I20260709 12:25:08.312992 29536 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.355192 29815 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:08.360303 29815 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:08.361122 29732 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.361510 29815 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:08.367928 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.369156 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.370648 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:08.314585 29511 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:08.378724 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.379930 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:08.337320 26196 rpcz_store.cc:276] 0709 12:25:07.164076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.187325 (+ 23249us) service_pool.cc:224] Handling call
0709 12:25:07.202931 (+ 15606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.239073 (+ 36142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.239120 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.268836 (+ 29716us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.367763 (+ 98927us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:07.367821 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.367913 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.161152 (+793239us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:08.302113 (+140961us) raft_consensus.cc:1685] finished
0709 12:25:08.302144 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.323858 (+ 21714us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24446,"raft.queue_time_us":81668,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":25358,"spinlock_wait_cycles":27746688,"thread_start_us":73410,"threads_started":1,"wal-append.queue_time_us":128738}
I20260709 12:25:08.381464 26784 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:08.384091 29603 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.304281 29613 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.386824 29613 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:08.387286 29571 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.389912 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.350999 29527 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.392885 29605 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:08.396483 29536 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.435204 29519 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.436029 29519 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.401413 29755 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.437845 29755 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.439569 29519 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.402212 29746 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:08.443799 29746 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.446954 29746 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:08.419530 29808 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.108s	user 0.000s	sys 0.011s
I20260709 12:25:08.448681 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.449569 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.481817 26125 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:08.323993 29707 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.451519 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:08.451756 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81327.3us
I20260709 12:25:08.454866 29563 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:08.456574 26124 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:08.456724 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.459450 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.459684 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:08.413857 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.463480 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.465116 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:08.466048 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.467461 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51750.6us
I20260709 12:25:08.454866 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:08.472016 29537 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:08.472841 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.450379 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.435886 29536 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:08.495975 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 175374us
I20260709 12:25:08.496654 26122 raft_consensus.cc:2410] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:25:08.503346 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:08.504022 26129 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.511544 26049 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:08.515841 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.521332 26050 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:08.485641 29755 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.533053 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.522333 26048 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:08.533952 26196 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.546341 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:08.549849 26127 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:08.554477 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:08.515424 29613 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:08.552150 29305 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.016s	sys 0.051s
I20260709 12:25:08.569690 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.569303 26120 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:08.581578 26047 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:08.571086 29305 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.017s	sys 0.051s
W20260709 12:25:08.595532 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92832.6us
W20260709 12:25:08.595789 25952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 189) took 23480 ms (client timeout 15000 ms). Trace:
I20260709 12:25:08.601248 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:08.602941 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.602355 25949 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:25:08.604496 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 256477us
W20260709 12:25:08.605008 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1836065159995997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2316s for a response
W20260709 12:25:08.606418 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.607121 29537 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.607180 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:08.608160 26197 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:08.607959 29537 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.608999 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:08.609592 29852 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.003s	sys 0.000s
I20260709 12:25:08.549995 29536 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:25:08.608865 29730 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:08.619052 26093 leader_election.cc:400] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8.
I20260709 12:25:08.619527 26093 leader_election.cc:403] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:08.619835 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.620344 25949 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 5 to 8, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 8 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:08.621109 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.621953 26120 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:08.622359 29809 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:08.622785 29809 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.623003 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.622618 25949 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:08.623262 29605 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:08.623713 29605 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.623039 29613 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:25:08.610816 29746 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:25:08.623176 29809 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:08.631520 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.624120 25949 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.610531 29537 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:08.611158 29779 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.641727 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 185959us
I20260709 12:25:08.624059 29605 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:08.648519 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.590442 29755 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 12:25:08.596290 25952 rpcz_store.cc:269] 0709 12:24:45.114929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.115222 (+   293us) service_pool.cc:224] Handling call
0709 12:25:08.595498 (+23480276us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:59.256025 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:59.455028 (+199003us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:59.465640 (+ 10612us) write_op.cc:435] Acquired schema lock
0709 12:24:59.467792 (+  2152us) tablet.cc:662] Decoding operations
0709 12:24:59.817626 (+349834us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:59.854117 (+ 36491us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:59.856062 (+  1945us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:25:00.209011 (+352949us) tablet.cc:701] Row locks acquired
0709 12:25:00.209070 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:25:00.209296 (+   226us) write_op.cc:270] Start()
0709 12:25:00.209459 (+   163us) write_op.cc:276] Timestamp: P: 1783599900209194 usec, L: 0
0709 12:25:00.209482 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:25:00.223030 (+ 13548us) log.cc:844] Serialized 3727 byte log entry
0709 12:25:03.233998 (+3010968us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b
0709 12:25:03.573612 (+339614us) op_driver.cc:464] REPLICATION: finished
0709 12:25:03.673447 (+ 99835us) write_op.cc:301] APPLY: starting
0709 12:25:03.673558 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:03.673943 (+   385us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:03.673989 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:03.675230 (+  1241us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:03.675586 (+   356us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:25:03.675639 (+    53us) write_op.cc:312] APPLY: finished
0709 12:25:03.676051 (+   412us) log.cc:844] Serialized 71 byte log entry
0709 12:25:03.678000 (+  1949us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:03.678142 (+   142us) write_op.cc:454] Released schema lock
0709 12:25:03.678213 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:03.734111 (+ 55898us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:03.735607 (+  1496us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2993928,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":99446,"apply.run_cpu_time_us":3575,"apply.run_wall_time_us":28072,"num_ops":9,"prepare.queue_time_us":231220,"prepare.run_cpu_time_us":58498,"prepare.run_wall_time_us":4209757,"raft.queue_time_us":145707,"raft.run_cpu_time_us":5704,"raft.run_wall_time_us":54459,"replication_time_us":3363827,"spinlock_wait_cycles":160786176,"thread_start_us":3467521,"threads_started":4,"wal-append.queue_time_us":3201749,"wal-append.run_cpu_time_us":20270,"wal-append.run_wall_time_us":1266479}]]}
W20260709 12:25:08.647102 29834 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.516s	user 0.014s	sys 0.000s
I20260709 12:25:08.668009 29345 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.668440 29506 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:08.668782 29345 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.669131 29506 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.671329 29506 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:08.676054 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:08.671533 25949 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:08.681046 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.666791 29537 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8.
I20260709 12:25:08.663658 29571 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:08.703554 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.708292 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53242.5us
I20260709 12:25:08.671363 29345 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:08.734746 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.690054 29707 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:08.700109 29732 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:08.745460 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:08.736598 29776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.468s	user 0.045s	sys 0.051s
W20260709 12:25:08.757556 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 149657us
I20260709 12:25:08.761385 26199 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:08.765605 26124 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:25:08.758579 29776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.629s	user 0.046s	sys 0.052s
W20260709 12:25:08.779095 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 133161us
I20260709 12:25:08.752323 25949 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 5, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 5 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:08.800465 29769 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.502s	user 0.002s	sys 0.012s
I20260709 12:25:08.808483 26783 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:25:08.801473 29773 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:08.746121 29779 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:08.823439 29859 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:25:08.822480 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:08.779011 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:08.840845 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53708.9us
I20260709 12:25:08.838476 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1795) took 2239 ms. Trace:
I20260709 12:25:08.821601 29537 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:25:08.797053 26127 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:25:08.818933 29860 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:08.819363 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:08.875159 26785 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:08.858233 26790 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:08.858180 26196 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
W20260709 12:25:08.798545 29853 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:25:08.817823 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:08.861044 29794 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.002s
W20260709 12:25:08.833457 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.819404 26129 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:08.808578 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:08.775563 29746 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:08.791774 29345 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:08.819190 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:08.903457 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81156.5us
W20260709 12:25:08.905167 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 132237us
I20260709 12:25:08.875914 26203 rpcz_store.cc:276] 0709 12:25:06.544106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.547262 (+  3156us) service_pool.cc:224] Handling call
0709 12:25:06.585805 (+ 38543us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.586222 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.586261 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.586904 (+   643us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.029069 (+442165us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:07.253943 (+224874us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.270442 (+ 16499us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.317207 (+ 46765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.317316 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.418572 (+1101256us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:08.769159 (+350587us) raft_consensus.cc:1685] finished
0709 12:25:08.769194 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.783124 (+ 13930us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69861,"prepare.queue_time_us":164048,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":7568,"raft.queue_time_us":291439,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":53390,"spinlock_wait_cycles":11861888,"thread_start_us":526213,"threads_started":2,"wal-append.queue_time_us":693322}
I20260709 12:25:08.897604 26787 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.906729 26047 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:08.908748 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.894852 25949 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 6 to 8, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 8 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:08.895018 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:08.888036 29537 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:08.839208 29506 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:25:08.933802 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:08.901580 29779 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:08.894577 29614 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.939703 26124 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:08.942036 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 272041us
I20260709 12:25:08.949071 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:08.956646 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.894209 29746 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.964790 26783 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:08.895147 29794 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.933584 25949 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 4, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 4 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.934541 29732 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:08.964478 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:08.977016 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.945279 29776 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.980363 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.983304 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.938199 29345 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:08.987145 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:08.989480 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:08.993155 26122 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:08.952874 29537 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:08.998096 26788 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:08.976701 29305 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.009672 26786 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.976869 29732 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:09.013731 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:09.015900 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:08.966226 29614 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:09.023396 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78783.6us
I20260709 12:25:09.024513 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:09.024822 29614 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:09.025437 29305 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.026010 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.027423 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:09.028488 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48087.4us
W20260709 12:25:09.029409 29856 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:09.030637 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:09.033454 29731 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:09.034029 29731 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:09.034262 29773 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:09.034816 29773 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:09.035224 29731 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:08.986665 29776 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.040694 29776 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:09.040982 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2670) took 1058 ms. Trace:
I20260709 12:25:09.027959 29305 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.041386 29746 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:09.041549 26045 rpcz_store.cc:276] 0709 12:25:07.982148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.986377 (+  4229us) service_pool.cc:224] Handling call
0709 12:25:07.986910 (+   533us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.987565 (+   655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.987600 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:08.023721 (+ 36121us) log.cc:844] Serialized 52 byte log entry
0709 12:25:08.551159 (+527438us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:08.551209 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.551277 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.040685 (+489408us) raft_consensus.cc:1685] finished
0709 12:25:09.040718 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.040906 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1052,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":288,"thread_start_us":450688,"threads_started":1,"wal-append.queue_time_us":899216}
I20260709 12:25:09.043519 29746 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.045723 29877 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:09.046732 29877 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.047489 29877 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:09.015784 29732 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.053303 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:08.981464 29809 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.066337 26787 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:09.070443 29769 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:09.078665 26786 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:09.088918 29794 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.089501 26784 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:09.091132 29345 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:09.048168 29877 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.092671 29536 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:09.093169 29536 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:09.094331 29877 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.094185 29536 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.095386 29536 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:09.096596 29527 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.003s
I20260709 12:25:09.044142 29746 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:09.097644 26052 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:09.097936 29746 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.101603 29605 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:09.103456 29861 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.435s	user 0.003s	sys 0.000s
I20260709 12:25:09.035833 29773 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.104432 29527 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:09.105095 29527 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.106695 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:09.107205 29527 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.108022 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:09.109226 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:09.109881 26783 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.110504 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.111649 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.112191 26123 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:09.112823 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.113122 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.113575 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.113933 26123 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:09.114339 26124 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:09.114012 29614 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:09.115598 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.118571 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.118387 29613 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:09.119266 26120 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:09.121390 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.120282 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.130415 26121 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.135107 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:09.142254 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:09.143301 26046 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:09.146335 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:09.145499 29305 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:09.154410 26128 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:09.154924 29305 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:09.155689 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.156176 29305 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:09.158569 29867 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:25:09.063320 25949 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 5, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
W20260709 12:25:09.156256 29862 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:25:09.141548 29511 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:25:09.162062 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1818) took 1023 ms. Trace:
I20260709 12:25:09.162473 26201 rpcz_store.cc:276] 0709 12:25:08.138880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.139188 (+   308us) service_pool.cc:224] Handling call
0709 12:25:08.139682 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.140405 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:08.177734 (+ 37329us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.510319 (+332585us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:08.542113 (+ 31794us) log.cc:844] Serialized 26 byte log entry
0709 12:25:08.542312 (+   199us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:08.542361 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.542456 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.161845 (+619389us) raft_consensus.cc:1685] finished
0709 12:25:09.161865 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.161981 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24251,"prepare.queue_time_us":131811,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":8664,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":10025,"thread_start_us":236336,"threads_started":1,"wal-append.queue_time_us":660579}
I20260709 12:25:09.144690 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.170651 26124 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:09.171480 26123 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:09.166047 25949 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:09.185053 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.154261 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:09.191459 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.194619 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.196323 26123 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:09.139516 29605 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:09.173575 29603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.022s	sys 0.031s
I20260709 12:25:09.233855 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.166199 29809 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:09.241055 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.244849 29794 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.245371 29837 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:09.245839 29837 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:09.246908 29837 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.249440 29561 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.250115 29561 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:09.251217 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 97385.6us
I20260709 12:25:09.252110 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.254515 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.255368 29834 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:09.255520 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 65143.7us
I20260709 12:25:09.255798 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.258157 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.260721 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:09.244880 29864 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.646s	user 0.000s	sys 0.003s
I20260709 12:25:09.261744 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.272140 26128 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:09.264950 29561 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.265158 29732 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:09.236498 29603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.023s	sys 0.032s
I20260709 12:25:09.291006 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.272275 26120 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:09.289767 29809 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:09.294813 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.294219 26783 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:09.248090 29837 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.297837 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.292377 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.307596 26199 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:09.308452 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.308831 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:09.309242 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:09.309926 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.310103 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:09.310686 26198 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:09.312667 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 805) took 1043 ms. Trace:
I20260709 12:25:09.312803 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.313705 26194 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:09.314045 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.308596 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.315032 26203 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:09.314967 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.316833 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.313265 26790 rpcz_store.cc:276] 0709 12:25:08.268901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.285567 (+ 16666us) service_pool.cc:224] Handling call
0709 12:25:09.312454 (+1026887us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540056aed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:09.312593 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875541376}
I20260709 12:25:09.321285 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:09.322121 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.322779 26124 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:09.323666 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.324148 26124 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:09.332808 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 77109us
I20260709 12:25:09.333485 26121 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:09.334645 29706 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.335860 26198 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
W20260709 12:25:09.337553 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45247.2us
I20260709 12:25:09.340652 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.290513 29794 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.349604 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:09.336324 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.342442 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.344765 26196 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:09.350684 26128 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.350848 26121 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:09.353381 26194 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:09.354668 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.345166 25949 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:09.357332 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.359315 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.361323 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.362061 25949 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 7, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:09.359054 29732 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:09.363981 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43766.8us
I20260709 12:25:09.369204 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.371136 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.374919 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.294833 29809 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.379794 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.380963 26121 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:09.382503 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:09.383183 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 352608us
I20260709 12:25:09.385354 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.386714 26783 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:09.391060 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.364073 29779 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.394272 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:09.395922 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.368062 29706 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:09.403585 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.392674 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:09.407738 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.354638 29605 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 12:25:09.413300 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52692.5us
I20260709 12:25:09.409485 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1895) took 3089 ms. Trace:
I20260709 12:25:09.383193 25949 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 6 to 7, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 7 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:25:09.416208 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.418583 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.379040 29730 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:09.432065 26784 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:25:09.430177 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:09.436504 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:09.432490 26784 pending_rounds.cc:85] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 6
W20260709 12:25:09.439340 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.381913 29776 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:25:09.445855 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 808) took 1142 ms. Trace:
I20260709 12:25:09.438081 26784 pending_rounds.cc:107] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:25:09.448203 26784 raft_consensus.cc:2889] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:09.448534 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:09.431303 29794 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.456269 26783 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:09.461539 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 77948us
I20260709 12:25:09.434124 29732 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.414243 26126 rpcz_store.cc:276] 0709 12:25:06.095272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.511200 (+2415928us) service_pool.cc:224] Handling call
0709 12:25:08.556070 (+ 44870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.574784 (+ 18714us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:08.768662 (+193878us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.172272 (+403610us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.172317 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:09.172355 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.172407 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.184626 (+ 12219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":459469,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":115,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":56102,"thread_start_us":514800,"threads_started":2}
I20260709 12:25:09.471630 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:09.447829 29613 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:09.482903 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.465056 29834 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:09.448019 26782 rpcz_store.cc:276] 0709 12:25:08.270087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.290981 (+ 20894us) service_pool.cc:224] Handling call
0709 12:25:08.293339 (+  2358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.293927 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.293961 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:08.294909 (+   948us) log.cc:844] Serialized 26 byte log entry
0709 12:25:08.500895 (+205986us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:08.500957 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.501052 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.243999 (+742947us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:09.407243 (+163244us) raft_consensus.cc:1685] finished
0709 12:25:09.407278 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.412148 (+  4870us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":252,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":273,"spinlock_wait_cycles":37837056,"thread_start_us":204728,"threads_started":1,"wal-append.queue_time_us":303041}
I20260709 12:25:09.448287 29605 consensus_queue.cc:1243] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:09.500190 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.480041 29506 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.402403 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:09.446892 29614 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:25:09.486812 29877 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:09.485958 29731 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:09.508770 26120 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:09.448765 29730 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.508999 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:09.512993 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.517506 26126 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.488091 29603 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.488559 29732 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:09.522550 29815 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.526044 26052 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
W20260709 12:25:09.554165 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105722us
W20260709 12:25:09.560015 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.491945 29345 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:09.515022 29693 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.570045 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:09.579118 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.571547 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.580701 26196 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:09.559525 29561 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:09.574679 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.585246 26051 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:09.515390 29731 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:09.525658 29794 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:09.579958 26198 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:09.588152 29731 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.590765 29779 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:09.591270 29779 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:09.592437 29794 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.592883 29794 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.560385 29877 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:09.585654 26203 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:09.587324 29561 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.585609 26782 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:09.588092 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.513325 29746 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.572814 29706 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.516137 29506 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:09.604441 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.007s	sys 0.015s
W20260709 12:25:09.606441 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 106302us
W20260709 12:25:09.613198 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.007s	sys 0.015s
I20260709 12:25:09.593489 29834 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:09.614050 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:09.618386 26120 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
W20260709 12:25:09.599242 29755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.031s	sys 0.069s
W20260709 12:25:09.621105 29880 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:09.616225 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:09.617056 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 104171us
I20260709 12:25:09.625424 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:09.621400 29755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.032s	sys 0.069s
I20260709 12:25:09.627774 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:09.628700 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:09.572320 29603 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.614068 26020 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:09.588385 29571 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:09.621987 26129 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:09.627266 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:09.577922 29345 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.631340 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:09.629504 26051 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:09.615461 29563 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.615195 29809 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:09.639505 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:09.640625 26049 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.641876 26198 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:09.642935 29834 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.643244 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.644136 26196 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:09.588320 29875 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.149s
W20260709 12:25:09.645933 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57947.7us
W20260709 12:25:09.648468 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 621078us
I20260709 12:25:09.646759 26045 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:09.657055 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.695008 26128 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:09.693559 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:09.692394 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.755115 26126 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:09.760368 26124 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:09.639425 29755 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:25:09.764223 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:09.727890 26049 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
W20260709 12:25:09.668100 26787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.020s	sys 0.025s
W20260709 12:25:09.777633 26787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.021s	sys 0.026s
W20260709 12:25:09.781899 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.681288 29746 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:09.649407 29837 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:09.786513 26198 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:09.787508 26125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.007s	sys 0.014s
I20260709 12:25:09.727511 29707 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:09.791225 26125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.007s	sys 0.014s
I20260709 12:25:09.681746 29563 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:09.655814 29706 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.700132 29345 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.717013 29730 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:09.843255 29730 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.681049 29603 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:09.764803 29773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.009s	sys 0.015s
I20260709 12:25:09.688304 29561 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:09.836728 26128 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:09.851581 26128 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:09.693794 29506 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:09.837189 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 180204us
W20260709 12:25:09.760331 29305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.013s	sys 0.006s
I20260709 12:25:09.646610 29571 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.839798 29706 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.842664 29345 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.786028 29511 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.844789 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:09.867163 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.823405 29707 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:09.682143 29876 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.164s
W20260709 12:25:09.841130 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 147665us
I20260709 12:25:09.867602 29563 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.891006 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.875936 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:09.892206 29693 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.893178 29563 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:09.893589 29834 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:09.884542 29706 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.894135 29834 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:09.840912 29882 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:09.898519 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:09.681592 29877 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.658571 29732 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:09.868126 29305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.013s	sys 0.006s
I20260709 12:25:09.843751 29730 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.836169 29837 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.899252 29794 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:09.901374 26195 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:09.689111 29779 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:09.851651 29773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.009s	sys 0.015s
I20260709 12:25:09.923300 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.887506 26198 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
W20260709 12:25:09.904302 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.843741 29731 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.913955 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:09.910459 26128 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:09.943794 26198 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:09.948350 26203 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:09.917826 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 273216us
I20260709 12:25:09.917471 29755 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.954075 26128 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:09.955632 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.016s
I20260709 12:25:09.955681 26198 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:09.956671 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 473805us
W20260709 12:25:09.956053 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.009s	sys 0.016s
W20260709 12:25:09.964999 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 553416us
I20260709 12:25:09.930615 29882 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.944206 29561 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.943683 29876 consensus_queue.cc:1243] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:09.975524 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 182921us
W20260709 12:25:09.990634 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.992676 26197 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:25:10.002854 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 104412us
W20260709 12:25:10.003809 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62429.5us
I20260709 12:25:10.004846 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:10.015847 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.019784 26128 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:10.019979 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.979851 29561 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:10.022555 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.028052 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.024857 26198 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:10.035202 26121 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:10.036361 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.043787 29882 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:10.044595 29891 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:25:10.046213 29693 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:10.047735 29893 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:10.048918 26121 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:10.051286 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.052912 29571 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.052986 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.054394 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:10.054622 29571 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:10.055483 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:10.056936 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:10.064960 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60247.9us
I20260709 12:25:10.023245 29776 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:10.030001 29815 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:10.032287 29561 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.007067 29706 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.106241 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.078723 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.107075 26195 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
I20260709 12:25:10.079147 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:09.991531 29707 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.056174 29571 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.127843 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:10.128177 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:10.115900 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.056555 29882 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:10.104470 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:10.113400 26199 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:10.134852 26052 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:10.138087 26049 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:25:10.137930 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68196.6us
I20260709 12:25:10.065095 29876 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:25:10.108328 29563 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:10.154112 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50145.4us
I20260709 12:25:10.145380 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:10.155604 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:10.057475 29693 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:25:10.189942 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:10.202203 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 228845us
I20260709 12:25:10.197669 26053 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:10.200160 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:10.213649 26783 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:10.101138 29776 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:10.224918 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 231834us
I20260709 12:25:10.164108 29773 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:10.231087 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85838.9us
I20260709 12:25:10.199553 29913 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:10.209707 29892 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:10.232950 26198 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
W20260709 12:25:10.233763 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.193501 29911 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:10.149771 29571 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.201514 29815 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:10.207500 29527 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:10.256145 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:10.146996 29886 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:25:10.211802 29345 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:10.228067 29794 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:10.291581 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43520.6us
W20260709 12:25:10.284596 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 92475.1us
I20260709 12:25:10.276645 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:10.279268 29894 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.252s	user 0.001s	sys 0.001s
I20260709 12:25:10.279450 29923 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:10.235112 29913 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.280529 29882 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:10.271754 29815 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.282321 29693 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:10.234163 29773 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.233587 29563 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:10.290211 29908 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:10.269116 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.310848 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 930) took 1204 ms. Trace:
W20260709 12:25:10.280153 29915 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 12:25:10.310320 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:10.315550 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50607.6us
I20260709 12:25:10.315461 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:10.248210 29898 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:25:10.336299 26787 rpcz_store.cc:276] 0709 12:25:09.105940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.107885 (+  1945us) service_pool.cc:224] Handling call
0709 12:25:09.108472 (+   587us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:09.109721 (+  1249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.109793 (+    72us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:09.122020 (+ 12227us) log.cc:844] Serialized 52 byte log entry
0709 12:25:09.790427 (+668407us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:10.106065 (+315638us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.124744 (+ 18679us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.310503 (+185759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.310587 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.310633 (+    46us) raft_consensus.cc:1685] finished
0709 12:25:10.310649 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.310791 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":300,"prepare.queue_time_us":731187,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":328188,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":306,"thread_start_us":1097392,"threads_started":3,"wal-append.queue_time_us":728804}
I20260709 12:25:10.314622 29527 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:10.309491 29882 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:10.309988 29913 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.312536 29815 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.323117 29773 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:10.304888 29925 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:25:10.313583 29693 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:10.396265 29922 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:25:10.349186 29902 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:25:10.294081 29911 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:10.398936 29919 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:25:10.398581 29693 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.419252 29693 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.417145 29932 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:10.420938 29932 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:10.422207 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 655) took 1714 ms. Trace:
I20260709 12:25:10.421873 29932 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.422596 26791 rpcz_store.cc:276] 0709 12:25:08.707828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.769946 (+ 62118us) service_pool.cc:224] Handling call
0709 12:25:08.798638 (+ 28692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.799264 (+   626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.799303 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.275644 (+476341us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b2c000471f8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:25:09.276144 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.765511 (+489367us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:09.765550 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.765626 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.421806 (+656180us) raft_consensus.cc:1685] finished
0709 12:25:10.421824 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.422036 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4268,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":62153472,"thread_start_us":488910,"threads_started":1,"wal-append.queue_time_us":680306}
I20260709 12:25:10.425297 26128 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:10.425724 26128 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:10.426468 26128 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.427573 26128 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.428833 29910 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:10.406741 29925 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:10.409080 29897 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:10.310313 29923 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.409832 29911 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:10.409837 29906 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:25:10.411602 26203 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:10.340355 29908 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.393261 29882 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:10.439824 29897 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.440727 29923 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:10.414407 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1927) took 1152 ms. Trace:
I20260709 12:25:10.414883 29571 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.442193 29897 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.441776 26126 rpcz_store.cc:276] 0709 12:25:09.262373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.768603 (+506230us) service_pool.cc:224] Handling call
0709 12:25:09.768930 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.769297 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.769323 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.793817 (+ 24494us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.029331 (+235514us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:10.029367 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.029423 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.414151 (+384728us) raft_consensus.cc:1685] finished
0709 12:25:10.414178 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.414337 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2417,"raft.queue_time_us":116,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":37717,"thread_start_us":164850,"threads_started":1,"wal-append.queue_time_us":322794}
I20260709 12:25:10.444368 29925 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.448372 29908 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:10.448406 29815 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.450910 29730 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.451776 29815 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:10.341903 29563 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.453963 29563 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:10.389101 29527 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:10.416343 29345 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:10.454645 29527 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:10.455001 29921 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.174s	user 0.004s	sys 0.000s
I20260709 12:25:10.455261 29345 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:10.455698 29345 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:10.456861 29345 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.458143 29345 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:10.416952 29899 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 12:25:10.461921 29912 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:25:10.463156 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1919) took 1203 ms. Trace:
I20260709 12:25:10.417106 29776 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.417725 29706 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:10.441283 29731 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:10.463521 26129 rpcz_store.cc:276] 0709 12:25:09.259254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.732634 (+473380us) service_pool.cc:224] Handling call
0709 12:25:09.732979 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.733392 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.733423 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.733900 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.076342 (+342442us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:10.076397 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.076489 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.461251 (+384762us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:10.462856 (+  1605us) raft_consensus.cc:1685] finished
0709 12:25:10.462881 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.463091 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":958,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":22108,"spinlock_wait_cycles":7110400,"thread_start_us":314014,"threads_started":1,"wal-append.queue_time_us":486221}
W20260709 12:25:10.465687 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 124393us
I20260709 12:25:10.466430 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:10.478839 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.487340 26128 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:10.435911 29910 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:10.488973 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 561) took 1012 ms. Trace:
W20260709 12:25:10.489957 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 359797us
I20260709 12:25:10.492179 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1926) took 1230 ms. Trace:
W20260709 12:25:10.490271 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 434814us
W20260709 12:25:10.490660 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 180393us
W20260709 12:25:10.490861 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 272082us
I20260709 12:25:10.489493 26128 rpcz_store.cc:276] 0709 12:25:09.476409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.021232 (+544823us) service_pool.cc:224] Handling call
0709 12:25:10.058048 (+ 36816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.425190 (+367142us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54002a56a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:25:10.488079 (+ 62889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.488880 (+   801us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517094784}
I20260709 12:25:10.496253 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.497056 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:10.467836 29904 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
W20260709 12:25:10.501360 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.700s (SENT) [suppressed 27 similar messages]
W20260709 12:25:10.506181 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.495822 29537 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 12:25:10.508509 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 193140us
I20260709 12:25:10.508431 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:10.483214 29511 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:10.511438 29511 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.499752 29925 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.512326 29938 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:10.506467 29561 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.491827 29910 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.514618 29561 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:10.502799 29815 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:25:10.518783 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:10.520745 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.511619 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:10.533231 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.492748 26124 rpcz_store.cc:276] 0709 12:25:09.262158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.784178 (+522020us) service_pool.cc:224] Handling call
0709 12:25:09.794347 (+ 10169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.840471 (+ 46124us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:09.903772 (+ 63301us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.491457 (+587685us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.491736 (+   279us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:10.491814 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.491920 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.492092 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40655,"prepare.queue_time_us":69254,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":42155,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":32591,"thread_start_us":557375,"threads_started":2,"wal-append.queue_time_us":483714}
I20260709 12:25:10.520709 29938 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:10.520080 29776 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:10.543227 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.540674 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:10.524434 29732 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:10.560825 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1917) took 1313 ms. Trace:
I20260709 12:25:10.561215 29732 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.561556 29731 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.562199 29731 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.524312 29773 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.561483 26122 rpcz_store.cc:276] 0709 12:25:09.233246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.447717 (+214471us) service_pool.cc:224] Handling call
0709 12:25:09.477926 (+ 30209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.509700 (+ 31774us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:09.944989 (+435289us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.452847 (+507858us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b1800130900. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db
0709 12:25:10.454238 (+  1391us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:10.546144 (+ 91906us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.546309 (+   165us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:10.546358 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.546445 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.546609 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4537,"prepare.queue_time_us":212689,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":30,"raft.queue_time_us":160529,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":26356,"spinlock_wait_cycles":35832448,"thread_start_us":925223,"threads_started":3,"wal-append.queue_time_us":333085}
I20260709 12:25:10.566505 29773 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.547055 29943 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:10.567246 29943 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:10.539041 29809 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:10.543286 29776 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.566713 29815 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:10.568290 29943 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.569078 29776 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.569348 29815 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:10.569747 29943 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.570478 29815 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.546047 29730 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.572590 29730 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.574879 29730 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.547070 29935 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:10.576855 29889 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:25:10.577329 29889 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:25:10.580494 29896 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.628s	user 0.003s	sys 0.000s
I20260709 12:25:10.580379 29345 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:10.580881 29731 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:10.581930 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:10.581800 29876 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:10.583266 29876 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:10.588830 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.544409 29563 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:10.589550 26120 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:10.556576 29913 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.558111 29938 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.576716 29935 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:10.600925 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:10.602012 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.594182 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.602002 26052 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:10.602975 26124 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:10.596308 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:10.546088 29561 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:10.608103 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.608366 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:10.604875 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.606528 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:10.609875 26198 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:10.603076 26128 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:10.613324 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:10.614032 26124 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.613706 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:10.614755 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.617894 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:10.619107 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.620810 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.622927 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.623627 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:10.592905 29925 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:10.625855 29925 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.626071 29935 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.593328 29732 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.546855 29885 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 12:25:10.629019 29944 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:10.629530 29944 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:10.629935 29944 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.592756 29345 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:10.631709 29345 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.607581 29561 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.632774 29561 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.634182 29345 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.636545 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:10.637486 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.638073 29913 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.638865 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.639081 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.605510 29947 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:25:10.641755 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.642143 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2682) took 1206 ms. Trace:
I20260709 12:25:10.642635 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:10.642832 29875 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:10.641635 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:10.607045 29537 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:10.644634 29875 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:10.644940 29537 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:10.645143 29875 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:10.646487 29809 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.646334 29537 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.647986 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.549912 29882 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.623018 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.650101 26122 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 6.
W20260709 12:25:10.655860 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 113287us
I20260709 12:25:10.665062 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:10.647058 29809 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.644857 29794 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:10.668674 29794 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:10.639017 29925 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.642509 26127 rpcz_store.cc:276] 0709 12:25:09.436198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.977517 (+541319us) service_pool.cc:224] Handling call
0709 12:25:09.977928 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.003466 (+ 25538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.003519 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.041307 (+ 37788us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.257197 (+215890us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:10.567144 (+309947us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.641846 (+ 74702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.641918 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.641956 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:10.641971 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.642091 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":403585,"prepare.queue_time_us":309996,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":77177,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":15715,"thread_start_us":127091,"threads_started":1,"wal-append.queue_time_us":417012}
I20260709 12:25:10.660738 29939 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:10.661675 26202 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:10.671424 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:10.677982 26120 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.673348 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:10.669631 29794 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.602427 29571 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.683537 26195 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.684553 26124 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
W20260709 12:25:10.685642 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72057.8us
I20260709 12:25:10.689491 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.690155 26203 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:10.691084 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.692341 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.683393 29889 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:10.710827 29889 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:10.716666 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.676110 29939 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.721861 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.670632 29563 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:10.726478 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 230306us
I20260709 12:25:10.728314 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.727818 29537 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.755538 26203 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:10.757432 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.758929 26784 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:10.683545 29345 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:10.760269 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.779528 26788 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:10.784055 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.646335 29875 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:10.734385 29946 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:10.824029 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.840368 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.835395 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.789050 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:10.784621 26094 connection.cc:256] RPC call timeout handler was delayed by -2.5484250949998568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 12:25:10.786456 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.832204 29889 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.856879 29940 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:10.857915 29940 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:10.859164 29940 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.725966 29939 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:10.829247 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.706290 29614 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:25:10.765178 29794 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.726720 29815 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:10.706446 29571 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:10.846916 26782 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:10.787318 26786 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:25:10.955384 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:10.850764 26791 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:10.851327 26123 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:10.860291 29927 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:10.860884 29706 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:10.866065 29936 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:10.872377 29887 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:25:10.872546 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:10.971107 26198 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9.
I20260709 12:25:10.778905 29345 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.876827 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:10.795316 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.850960 26787 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
W20260709 12:25:10.856065 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1537959709999086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 12:25:10.892544 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 142835us
I20260709 12:25:10.784910 29898 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:10.825670 29908 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:10.953828 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:10.959354 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.876802 29886 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.962845 26201 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:10.864102 29837 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.964181 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:10.978114 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0286959710001611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 12:25:10.978544 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:10.976140 26788 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:10.982546 26049 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:10.982879 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:10.950495 29815 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:10.979831 26202 raft_consensus.cc:2376] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13. Re-sending same reply.
I20260709 12:25:11.012436 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:11.013406 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.014149 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 319672us
I20260709 12:25:10.965385 29875 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.980579 29345 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:11.022789 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 374854us
I20260709 12:25:10.954981 29923 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.111754 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:11.053922 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75487.6us
I20260709 12:25:10.978636 29706 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.050482 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.120780 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:11.121515 26788 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:11.010651 29936 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:11.074693 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:11.078253 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:11.078612 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:11.079118 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:11.079260 26784 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:11.079766 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:11.105274 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.974664 29809 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:10.952456 29946 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.005735 29908 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.120436 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:11.123389 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.120162 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.123462 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:11.124571 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:10.987578 29910 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:11.129206 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.129536 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:11.129762 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51235.4us
W20260709 12:25:11.136739 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 85917.8us
I20260709 12:25:11.129700 26126 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:11.128532 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47809.5us
I20260709 12:25:11.138628 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:11.136004 29907 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.141306 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:11.141901 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.075423 29837 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:11.143481 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:11.144179 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.147363 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:11.050100 29945 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:11.005484 29898 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.100304 29815 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.128517 29911 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.086341 29834 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.148543 26121 raft_consensus.cc:2368] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:11.150467 26786 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.187318 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:11.219919 26126 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:11.009639 29886 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.197290 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:11.234241 26122 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:11.085125 29947 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:11.253077 29947 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.197309 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.254652 26785 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:11.198431 26052 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.013s	sys 0.028s
I20260709 12:25:11.198977 26790 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:11.155046 29837 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:11.202790 26196 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:11.061003 29882 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:11.143424 29932 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:11.126772 29923 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.223102 26202 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:25:11.277990 26786 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:11.245396 29964 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:11.249754 29908 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:11.255558 26052 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.014s	sys 0.028s
I20260709 12:25:11.198426 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:11.279094 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.287151 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58316 (request call id 18) took 2024 ms. Trace:
I20260709 12:25:11.288343 29958 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:11.289469 29960 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:25:11.289732 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.148609 29933 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.421s
W20260709 12:25:11.291041 29955 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
W20260709 12:25:11.292150 29957 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 12:25:11.293015 29951 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
I20260709 12:25:11.299631 29707 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:11.301817 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:11.303054 29693 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:11.303107 29563 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:11.307472 29898 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.313460 29603 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:11.337559 29964 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:11.278219 29885 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:25:11.325353 26123 rpcz_store.cc:276] 0709 12:25:09.263047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.402220 (+139173us) service_pool.cc:224] Handling call
0709 12:25:11.286836 (+1884616us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540010f4d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:11.286967 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3093235584}
W20260709 12:25:11.330440 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 205977us
I20260709 12:25:11.328619 26198 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:25:11.266616 29837 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:11.290280 29954 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
W20260709 12:25:11.316645 29959 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
W20260709 12:25:11.316213 29776 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.746s	user 0.007s	sys 0.000s
W20260709 12:25:11.322717 26120 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.000s
I20260709 12:25:11.417460 26120 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:11.350128 26784 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:11.350363 26122 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:11.192668 29907 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:11.344797 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 206244us
W20260709 12:25:11.335927 29943 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.002s
I20260709 12:25:11.233937 29911 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.257395 29834 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:11.238339 26195 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
W20260709 12:25:11.386551 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84811us
I20260709 12:25:11.406064 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:11.475852 26126 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8.
I20260709 12:25:11.257851 29945 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:11.360337 29571 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.413100 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.413686 26196 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:11.398768 29732 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.757s	user 0.007s	sys 0.003s
I20260709 12:25:11.422119 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.407820 29693 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.422870 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.423034 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.424522 29975 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:11.425240 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:11.425890 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.427819 29947 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.423895 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:11.431313 26054 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:11.431435 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.440272 29706 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.440604 29964 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:11.443112 29527 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.988s	user 0.012s	sys 0.000s
I20260709 12:25:11.398269 29603 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:11.444324 29886 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:11.415504 29561 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.782s	user 0.007s	sys 0.004s
I20260709 12:25:11.412281 29731 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:11.451002 29511 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.916s	user 0.004s	sys 0.004s
I20260709 12:25:11.453686 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:11.347805 29936 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:11.435053 29955 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:11.436724 29776 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.461896 29910 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:11.462296 29907 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:11.462461 29961 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:11.464579 29923 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.475895 26195 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:11.422554 29946 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:11.521716 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:11.477075 29944 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.847s	user 0.009s	sys 0.004s
W20260709 12:25:11.524583 26194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.002s	sys 0.011s
W20260709 12:25:11.523913 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.659164 29946 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:11.568085 26049 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.659622 29946 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:11.523404 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:11.571787 26050 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:25:11.569273 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:11.586978 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.637280 29875 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
W20260709 12:25:11.522411 29794 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.004s
I20260709 12:25:11.620359 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:11.620937 29939 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.001s
I20260709 12:25:11.621296 29779 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:11.622835 26198 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
W20260709 12:25:11.625423 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 171805us
I20260709 12:25:11.456715 29345 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:11.626118 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 478845us
I20260709 12:25:11.628011 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.630584 29706 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:11.623318 29527 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:11.635469 29563 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:11.643573 26783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.015s	sys 0.019s
I20260709 12:25:11.520929 29732 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.687242 26125 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
W20260709 12:25:11.636287 29537 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.862s	user 0.009s	sys 0.001s
I20260709 12:25:11.648443 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.664609 26199 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:11.659636 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.777246 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:11.655431 29603 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:11.658078 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 136534us
I20260709 12:25:11.658715 29837 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.658797 29944 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:11.599043 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.768178 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.770174 26046 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:11.789595 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 266264us
I20260709 12:25:11.674336 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1846) took 1087 ms. Trace:
I20260709 12:25:11.796790 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:11.670830 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:11.669690 29962 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 12:25:11.777580 26194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.007s	sys 0.015s
I20260709 12:25:11.804275 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1844) took 1218 ms. Trace:
I20260709 12:25:11.777683 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:11.660080 29946 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.791224 29897 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:11.808861 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2707) took 1196 ms. Trace:
I20260709 12:25:11.600978 29975 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.671144 26049 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:11.766965 29776 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:11.798187 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:11.801205 26048 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:11.678352 29961 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.810647 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.663435 29911 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:11.629632 29561 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.809305 26052 rpcz_store.cc:276] 0709 12:25:10.612664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.651738 (+ 39074us) service_pool.cc:224] Handling call
0709 12:25:10.652189 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.652485 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:11.324818 (+672333us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.808554 (+483736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:11.808603 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:11.808644 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.808705 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.808807 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9575,"prepare.queue_time_us":555340,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":23309,"thread_start_us":1118699,"threads_started":2}
I20260709 12:25:11.797327 26195 rpcz_store.cc:276] 0709 12:25:10.586748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.588337 (+  1589us) service_pool.cc:224] Handling call
0709 12:25:11.674135 (+1085798us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b5400378554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:11.674250 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148894976}
I20260709 12:25:11.815515 26045 raft_consensus.cc:2368] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:11.782397 29877 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.816679 26053 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:11.671046 29834 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.721693 29931 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:11.819064 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:11.824214 29889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.006s	sys 0.012s
I20260709 12:25:11.643832 29815 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:11.786777 29956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.008s	sys 0.015s
I20260709 12:25:11.812951 29958 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.852173 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:11.637110 29511 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.828708 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.826845 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.829624 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:11.827625 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.830355 26786 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:11.827991 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:11.804945 26194 rpcz_store.cc:276] 0709 12:25:10.585842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.585975 (+   133us) service_pool.cc:224] Handling call
0709 12:25:10.616173 (+ 30198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.616682 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.616725 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.671573 (+ 54848us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.803820 (+1132247us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:11.803872 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.803966 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.804022 (+    56us) raft_consensus.cc:1685] finished
0709 12:25:11.804041 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.804188 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":54680,"thread_start_us":1077204,"threads_started":1,"wal-append.queue_time_us":964250}
I20260709 12:25:11.821089 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:11.839221 26054 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:11.839156 26052 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:11.839380 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:11.848556 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:11.637346 29731 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.846832 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:11.787169 29563 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.786628 29939 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.825191 29979 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.010537 29979 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.013281 29979 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.978049 29931 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.981801 29910 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:25:12.029008 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:12.030756 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.977913 29925 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.033056 29925 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.033758 26200 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:11.851320 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:12.034854 26048 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
W20260709 12:25:12.035243 29978 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.262s	user 0.001s	sys 0.003s
I20260709 12:25:11.801807 29990 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:25:12.035689 29925 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.978466 29897 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.036667 29897 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.037118 29990 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.037837 29990 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.039233 29897 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.040702 29732 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:12.040776 29990 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.043042 29731 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:12.043895 26200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1841) took 1968 ms (client timeout 2098 ms). Trace:
W20260709 12:25:12.044356 26200 rpcz_store.cc:269] 0709 12:25:10.076026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.076355 (+   329us) service_pool.cc:224] Handling call
0709 12:25:12.043714 (+1967359us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b5400336b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:12.043833 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4365542656}
I20260709 12:25:12.045151 29563 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:12.045537 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.859658 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:11.846967 29956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.009s	sys 0.016s
W20260709 12:25:11.982391 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 133966us
I20260709 12:25:12.048898 29956 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:11.822992 29614 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.049568 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:12.049983 29614 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.050690 29983 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.051254 29983 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.049578 29956 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:12.052554 29956 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:12.054384 29945 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:12.054984 29945 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:12.055478 29945 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.058490 29693 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.052554 29913 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:12.060882 29913 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:12.060690 29983 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:12.060264 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.985883 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1848) took 1392 ms. Trace:
I20260709 12:25:12.062337 26199 rpcz_store.cc:276] 0709 12:25:10.593405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.593578 (+   173us) service_pool.cc:224] Handling call
0709 12:25:11.985645 (+1392067us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002b5054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:11.985810 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2362201344}
I20260709 12:25:12.062844 29913 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.064373 29913 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.065064 26199 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:12.065897 29975 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:11.987527 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 148208us
I20260709 12:25:11.984073 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.827706 29773 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.999751 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:11.977730 29537 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.984030 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:12.068118 29773 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.068960 29537 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.071148 29773 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.071396 29537 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:11.996385 29706 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:12.072618 26198 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:12.072654 29976 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:12.073874 29976 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.074327 29976 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.074537 29939 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.984911 26049 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:11.854476 29911 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:12.077564 29911 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.078088 29911 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.833846 29961 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.009298 29837 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.033758 29910 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:12.079931 29910 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:11.977260 29958 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:12.048054 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 188501us
I20260709 12:25:12.068284 26203 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:12.006224 29815 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:11.983274 26789 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:12.061275 29935 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.082532 29935 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.082875 26203 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:12.084194 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:12.085702 29910 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.086359 29815 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:12.086505 29935 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:12.086871 29815 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.087297 29815 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.087759 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 936) took 1014 ms. Trace:
I20260709 12:25:12.082808 26783 raft_consensus.cc:2376] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11. Re-sending same reply.
I20260709 12:25:12.088315 26782 rpcz_store.cc:276] 0709 12:25:11.073248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.073370 (+   122us) service_pool.cc:224] Handling call
0709 12:25:11.073890 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.082101 (+  8211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.082161 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.125386 (+ 43225us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.520809 (+395423us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:11.818683 (+297874us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.832051 (+ 13368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.832159 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.087524 (+255365us) raft_consensus.cc:1685] finished
0709 12:25:12.087544 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.087694 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10282,"raft.queue_time_us":59310,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":31719,"thread_start_us":524523,"threads_started":2,"wal-append.queue_time_us":696799}
I20260709 12:25:12.093410 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:12.004971 29969 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:12.094684 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:12.096264 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:12.067592 29944 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.006891 29561 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.064008 29603 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.032182 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:12.124863 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:12.123936 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:12.101666 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.126573 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:12.128101 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:12.144364 26197 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:12.143843 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:11.847242 29730 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:12.077591 26198 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:12.117897 29773 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:25:12.121495 29935 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:12.131656 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.121765 29883 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.121289 29961 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.135541 29511 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:11.836966 29776 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.299s
I20260709 12:25:12.136521 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.177096 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:12.177510 26787 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:12.177793 26054 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:11.853745 29882 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:12.136662 29976 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.137831 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.138897 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.152844 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:25:12.159092 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.036s	sys 0.017s
I20260709 12:25:12.162310 26790 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:12.163934 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.147663 29927 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.014s	sys 0.045s
I20260709 12:25:12.099298 29931 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:12.160691 29706 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:12.249862 29706 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.160198 29614 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:12.167693 29907 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:12.253239 29907 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:12.254513 29907 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.257021 29945 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:11.851763 29834 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.258739 29834 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.129153 29979 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:11.839097 29877 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.260987 29345 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:12.160046 26202 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.029s	sys 0.026s
W20260709 12:25:12.262890 26202 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.030s	sys 0.026s
I20260709 12:25:12.180457 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:12.184599 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:12.263886 26049 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:12.205058 29958 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.168051 29923 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:25:12.160472 29969 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.167850 29561 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.149215 29563 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:12.179185 29883 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.148310 29693 raft_consensus.cc:2764] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:25:12.241899 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89171.4us
W20260709 12:25:12.242426 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.039s	sys 0.022s
I20260709 12:25:12.244722 29511 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:12.241803 29927 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.014s	sys 0.046s
I20260709 12:25:12.226325 29975 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:12.274489 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:12.270010 29561 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:12.277100 29882 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.267639 29945 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.283555 29536 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.294409 29707 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.295426 29889 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.334487 29707 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.334621 29889 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:12.296126 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.336725 29707 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.338327 26125 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 12:25:12.338804 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.339433 26129 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:12.296797 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:12.340206 29946 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.000s
I20260709 12:25:12.340777 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:12.341120 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.340695 29946 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.341470 26126 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.341686 26123 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:12.342051 26199 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:12.342556 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:12.343133 26123 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.344444 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.272162 29794 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:12.340580 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.347728 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.297853 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:12.277980 29511 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:12.351491 29511 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.351975 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.352555 29883 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:12.353382 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56738.9us
I20260709 12:25:12.353703 29911 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.354398 29345 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:12.354883 29345 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:12.329834 29961 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.356426 29923 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.356932 29923 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:12.356719 29961 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.357353 29923 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.317371 29561 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.358958 29961 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.340510 29536 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.360064 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.360491 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:12.284763 29706 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:12.349265 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51510.3us
I20260709 12:25:12.361395 29536 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.361707 29987 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.361850 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:12.347012 26128 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:12.355762 29345 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.361444 26125 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:25:12.316200 29940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.025s	sys 0.060s
I20260709 12:25:12.369418 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:12.368417 29706 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:12.292899 29834 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.283963 29563 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:12.374492 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.037s	sys 0.034s
W20260709 12:25:12.376420 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.037s	sys 0.034s
W20260709 12:25:12.378114 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 978228us
I20260709 12:25:12.379756 26054 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:12.380195 29706 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:12.380815 29834 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:12.381263 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 577) took 1010 ms. Trace:
I20260709 12:25:12.398880 26199 rpcz_store.cc:276] 0709 12:25:11.370875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.065955 (+695080us) service_pool.cc:224] Handling call
0709 12:25:12.066278 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.342470 (+276192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.381012 (+ 38542us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:12.381197 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633781504}
I20260709 12:25:12.381711 29980 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:12.344831 29931 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:12.356035 29911 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.347617 29946 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.288993 29815 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.397709 29603 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.401739 26126 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:12.348310 29794 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:12.286028 29956 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.278115 29969 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.284704 29975 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.402508 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:12.521605 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.518402 29947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.014s	sys 0.031s
W20260709 12:25:12.567931 29947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.014s	sys 0.032s
W20260709 12:25:12.490120 29964 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.015s	sys 0.020s
W20260709 12:25:12.569435 29964 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.015s	sys 0.020s
I20260709 12:25:12.570731 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:12.572682 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.003s	sys 0.016s
W20260709 12:25:12.573223 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.003s	sys 0.016s
W20260709 12:25:12.574040 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 777307us
I20260709 12:25:12.454283 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.560922 26790 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:12.576139 26201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.013s	sys 0.013s
W20260709 12:25:12.584722 26201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.013s	sys 0.013s
I20260709 12:25:12.585587 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:12.459802 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.590318 26787 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:12.595364 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.371620 29940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.025s	sys 0.060s
I20260709 12:25:12.597924 26169 leader_election.cc:400] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:12.629884 26169 leader_election.cc:403] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:25:12.629680 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54956.9us
I20260709 12:25:12.398566 29834 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.398329 29706 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:12.664472 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1472) took 1839 ms. Trace:
W20260709 12:25:12.646351 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.669147 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:12.419454 29527 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:12.673108 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.476142 29973 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.028s	sys 0.022s
I20260709 12:25:12.586912 29931 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.575110 29877 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.363294 29882 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:12.684281 26128 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:12.597028 29964 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.371343 29945 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.261s
I20260709 12:25:12.369237 29987 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:12.687948 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 113048us
I20260709 12:25:12.715533 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:12.717084 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:12.419862 29561 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.453900 29511 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.663403 29961 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.639799 29944 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:25:12.707948 29964 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.709285 29945 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:12.691314 29931 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.689244 29995 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:12.717964 26128 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:25:12.682956 29973 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.030s	sys 0.027s
I20260709 12:25:12.674867 29834 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:12.720181 26123 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:12.672415 29979 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.761734 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:12.761932 29979 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.762612 26128 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:12.763616 26125 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:12.577466 29603 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:12.766402 29603 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.681968 29883 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:12.767258 29883 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.767784 29883 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.381755 29563 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.681295 29965 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.733333 29946 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:12.757115 26048 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:12.727674 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:12.778663 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51124.5us
I20260709 12:25:12.735626 29945 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:12.737113 29940 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:12.702343 29794 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.787657 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:12.788138 26169 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8.
I20260709 12:25:12.798915 26169 leader_election.cc:403] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:12.789541 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:12.797708 26790 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 6.
I20260709 12:25:12.772500 29889 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.802146 26787 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:12.737757 29961 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.816064 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:12.821302 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.824070 26203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.025s	sys 0.024s
W20260709 12:25:12.827373 26203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.026s	sys 0.024s
W20260709 12:25:12.819985 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.022s	sys 0.016s
W20260709 12:25:12.828120 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.022s	sys 0.016s
W20260709 12:25:12.841301 26052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.014s	sys 0.023s
W20260709 12:25:12.841982 26052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.014s	sys 0.023s
I20260709 12:25:12.666947 26201 rpcz_store.cc:276] 0709 12:25:10.825358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.295392 (+470034us) service_pool.cc:224] Handling call
0709 12:25:11.295740 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.295999 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:11.778006 (+482007us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.663907 (+885901us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.664070 (+   163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:12.664115 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:12.664182 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.664281 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.664409 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47479,"prepare.queue_time_us":485895,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":85839,"thread_start_us":1250631,"threads_started":2}
I20260709 12:25:12.843297 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 432) took 1064 ms. Trace:
W20260709 12:25:12.858496 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69058.3us
I20260709 12:25:12.859695 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:12.875339 26049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.068s	sys 0.102s
W20260709 12:25:12.877374 26049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.069s	sys 0.102s
W20260709 12:25:12.846016 26782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.027s	sys 0.023s
W20260709 12:25:12.888520 26782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.027s	sys 0.023s
I20260709 12:25:12.887900 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:12.888036 26200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.011s	sys 0.033s
I20260709 12:25:12.889968 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:12.890451 26200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.012s	sys 0.033s
I20260709 12:25:12.793288 29815 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.817878 29773 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8.
I20260709 12:25:12.901647 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:12.902381 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:12.786405 29923 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:12.913319 26047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.005s	sys 0.029s
I20260709 12:25:12.929347 26045 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.906038 26128 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:12.934481 26047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.007s	sys 0.030s
W20260709 12:25:12.930015 26202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.016s	sys 0.029s
W20260709 12:25:12.935837 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76249.5us
I20260709 12:25:12.936983 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:12.864312 29794 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.802663 29940 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:12.940466 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:12.953506 26048 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:12.856441 29980 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:12.851486 29946 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.850649 29910 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:12.935838 26202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.016s	sys 0.029s
W20260709 12:25:12.935894 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.024s	sys 0.015s
I20260709 12:25:12.798985 29995 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:25:12.962211 29995 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.962685 29995 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:12.963444 29927 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.972396 29998 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:25:12.972595 29969 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:12.972388 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.024s	sys 0.015s
W20260709 12:25:12.973831 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 604437us
I20260709 12:25:12.975157 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:12.977248 29964 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.951736 29889 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:12.894652 29979 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:12.944483 29706 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:12.935933 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.006s	sys 0.032s
I20260709 12:25:12.889240 29693 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.852515 29988 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.960618 29707 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:12.963937 29927 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:13.293781 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.183704 26789 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.010s	sys 0.019s
W20260709 12:25:13.301700 26789 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.011s	sys 0.023s
W20260709 12:25:13.065716 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.843838 26052 rpcz_store.cc:276] 0709 12:25:11.778917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.098279 (+319362us) service_pool.cc:224] Handling call
0709 12:25:12.098678 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.098937 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:12.146647 (+ 47710us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.842760 (+696113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:12.842811 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:12.842874 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.842967 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.843116 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49229,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":93,"thread_start_us":695640,"threads_started":1}
W20260709 12:25:12.820436 29938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.002s	sys 0.033s
I20260709 12:25:13.224416 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 949) took 1128 ms. Trace:
W20260709 12:25:13.076256 26196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.227s	user 0.007s	sys 0.055s
I20260709 12:25:13.076310 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:13.305601 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:13.300926 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 325871us
I20260709 12:25:13.164951 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.324251 26782 rpcz_store.cc:276] 0709 12:25:12.095791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.097633 (+  1842us) service_pool.cc:224] Handling call
0709 12:25:12.098041 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.098354 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.098392 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.098990 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.888962 (+789972us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:12.889020 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.889111 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.224155 (+335044us) raft_consensus.cc:1685] finished
0709 12:25:13.224176 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.224348 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":770673,"threads_started":1,"wal-append.queue_time_us":1064211}
I20260709 12:25:13.331423 29927 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:13.332172 29961 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:13.288820 30003 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:13.333076 30003 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.333554 29961 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:13.334036 29961 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.334429 29961 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:13.335337 30003 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:13.168390 29882 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:13.075654 29907 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.023s	sys 0.025s
W20260709 12:25:13.336581 29907 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.023s	sys 0.039s
I20260709 12:25:13.337286 29907 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:13.305665 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.338097 29889 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:13.338927 26126 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.339478 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1475) took 2489 ms. Trace:
I20260709 12:25:13.339867 26200 rpcz_store.cc:276] 0709 12:25:10.849718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.046712 (+1196994us) service_pool.cc:224] Handling call
0709 12:25:12.047098 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.052969 (+  5871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.053004 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.053574 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.891080 (+837506us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:12.891142 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.891238 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.339250 (+448012us) raft_consensus.cc:1685] finished
0709 12:25:13.339269 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.339417 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5282,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":837036,"threads_started":1,"wal-append.queue_time_us":1246711}
W20260709 12:25:13.112529 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.304566 29964 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.342701 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.300845 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 363937us
I20260709 12:25:13.343541 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:13.098683 29837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.008s	sys 0.044s
I20260709 12:25:13.344105 26049 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
W20260709 12:25:13.098383 29936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.020s	sys 0.029s
I20260709 12:25:13.303884 26045 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
W20260709 12:25:13.324317 26196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.476s	user 0.011s	sys 0.056s
W20260709 12:25:13.306092 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.015s	sys 0.044s
W20260709 12:25:13.196555 29947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.008s	sys 0.022s
I20260709 12:25:13.310190 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.353803 26786 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:13.301179 29707 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:12.985687 29995 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:13.279475 30009 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:25:13.326457 29938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.002s	sys 0.040s
I20260709 12:25:13.053908 29561 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:12.955498 29511 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:13.379635 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:13.168295 29923 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:13.435132 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:13.435387 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91906.6us
I20260709 12:25:13.010113 29946 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:13.034003 29956 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:13.336181 29834 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.000s
W20260709 12:25:13.354696 29936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.028s	sys 0.036s
W20260709 12:25:13.374372 29947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.010s	sys 0.029s
I20260709 12:25:13.444741 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.358613 29707 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.423976 26123 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:13.453298 26128 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:13.392879 29988 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:13.455240 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.458657 26126 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:13.053534 29980 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.333878 29945 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.463464 29945 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.463979 29980 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:13.448454 29834 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.442232 29946 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.324088 29693 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:13.444065 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:13.473834 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.474849 26128 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:13.459846 29883 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.674s	user 0.006s	sys 0.004s
I20260709 12:25:13.472841 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:13.354408 29837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.015s	sys 0.047s
I20260709 12:25:13.486920 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.487735 29706 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:13.478078 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.478377 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.452482 29511 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:13.490310 29511 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:13.491261 29511 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.480799 26126 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:13.492728 29936 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:13.493069 29877 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:13.493518 29877 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:13.493721 26121 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:13.494452 29877 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:13.495764 29877 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.497570 29834 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:13.501952 29976 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:13.479168 29956 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.504679 29956 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.507114 29956 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:13.508594 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64641.8us
W20260709 12:25:13.511685 26048 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
I20260709 12:25:13.509362 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:13.514098 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.514546 26128 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:13.514787 26125 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.358861 29964 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.519100 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:13.520689 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:13.524842 26094 leader_election.cc:400] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:13.525202 26094 leader_election.cc:403] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:13.512163 26048 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
W20260709 12:25:13.542591 26019 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.013s	sys 0.021s
W20260709 12:25:13.543128 26783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.276s	user 0.039s	sys 0.059s
I20260709 12:25:13.541051 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:13.546718 26019 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.013s	sys 0.021s
W20260709 12:25:13.551177 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 768900us
W20260709 12:25:13.546950 26783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.280s	user 0.039s	sys 0.059s
I20260709 12:25:13.516335 29988 raft_consensus.cc:2764] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:13.564476 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.522938 29889 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:13.484699 29883 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.580901 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.525461 29947 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:13.586731 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.587361 26128 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.589303 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:13.545758 29996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.084s	user 0.015s	sys 0.026s
I20260709 12:25:13.583514 26125 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
W20260709 12:25:13.596325 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.016s	sys 0.036s
I20260709 12:25:13.590613 26126 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:25:13.598531 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.016s	sys 0.036s
I20260709 12:25:13.526026 29936 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:25:13.544521 29944 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:25:13.512714 29940 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.636705 29944 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:13.637099 29944 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:13.637964 29944 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.639524 29944 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.618309 26120 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:25:13.620008 29988 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:13.640619 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1474) took 2795 ms. Trace:
I20260709 12:25:13.640792 29988 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:13.641237 29988 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.621222 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 833) took 1286 ms. Trace:
I20260709 12:25:13.644392 26786 rpcz_store.cc:276] 0709 12:25:12.334729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.346157 (+ 11428us) service_pool.cc:224] Handling call
0709 12:25:12.369384 (+ 23227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.356664 (+987280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.620980 (+264316us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54005708d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:13.621155 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263442304}
I20260709 12:25:13.645068 29693 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:13.598091 29536 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:13.621593 29964 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:13.623354 29883 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:13.630967 29987 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:13.633723 29931 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.857s	user 0.010s	sys 0.000s
I20260709 12:25:13.633862 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 572) took 1294 ms. Trace:
I20260709 12:25:13.652524 26127 rpcz_store.cc:276] 0709 12:25:12.338925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.346257 (+  7332us) service_pool.cc:224] Handling call
0709 12:25:12.346540 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.346858 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.346881 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.351265 (+  4384us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.356036 (+1004771us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:13.356091 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.356177 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.633635 (+277458us) raft_consensus.cc:1685] finished
0709 12:25:13.633652 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.633799 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4737,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":37632,"thread_start_us":1004330,"threads_started":1,"wal-append.queue_time_us":1230580}
I20260709 12:25:13.633987 30027 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:13.655550 29931 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.657007 29965 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:13.657688 29965 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.634621 29936 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:13.658229 29936 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:13.659332 29936 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:13.660543 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2714) took 1387 ms. Trace:
W20260709 12:25:13.596885 29996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.015s	sys 0.027s
I20260709 12:25:13.620886 29946 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:13.589082 29345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.069s	sys 0.061s
I20260709 12:25:13.641041 26196 rpcz_store.cc:276] 0709 12:25:10.844874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.774965 (+930091us) service_pool.cc:224] Handling call
0709 12:25:11.775392 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:11.776450 (+  1058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.776481 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:11.803124 (+ 26643us) log.cc:844] Serialized 52 byte log entry
0709 12:25:13.355510 (+1552386us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:13.355569 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.355665 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.640387 (+284722us) raft_consensus.cc:1685] finished
0709 12:25:13.640408 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.640555 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11004,"raft.queue_time_us":1429,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":11229,"thread_start_us":1506685,"threads_started":1,"wal-append.queue_time_us":1763900}
I20260709 12:25:13.663015 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:13.678447 29345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.069s	sys 0.062s
I20260709 12:25:13.678790 26126 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:13.660954 26047 rpcz_store.cc:276] 0709 12:25:12.273099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.273242 (+   143us) service_pool.cc:224] Handling call
0709 12:25:12.273632 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.274009 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.274039 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:12.297831 (+ 23792us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.935492 (+637661us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:12.935532 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.935592 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.660306 (+724714us) raft_consensus.cc:1685] finished
0709 12:25:13.660328 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.660483 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":952,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":17364,"thread_start_us":573267,"threads_started":1,"wal-append.queue_time_us":1297898}
I20260709 12:25:13.677453 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:13.681670 29837 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:13.682067 29837 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.682433 29837 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.684439 29956 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:13.685837 29956 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:13.686187 29956 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:13.686959 29956 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.677690 29964 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:13.688890 29995 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:13.677989 29946 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:13.688103 29956 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:13.696372 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 368701us
I20260709 12:25:13.696810 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:13.660966 29936 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.661486 29931 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:13.698254 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:13.705941 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:13.678006 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83999.1us
I20260709 12:25:13.707379 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:13.699908 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:13.697898 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:13.703307 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.014s	sys 0.015s
W20260709 12:25:13.711941 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.015s	sys 0.015s
W20260709 12:25:13.699700 29794 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.001s
W20260709 12:25:13.721415 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 468469us
I20260709 12:25:13.727598 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:13.729941 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.701014 29563 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 12:25:13.731281 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.732937 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.734393 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:13.735155 26196 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.702932 29995 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:13.737139 26128 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:13.733174 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:13.748324 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.755035 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:13.757577 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.758351 26197 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:13.758599 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.759234 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:13.759795 26197 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.749220 29995 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.768489 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:13.769181 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:13.771139 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.772250 26201 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:13.774322 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.774554 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.774698 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:13.775173 26194 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.779559 29511 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.780578 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:13.781378 26786 raft_consensus.cc:2368] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:13.782516 26784 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
W20260709 12:25:13.782716 30016 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:13.783540 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:13.784286 26754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.017s	sys 0.020s
W20260709 12:25:13.784595 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.017s	sys 0.021s
I20260709 12:25:13.785121 29882 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.785719 29882 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:13.789203 26788 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:25:13.789963 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.790534 26784 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:13.785337 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.732067 30003 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:13.793151 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.797075 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.799568 26788 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.733655 29946 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:13.775230 26127 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:13.805069 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.805441 26196 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:13.735287 29927 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:13.810421 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.720701 29345 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:25:13.809327 26784 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:13.812003 26194 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
W20260709 12:25:13.792415 30013 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:13.811090 26787 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:13.845234 26790 raft_consensus.cc:2368] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:13.727366 29976 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:25:13.858276 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.773370 29794 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.848996 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.865316 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.866127 26124 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:13.787918 29882 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:13.868919 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:13.869521 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:13.870132 26787 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:13.852275 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:13.801342 29995 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:13.872341 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.809527 29693 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:13.847579 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.858914 26784 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.880745 26790 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:13.864920 30006 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.131s	user 0.003s	sys 0.000s
W20260709 12:25:13.861272 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.862073 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.798281 29975 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:13.867825 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:13.892452 26198 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:13.798413 30003 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.874433 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.871121 26791 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:13.751843 29707 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.855695 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:13.856526 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.904187 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.917222 26786 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.868976 29561 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:13.882115 29979 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:13.888183 26784 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:13.934758 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:13.899914 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:13.900992 30012 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:13.953632 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:13.912312 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.913125 26120 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
W20260709 12:25:13.876874 29961 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.000s
I20260709 12:25:13.877620 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 832) took 1545 ms. Trace:
I20260709 12:25:13.919476 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:13.956600 26123 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.957125 26124 raft_consensus.cc:2368] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:13.752311 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:13.878399 29693 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:13.916247 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 375245us
I20260709 12:25:13.937597 26790 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:13.873489 29794 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:13.914516 30003 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:13.978549 26198 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:13.985642 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.018388 26127 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.956296 26789 rpcz_store.cc:276] 0709 12:25:12.331873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.332973 (+  1100us) service_pool.cc:224] Handling call
0709 12:25:12.365791 (+ 32818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.365992 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:13.302305 (+936313us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.846310 (+544005us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.877123 (+ 30813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.877203 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.877273 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.877360 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.877544 (+   184us) inbound_call.cc:177] Queueing success response
0709 12:25:13.901680 (+ 24136us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b18003d75c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
Metrics: {"mutex_wait_us":25506,"prepare.queue_time_us":432018,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":74260,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11394,"spinlock_wait_cycles":66748928,"thread_start_us":1362856,"threads_started":2}
W20260709 12:25:14.010380 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.007823 26786 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:14.023864 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.024165 26123 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:14.025034 26194 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:14.027335 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1471) took 3240 ms. Trace:
W20260709 12:25:14.031517 26094 connection.cc:256] RPC call timeout handler was delayed by -1.8303147349997744s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 12:25:14.032271 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7862147349997031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1675s for a response
W20260709 12:25:14.032663 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7439147350000894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 12:25:14.033054 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6395147349999206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 12:25:14.033416 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5936147349998464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 12:25:14.033789 26094 connection.cc:256] RPC call timeout handler was delayed by -1.45681473500008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 12:25:14.034140 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3812147349999577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
W20260709 12:25:14.034513 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3407147349998922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 12:25:14.034874 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3398147349998908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
W20260709 12:25:14.035414 26094 connection.cc:256] RPC call timeout handler was delayed by -1.304714734999834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
I20260709 12:25:14.042416 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:14.035941 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2615147349997642s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2258s for a response
I20260709 12:25:14.044970 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.045778 26198 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.982702 29693 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:14.042449 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 586) took 2219 ms. Trace:
I20260709 12:25:14.027889 26202 rpcz_store.cc:276] 0709 12:25:10.786806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.324317 (+537511us) service_pool.cc:224] Handling call
0709 12:25:11.346918 (+ 22601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:11.355258 (+  8340us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:12.263244 (+907986us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.437493 (+1174249us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:13.701425 (+263932us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:25:13.715637 (+ 14212us) log.cc:844] Serialized 52 byte log entry
0709 12:25:13.715924 (+   287us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:13.715984 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.716049 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.016810 (+300761us) raft_consensus.cc:1685] finished
0709 12:25:14.027025 (+ 10215us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.027253 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7777,"prepare.queue_time_us":418466,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":32370,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":10398,"spinlock_wait_cycles":8763136,"thread_start_us":2004908,"threads_started":3,"wal-append.queue_time_us":1057524}
I20260709 12:25:13.968401 29979 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:14.050469 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.011504 29779 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:13.917804 29965 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.051229 26785 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
W20260709 12:25:14.052987 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.053841 26790 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:14.054039 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.066061 29976 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:14.057466 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 324346us
W20260709 12:25:14.057998 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 325127us
I20260709 12:25:14.043633 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:13.952551 29561 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.066983 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.058691 26203 rpcz_store.cc:276] 0709 12:25:11.822739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.084828 (+262089us) service_pool.cc:224] Handling call
0709 12:25:12.085180 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.085567 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.085597 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:12.086290 (+   693us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.827778 (+741488us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:12.933694 (+105916us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.626559 (+692865us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b180029b3c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:25:14.002404 (+375845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.002507 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.002563 (+    56us) raft_consensus.cc:1685] finished
0709 12:25:14.040616 (+ 38053us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.042357 (+  1741us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":104962,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":52650,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":169,"spinlock_wait_cycles":9004416,"thread_start_us":1116769,"threads_started":2,"wal-append.queue_time_us":1080844}
I20260709 12:25:14.056614 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.074450 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:13.967163 29961 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.073385 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:14.075994 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.078013 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 950) took 1955 ms. Trace:
I20260709 12:25:14.078809 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:14.083487 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.077790 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:14.080415 26783 rpcz_store.cc:276] 0709 12:25:12.122396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.122534 (+   138us) service_pool.cc:224] Handling call
0709 12:25:12.141410 (+ 18876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.261818 (+120408us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:12.261848 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.262365 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.554243 (+1291878us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:13.554302 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.623401 (+ 69099us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54001162d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:13.623452 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.071981 (+448529us) raft_consensus.cc:1685] finished
0709 12:25:14.072010 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.077951 (+  5941us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20713,"raft.queue_time_us":1102,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":4100,"spinlock_wait_cycles":275963264,"thread_start_us":1286924,"threads_started":1,"wal-append.queue_time_us":1447209}
I20260709 12:25:14.085734 26200 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:14.086669 26201 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:14.087239 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.088575 26200 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:14.090324 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:14.089720 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.036584 29883 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:25:14.061540 29882 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:14.093254 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.093096 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.095355 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:14.098001 26198 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:14.099211 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.079360 25941 maintenance_manager.cc:419] P 5eede3841e1242afbeab30840148b558: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033396
I20260709 12:25:14.100575 26784 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:14.101064 26198 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:14.100950 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 189730us
I20260709 12:25:14.102253 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:14.102840 26783 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:14.108433 26791 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:14.104491 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:14.105213 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.111409 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:14.114387 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:14.115031 26791 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:14.084524 29947 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.118996 26094 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9.
I20260709 12:25:14.119509 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:14.119666 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.119724 26094 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:14.120671 26125 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:14.121863 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.122602 26125 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
W20260709 12:25:14.105579 30029 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.296s	user 0.002s	sys 0.000s
I20260709 12:25:14.125535 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:14.127856 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:14.080938 29927 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:14.088639 29946 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:14.130234 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.099880 29889 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:14.134454 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:14.077837 29976 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.111217 29877 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:14.158015 26783 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:14.146782 26198 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:14.160166 26169 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:14.163287 26788 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:14.143953 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.128755 29940 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.129998 29883 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:14.174015 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83773.1us
I20260709 12:25:14.179649 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:14.181311 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:14.183743 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64873.5us
I20260709 12:25:14.185312 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:14.133476 30037 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.137594 30039 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:14.104259 29882 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:14.165663 30025 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:14.164906 29877 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:14.196142 29707 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:14.216324 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.176586 29345 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:14.249850 26120 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:14.250990 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.135279 29979 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.222441 30045 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.225486 30036 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:14.228668 30028 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:25:14.229964 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.222994 29940 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:14.256496 26128 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:14.256695 29940 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.230664 29927 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.136837 29987 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9.
I20260709 12:25:14.233397 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:14.176414 29975 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.173173 30026 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:14.243707 26127 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:14.177810 30040 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.221s
I20260709 12:25:14.219930 29946 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.257687 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:14.442559 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.297910 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.617404 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 457287us
I20260709 12:25:14.394002 26203 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:14.269951 26120 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:14.619478 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:14.389973 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.620998 26120 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:14.391741 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:14.622733 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 437471us
I20260709 12:25:14.409647 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:14.623733 26129 raft_consensus.cc:606] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:14.561612 29936 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.003s
I20260709 12:25:14.624747 26122 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:14.625600 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:14.232544 30025 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.626683 30025 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.627193 29940 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:14.627133 30026 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.629451 30026 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:14.388459 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 844) took 1046 ms. Trace:
I20260709 12:25:14.630194 26782 rpcz_store.cc:276] 0709 12:25:13.342085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.342432 (+   347us) service_pool.cc:224] Handling call
0709 12:25:13.342819 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:13.343227 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.343255 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:13.438180 (+ 94925us) log.cc:844] Serialized 52 byte log entry
0709 12:25:13.749973 (+311793us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18003d7800. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:14.049281 (+299308us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:14.049341 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.049417 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.387854 (+338437us) raft_consensus.cc:1685] finished
0709 12:25:14.387875 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.388391 (+   516us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108649,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":2442,"spinlock_wait_cycles":5864704,"thread_start_us":299093,"threads_started":1,"wal-append.queue_time_us":712537}
W20260709 12:25:14.265458 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.341145 30037 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:14.632761 30037 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.498044 29961 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:14.634488 30025 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.635739 30037 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:14.636791 30025 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:14.637200 30040 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.637799 30040 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:14.624101 26129 raft_consensus.cc:614] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:14.193094 30032 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:25:14.407437 26197 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.004s
W20260709 12:25:14.348642 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.642521 26123 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
W20260709 12:25:14.643884 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 462603us
I20260709 12:25:14.649753 26197 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:14.650202 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:14.406546 29837 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.724s	user 0.003s	sys 0.009s
I20260709 12:25:14.663771 26169 leader_election.cc:400] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:14.663901 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.667722 26169 leader_election.cc:403] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:25:14.339951 29988 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.698s	user 0.005s	sys 0.006s
W20260709 12:25:14.616742 30049 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.429s	user 0.001s	sys 0.000s
I20260709 12:25:14.665089 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.669164 26197 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.415551 30033 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:14.377156 29794 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:14.639232 29975 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.391500 29965 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:14.563201 29882 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:14.414469 29964 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:14.340386 29877 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:14.639866 29561 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:14.638769 30040 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:14.699885 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:14.700507 30056 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.700639 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:14.668143 30026 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.661188 29936 raft_consensus.cc:2764] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:25:14.704842 26197 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:14.700482 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.349893 30053 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:14.714157 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 88610.6us
I20260709 12:25:14.563946 30045 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:14.675459 29511 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.857s	user 0.014s	sys 0.000s
W20260709 12:25:14.719843 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 486512us
W20260709 12:25:14.725172 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75035.7us
I20260709 12:25:14.676366 30033 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:14.727214 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
W20260709 12:25:14.759055 26019 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.679s (SENT)
I20260709 12:25:14.759526 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:14.234086 30039 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.724855 29975 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.740195 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:14.765036 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:14.758487 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.676657 29837 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:14.723619 30027 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.006s	sys 0.017s
I20260709 12:25:14.760802 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.728983 29964 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.676872 29988 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.757557 29965 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:14.770449 29837 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:14.782527 26120 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:14.773452 26197 raft_consensus.cc:2376] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7. Re-sending same reply.
W20260709 12:25:14.773823 30027 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.006s	sys 0.018s
I20260709 12:25:14.776575 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.776666 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:14.777292 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.758613 29910 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:14.766245 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:14.759130 29561 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:14.779914 26126 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:14.778368 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:14.757685 29976 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:14.774498 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:14.784756 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.762754 30053 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:14.775323 29882 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:14.777853 29964 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.792507 29561 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.789417 29927 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.789454 30044 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:14.785153 29563 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.765851 29345 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:14.803593 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.781657 29965 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:14.788618 29995 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.004s
W20260709 12:25:14.788792 29693 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.727s	user 0.005s	sys 0.006s
I20260709 12:25:14.796380 29837 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.762482 30045 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.804941 30063 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:25:14.779805 30050 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.814536 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.797152 29882 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.832718 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.830271 29940 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.831915 29976 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:14.799232 30053 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.791550 29910 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.805105 29931 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.785427 30025 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.838152 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:14.810015 30003 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.000s
I20260709 12:25:14.815109 29936 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.816812 29563 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.802003 29561 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:14.846000 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 717902us
I20260709 12:25:14.850054 29882 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:14.848832 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.886284 26197 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:14.887492 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:14.852572 26198 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:14.786280 29779 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:25:14.856603 26782 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:14.889658 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2712) took 1070 ms. Trace:
I20260709 12:25:14.858127 29961 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.842176 29693 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.862561 29931 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.808840 29927 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.864269 29988 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:14.864434 30022 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 1.130s	user 0.010s	sys 0.006s
I20260709 12:25:14.814316 30044 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.864884 29976 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:14.857996 29910 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.867532 30050 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 12:25:14.855633 30060 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:25:14.856426 29940 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.862607 29345 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.871460 30045 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:14.872642 29956 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.180s	user 0.012s	sys 0.000s
I20260709 12:25:14.817309 30033 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.873557 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2701) took 1101 ms. Trace:
I20260709 12:25:14.878190 30003 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.834101 29965 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.885186 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:14.944207 26785 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:14.852212 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.09991e+06us
I20260709 12:25:14.946333 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:14.839769 29995 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:14.890134 26128 rpcz_store.cc:276] 0709 12:25:13.818959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.643974 (+825015us) service_pool.cc:224] Handling call
0709 12:25:14.644382 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.662595 (+ 18213us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:14.760855 (+ 98260us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.844097 (+ 83242us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.846200 (+  2103us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.846637 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.846810 (+   173us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:14.846861 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.846958 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.886293 (+ 39335us) raft_consensus.cc:1685] finished
0709 12:25:14.886322 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.886485 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13450,"prepare.queue_time_us":115760,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":4631,"raft.queue_time_us":21899,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7103,"spinlock_wait_cycles":29462912,"thread_start_us":111729,"threads_started":2,"wal-append.queue_time_us":91269}
I20260709 12:25:14.891340 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:14.950217 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 58964.1us
I20260709 12:25:14.861512 30039 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:14.889364 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:14.862483 30025 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.877656 30024 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:14.914419 29563 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:14.917374 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 120018us
I20260709 12:25:14.876478 29794 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:14.867096 29837 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:14.897477 29944 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.258s	user 0.017s	sys 0.000s
W20260709 12:25:14.919879 30057 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:25:14.928973 26197 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:14.895752 29936 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:14.929479 26194 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:14.930787 26122 rpcz_store.cc:276] 0709 12:25:13.803467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.804090 (+   623us) service_pool.cc:224] Handling call
0709 12:25:13.804504 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.625086 (+820582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.873280 (+248194us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b540006fed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:14.873497 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1885635328}
I20260709 12:25:14.975044 26196 raft_consensus.cc:2368] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:14.975417 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:14.875496 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.914561 29779 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:25:14.957512 26047 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:14.958380 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.977737 26128 raft_consensus.cc:2376] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7. Re-sending same reply.
I20260709 12:25:14.978778 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:14.979329 26128 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.959908 29561 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.949672 30033 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:14.962343 26018 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.806s (SENT)
I20260709 12:25:14.962720 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.964529 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1881) took 1027 ms. Trace:
I20260709 12:25:14.984054 26122 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.916414 29961 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:14.972358 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1979) took 1062 ms. Trace:
I20260709 12:25:14.984549 26200 rpcz_store.cc:276] 0709 12:25:13.937059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.089428 (+152369us) service_pool.cc:224] Handling call
0709 12:25:14.964286 (+874858us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54002b5f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:14.964448 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539303296}
I20260709 12:25:14.971796 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:14.982102 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.985524 26129 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:14.957413 30035 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.985435 26121 rpcz_store.cc:276] 0709 12:25:13.910269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.622185 (+711916us) service_pool.cc:224] Handling call
0709 12:25:14.622489 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.622827 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.622858 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.623356 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.696145 (+ 72789us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:14.696206 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.696322 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.972079 (+275757us) raft_consensus.cc:1685] finished
0709 12:25:14.972100 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.972283 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2580,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":72280,"threads_started":1,"wal-append.queue_time_us":137833}
I20260709 12:25:15.022159 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:15.022182 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:14.997366 26128 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:15.023479 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:15.025779 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:15.027040 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.024017 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:15.032188 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2709) took 1224 ms. Trace:
W20260709 12:25:15.032474 30062 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:25:15.032758 26123 rpcz_store.cc:276] 0709 12:25:13.808017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.389710 (+581693us) service_pool.cc:224] Handling call
0709 12:25:15.032019 (+642309us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:15.032119 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48273152}
I20260709 12:25:15.033411 30063 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:15.033978 30063 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.036764 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.037456 26048 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:14.957229 30024 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:15.039054 26782 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:15.039762 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:15.034097 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:15.042399 26121 raft_consensus.cc:2368] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
W20260709 12:25:15.042513 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.025245 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:15.043397 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:15.040038 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68321.5us
I20260709 12:25:15.044303 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:15.040750 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.047103 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:15.047542 26052 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.044505 26785 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:15.045171 26124 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:15.045425 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:15.056862 26128 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:14.970292 30026 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:25:14.977105 29837 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:15.046411 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:15.044404 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:15.066627 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.068147 26782 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:15.069404 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.046763 26123 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
I20260709 12:25:15.059211 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.046151 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:15.060087 26018 leader_election.cc:400] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:15.074136 26018 leader_election.cc:403] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:15.074847 26046 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:15.075130 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.075402 26200 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:15.023998 30035 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:15.060716 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:15.077479 26785 raft_consensus.cc:2368] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
W20260709 12:25:15.037675 30061 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 12:25:15.038206 30071 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:15.073236 26050 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:15.085865 26054 raft_consensus.cc:2368] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9.
I20260709 12:25:15.076660 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:15.045063 30024 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:15.062394 30039 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:15.091781 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:15.097405 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:15.053974 30066 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:15.093623 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:15.021153 30037 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:25:14.973553 29794 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.112669 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:15.098726 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:15.104202 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.107975 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:15.077397 29946 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:15.106915 30041 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:15.121073 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.121400 30041 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:15.122181 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173937us
I20260709 12:25:15.122884 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:15.122602 30041 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.123191 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:15.124449 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:15.087060 30027 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:15.086371 30035 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.099834 30066 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.988211 29976 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.107388 30052 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:15.092126 29940 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:15.092669 30064 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.980553 30004 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
W20260709 12:25:15.140105 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.096253 30065 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:15.126132 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:15.124747 26201 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:15.156162 26200 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.149655 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:15.125193 29938 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:15.153172 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.144035 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1874) took 1293 ms. Trace:
I20260709 12:25:15.161531 26199 rpcz_store.cc:276] 0709 12:25:13.850974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.415733 (+564759us) service_pool.cc:224] Handling call
0709 12:25:14.416129 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.633978 (+217849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.634008 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:14.634051 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:14.634080 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.143790 (+509710us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400301e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:15.143836 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.143971 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500432640}
I20260709 12:25:15.143435 30066 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:15.170483 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71814us
I20260709 12:25:15.145905 29946 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:15.172509 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.173321 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.185810 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.187237 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.186028 26046 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:15.161335 30065 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:15.164551 29938 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:15.198046 26051 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:15.167608 29931 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.170188 30052 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:15.169540 29927 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:15.176373 29956 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:15.117748 29936 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.166141 29944 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:15.203652 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 163181us
W20260709 12:25:15.205114 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.211742 30079 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:25:15.215003 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.220301 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:15.220947 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 195790us
I20260709 12:25:15.194144 30003 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:15.198575 29995 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:15.232636 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.221174 29956 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.198071 30065 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.207752 29975 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.302917 30079 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.304672 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:15.170954 29837 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.327231 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:15.327986 26202 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:15.319069 26122 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
W20260709 12:25:15.321872 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 88495.9us
I20260709 12:25:15.233400 29561 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:15.247267 29961 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:15.333877 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 438) took 1627 ms (client timeout 1688 ms). Trace:
W20260709 12:25:15.334451 26047 rpcz_store.cc:269] 0709 12:25:13.697851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.697995 (+   144us) service_pool.cc:224] Handling call
0709 12:25:15.325493 (+1627498us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540035a2d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:15.325622 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2846700928}
I20260709 12:25:15.340715 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.289036 29956 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:15.331981 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:15.235425 29944 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.361805 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:15.369601 26127 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:15.363982 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:15.364827 29563 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:15.434882 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70963.5us
I20260709 12:25:15.373391 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:15.436753 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 358921us
I20260709 12:25:15.460813 29927 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:15.461416 29927 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:15.461908 29927 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.463109 29944 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.310199 29938 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.514348 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:15.472374 30068 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:25:15.563898 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:15.251717 29946 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.434451 29563 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:15.565343 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:15.572599 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:15.576789 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62591.7us
W20260709 12:25:15.577543 26094 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.541s (SENT) [suppressed 23 similar messages]
W20260709 12:25:15.578069 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 204750us
I20260709 12:25:15.561134 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:15.585431 26093 leader_election.cc:400] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:15.585811 26093 leader_election.cc:403] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:15.584714 29563 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.590390 29944 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:15.592126 29944 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:15.595361 29944 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.368170 30087 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.226s
W20260709 12:25:15.610349 26125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.002s	sys 0.014s
W20260709 12:25:15.612638 26125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.002s	sys 0.014s
I20260709 12:25:15.615166 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:15.617472 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:15.624934 26020 leader_election.cc:400] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:15.626787 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:15.601856 29910 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.004s
I20260709 12:25:15.599282 30081 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:15.633428 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:15.624675 30053 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.006s
I20260709 12:25:15.618353 30070 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:15.654170 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.655372 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.360890 30052 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.684348 30070 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.685266 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.686522 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.686785 30070 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:15.687088 26048 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:15.687487 29561 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:15.632763 29910 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:15.658777 29877 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.915s	user 0.014s	sys 0.001s
I20260709 12:25:15.688504 29877 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.689587 29910 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:15.690407 26052 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.690765 30044 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:15.692059 26054 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.694558 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1888) took 1073 ms. Trace:
I20260709 12:25:15.691884 26120 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:15.661638 30053 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.690778 29877 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:15.661167 30056 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.900s	user 0.015s	sys 0.000s
I20260709 12:25:15.697186 30056 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:15.624158 30039 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:15.697592 30044 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.698086 30039 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:15.698340 30053 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.698144 30044 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.699366 26051 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.699170 30039 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.637631 30097 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:15.677385 26202 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:15.728914 26054 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:15.683254 30081 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:15.694989 26200 rpcz_store.cc:276] 0709 12:25:14.621521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.156858 (+535337us) service_pool.cc:224] Handling call
0709 12:25:15.177180 (+ 20322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:15.184337 (+  7157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.184368 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:15.199100 (+ 14732us) log.cc:844] Serialized 52 byte log entry
0709 12:25:15.342482 (+143382us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:15.370103 (+ 27621us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.510888 (+140785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.511008 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.652205 (+141197us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b180039d060. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:15.694250 (+ 42045us) raft_consensus.cc:1685] finished
0709 12:25:15.694352 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.694486 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6647,"prepare.queue_time_us":90222,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":69726,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":79602304,"thread_start_us":106645,"threads_started":1,"wal-append.queue_time_us":394435}
W20260709 12:25:15.672446 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.597522 29536 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 12:25:15.679644 30024 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.008s
I20260709 12:25:15.737258 26198 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:15.683697 29975 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.631026 30040 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:15.683739 26199 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.687976 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:15.719125 30033 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
W20260709 12:25:15.729095 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1875) took 1871 ms (client timeout 1952 ms). Trace:
W20260709 12:25:15.679131 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52419.9us
I20260709 12:25:15.730451 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.699905 26197 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
W20260709 12:25:15.694505 29964 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.000s
I20260709 12:25:15.691882 30065 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:15.773329 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.748142 26121 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:15.744460 26048 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:15.746490 26202 rpcz_store.cc:269] 0709 12:25:13.858350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.074534 (+216184us) service_pool.cc:224] Handling call
0709 12:25:15.327118 (+1252584us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:25:15.729012 (+401894us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2728832}
I20260709 12:25:15.777281 26202 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:15.748574 26054 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:15.778136 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 200599us
W20260709 12:25:15.786141 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.768759 30065 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:15.833608 30075 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:15.834146 30075 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:15.747948 29882 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.849s	user 0.011s	sys 0.000s
I20260709 12:25:15.835075 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:15.834834 29882 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.835012 30075 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:15.836015 30097 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:15.836490 30097 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:15.836869 30097 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.750207 29964 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.683794 29927 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.843120 26053 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.803254 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.845516 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.864580 26054 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.743917 30040 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:15.868158 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1476) took 1052 ms. Trace:
I20260709 12:25:15.871003 26202 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:15.871855 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:15.882113 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:15.884024 26094 leader_election.cc:400] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9.
I20260709 12:25:15.884603 26094 leader_election.cc:403] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:15.898357 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.869076 26198 rpcz_store.cc:276] 0709 12:25:14.815665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.699896 (+884231us) service_pool.cc:224] Handling call
0709 12:25:15.737013 (+ 37117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.745834 (+  8821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.868066 (+122232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.899247 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:15.899689 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.900425 26054 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:15.788581 29794 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.900743 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:15.904026 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:15.904774 26049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.007s	sys 0.018s
W20260709 12:25:15.906131 26049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.007s	sys 0.019s
I20260709 12:25:15.902825 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.908972 26045 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:15.908845 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.916728 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.917627 26054 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:15.918838 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:15.922016 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.923213 26048 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:15.772519 30024 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:15.871028 29965 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.919s	user 0.004s	sys 0.009s
I20260709 12:25:15.870311 30040 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.869262 29964 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.739069 30081 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.899005 30044 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:15.901911 30033 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:15.948737 30109 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9.
I20260709 12:25:15.949154 30024 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:15.897673 29536 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:25:15.950094 26049 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:15.952560 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:15.956017 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:15.961704 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:15.971098 26202 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:15.957875 30033 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:15.962795 29927 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:15.966413 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.974577 26788 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:25:15.951103 30087 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:15.991106 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61586.6us
I20260709 12:25:15.993304 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.001842 26202 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:15.937750 29965 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:15.980120 30033 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.012110 26202 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:16.013903 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 598) took 1132 ms. Trace:
I20260709 12:25:16.014501 26202 rpcz_store.cc:276] 0709 12:25:14.881967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.011610 (+1129643us) service_pool.cc:224] Handling call
0709 12:25:16.011936 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.013628 (+  1692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.013842 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.023324 30107 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:16.023927 29536 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:16.024155 30107 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.024853 29536 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.024941 30107 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.025450 29536 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:16.027820 29536 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.028033 30107 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.028637 30118 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.029138 30118 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.031570 30118 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.032660 30108 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.033135 30108 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.035256 30108 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.035730 30108 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:16.036586 30119 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.037109 30119 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.039634 30119 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.036068 30108 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:16.041548 30108 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.042870 30108 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:15.990360 30109 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:15.990274 29707 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:15.955880 30102 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:25:15.988749 30087 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:15.989189 30106 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:15.979467 29563 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:16.057251 30035 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.906s	user 0.007s	sys 0.004s
W20260709 12:25:16.057727 30064 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.894s	user 0.009s	sys 0.005s
I20260709 12:25:16.057616 30087 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.084492 29563 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.077919 30064 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
W20260709 12:25:16.078179 30066 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.887s	user 0.012s	sys 0.001s
W20260709 12:25:16.087348 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.087495 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1985) took 1212 ms. Trace:
I20260709 12:25:16.057715 29707 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:16.093094 30093 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:25:16.091478 26125 rpcz_store.cc:276] 0709 12:25:14.874684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.056411 (+181727us) service_pool.cc:224] Handling call
0709 12:25:15.056717 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.615102 (+558385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.615133 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.615646 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.988056 (+372410us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:15.990040 (+  1984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.990561 (+   521us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.087272 (+ 96711us) raft_consensus.cc:1685] finished
0709 12:25:16.087292 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.087434 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33028,"raft.queue_time_us":44131,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13577,"thread_start_us":845038,"threads_started":2,"wal-append.queue_time_us":453517}
W20260709 12:25:16.096494 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1873) took 2246 ms (client timeout 2183 ms). Trace:
W20260709 12:25:16.097553 26203 rpcz_store.cc:269] 0709 12:25:13.850027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.075816 (+225789us) service_pool.cc:224] Handling call
0709 12:25:16.096299 (+2020483us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.096417 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690372864}
I20260709 12:25:16.098011 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.098584 26125 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.097349 26202 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:16.091847 29956 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.719s	user 0.016s	sys 0.001s
W20260709 12:25:16.104139 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1858) took 2342 ms (client timeout 1679 ms). Trace:
I20260709 12:25:16.113168 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:16.113977 26203 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13.
I20260709 12:25:16.114187 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2731) took 1158 ms. Trace:
W20260709 12:25:16.112879 26195 rpcz_store.cc:269] 0709 12:25:13.757476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.758305 (+   829us) service_pool.cc:224] Handling call
0709 12:25:16.095695 (+2337390us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.099683 (+  3988us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218409728}
I20260709 12:25:16.114605 26129 rpcz_store.cc:276] 0709 12:25:14.955905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.059247 (+103342us) service_pool.cc:224] Handling call
0709 12:25:15.059510 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.151473 (+ 91963us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:15.185119 (+ 33646us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.992098 (+806979us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.992711 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.992892 (+   181us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:15.992949 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.993068 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.113979 (+120911us) raft_consensus.cc:1685] finished
0709 12:25:16.114000 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.114126 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12446,"prepare.queue_time_us":37372,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":42740,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":9703,"thread_start_us":839088,"threads_started":3,"wal-append.queue_time_us":557944}
I20260709 12:25:16.109218 30063 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.110374 30081 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.116194 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.093905 30107 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:16.093504 30066 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.077836 30035 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.132577 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:16.150918 26195 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:16.151522 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.152446 26195 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:16.152349 26129 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:16.154935 26195 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:16.124511 29956 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.157344 26054 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:16.159585 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:16.160414 26203 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:16.168964 29965 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:16.169482 30100 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 12:25:16.149114 30081 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
W20260709 12:25:16.146962 30091 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.582s	user 0.003s	sys 0.000s
I20260709 12:25:16.176767 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.177805 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.178751 26783 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:16.178846 26788 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:16.147135 29877 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:16.180440 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.133011 30120 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:16.185887 26125 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:16.187438 26046 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:16.187295 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:16.174386 30110 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.098s	user 0.005s	sys 0.000s
I20260709 12:25:16.189409 26788 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:25:16.190471 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.06756e+06us
I20260709 12:25:16.191978 26129 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:16.193591 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.194240 26129 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:16.198268 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:16.202778 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:16.204542 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:16.265082 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.269553 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.270439 26050 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:16.320070 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.359500 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2725) took 1187 ms. Trace:
W20260709 12:25:16.320545 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 116061us
I20260709 12:25:16.370697 26052 rpcz_store.cc:276] 0709 12:25:15.172050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.172221 (+   171us) service_pool.cc:224] Handling call
0709 12:25:16.276774 (+1104553us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54001b3fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.359417 (+ 82643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180987776}
I20260709 12:25:16.360781 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.372143 26129 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:16.372324 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.373142 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.190804 29693 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.185204 29877 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:16.378502 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 157141us
I20260709 12:25:16.378670 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.381050 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 955) took 2322 ms. Trace:
I20260709 12:25:16.201525 29907 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:16.213644 29931 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:16.401099 29931 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:16.401491 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2698) took 2630 ms. Trace:
I20260709 12:25:16.401873 26126 rpcz_store.cc:276] 0709 12:25:13.771171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.773440 (+  2269us) service_pool.cc:224] Handling call
0709 12:25:13.773719 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.788580 (+1014861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.401241 (+1612661us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:16.401410 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2313563904}
I20260709 12:25:16.403069 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.403662 26126 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 6.
I20260709 12:25:16.404608 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.405108 26126 raft_consensus.cc:2376] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9. Re-sending same reply.
I20260709 12:25:16.405448 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2728) took 1231 ms. Trace:
I20260709 12:25:16.405820 26053 rpcz_store.cc:276] 0709 12:25:15.174035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.186944 (+ 12909us) service_pool.cc:224] Handling call
0709 12:25:16.405277 (+1218333us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b5400172d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.405391 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506425728}
I20260709 12:25:16.406790 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.407351 26053 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:16.373347 29794 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:16.408567 29794 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:16.409562 29794 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.402148 29931 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.408192 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.413794 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:16.414320 30117 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:25:16.414868 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:16.379925 30115 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.370s	user 0.005s	sys 0.008s
I20260709 12:25:16.381690 26052 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:16.374426 29889 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:16.363907 30081 raft_consensus.cc:2764] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:16.188432 30040 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.396930 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.369740 30036 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.399586 29877 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.400509 29907 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:16.318815 30113 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:16.425630 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.53619e+06us
I20260709 12:25:16.373709 29511 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:16.424840 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.6365e+06us
I20260709 12:25:16.426844 26051 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:16.381428 26789 rpcz_store.cc:276] 0709 12:25:14.048192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.048481 (+   289us) service_pool.cc:224] Handling call
0709 12:25:14.060185 (+ 11704us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:14.845147 (+784962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.845180 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:14.845764 (+   584us) log.cc:844] Serialized 52 byte log entry
0709 12:25:14.930122 (+ 84358us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:14.948572 (+ 18450us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.177825 (+1229253us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b18000ce740. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:25:16.369700 (+191875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.369907 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.370024 (+   117us) raft_consensus.cc:1685] finished
0709 12:25:16.370100 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.370261 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":791118,"prepare.queue_time_us":20152,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":2017486,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":19867,"spinlock_wait_cycles":9634560,"thread_start_us":270869,"threads_started":2,"wal-append.queue_time_us":82258,"wal-append.run_cpu_time_us":3444,"wal-append.run_wall_time_us":1169654}
I20260709 12:25:16.213176 30074 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.224s
I20260709 12:25:16.197064 30120 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:16.457649 26019 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.098s (SENT)
I20260709 12:25:16.459422 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.429394 26052 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:16.444708 30036 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.427655 30126 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.427075 30025 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.457823 26053 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:25:16.460512 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 441) took 1178 ms. Trace:
I20260709 12:25:16.473675 26052 rpcz_store.cc:276] 0709 12:25:15.282477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.427865 (+1145388us) service_pool.cc:224] Handling call
0709 12:25:16.428159 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.460179 (+ 32020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.460446 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.461793 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.475507 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.479810 26782 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:16.480753 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.423031 29889 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:16.482344 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.483018 26787 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:16.483093 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:16.483323 26120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2724) took 1707 ms (client timeout 1573 ms). Trace:
I20260709 12:25:16.481582 26791 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 6.
W20260709 12:25:16.485473 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 597) took 2722 ms (client timeout 1850 ms). Trace:
W20260709 12:25:16.485981 26194 rpcz_store.cc:269] 0709 12:25:13.762805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.025965 (+263160us) service_pool.cc:224] Handling call
0709 12:25:16.485223 (+2459258us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.485389 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996681600}
I20260709 12:25:16.487080 26200 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:16.487424 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:16.488703 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.489349 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:16.489464 26782 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8. Re-sending same reply.
I20260709 12:25:16.453012 30113 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.423408 30033 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:16.494204 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.500917 26789 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:16.483721 26120 rpcz_store.cc:269] 0709 12:25:14.775918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.796417 (+ 20499us) service_pool.cc:224] Handling call
0709 12:25:16.483139 (+1686722us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400185bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.483261 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948981504}
I20260709 12:25:16.496411 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:16.496788 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 958) took 2390 ms (client timeout 2147 ms). Trace:
W20260709 12:25:16.502763 26790 rpcz_store.cc:269] 0709 12:25:14.092310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.092484 (+   174us) service_pool.cc:224] Handling call
0709 12:25:16.482895 (+2390411us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400409a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.482997 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494760832}
I20260709 12:25:16.497258 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.497829 26051 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:16.505028 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2735) took 1196 ms. Trace:
I20260709 12:25:16.500094 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.502192 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.507150 26788 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:16.473047 30063 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.505463 26051 rpcz_store.cc:276] 0709 12:25:15.308963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.459051 (+1150088us) service_pool.cc:224] Handling call
0709 12:25:16.491351 (+ 32300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.504705 (+ 13354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.504958 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.513530 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.514806 26790 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:16.477303 29837 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:16.493242 29877 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:16.519709 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.522123 26784 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:25:16.522501 26784 pending_rounds.cc:85] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 9
I20260709 12:25:16.520934 26789 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:16.383244 30106 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:25:16.492111 29976 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:16.522723 26784 pending_rounds.cc:107] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:25:16.526057 26784 raft_consensus.cc:2889] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.500136 30113 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.524961 26125 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:16.530462 26051 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:16.531383 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.531976 26120 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:16.537310 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2737) took 1227 ms. Trace:
I20260709 12:25:16.537768 26051 rpcz_store.cc:276] 0709 12:25:15.309499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.520296 (+1210797us) service_pool.cc:224] Handling call
0709 12:25:16.530300 (+ 10004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.537051 (+  6751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.537254 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.484064 30126 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.547983 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.548792 26120 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11.
I20260709 12:25:16.553519 26051 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:16.553889 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64586.4us
I20260709 12:25:16.531200 29837 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:16.531591 30066 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.559149 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:16.531005 29956 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:16.562533 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.524459 29931 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:16.566344 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:16.566854 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 79521.9us
I20260709 12:25:16.567988 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2743) took 1232 ms. Trace:
I20260709 12:25:16.568583 26051 rpcz_store.cc:276] 0709 12:25:15.335419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.538734 (+1203315us) service_pool.cc:224] Handling call
0709 12:25:16.553325 (+ 14591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.557638 (+  4313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.567929 (+ 10291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.543979 30129 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:16.570138 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.574589 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.557770 29837 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.557111 29988 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:16.577425 26120 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:16.576804 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.585227 26120 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:16.582835 26051 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:16.587157 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2744) took 1245 ms. Trace:
I20260709 12:25:16.587556 26051 rpcz_store.cc:276] 0709 12:25:15.341794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.569330 (+1227536us) service_pool.cc:224] Handling call
0709 12:25:16.582672 (+ 13342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.586922 (+  4250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.587103 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.589555 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.571157 30035 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.591873 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.576699 29956 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:16.523103 29889 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.575361 30129 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:16.593380 29794 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:16.543524 29907 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.586745 29837 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.593451 30135 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:16.614217 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.591845 29945 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:16.624828 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:16.603821 30066 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.614750 29976 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.590792 29988 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.610044 30129 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:16.621073 29931 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:16.636370 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59649.1us
I20260709 12:25:16.650053 29794 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:16.669034 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 446) took 1096 ms. Trace:
I20260709 12:25:16.629681 30040 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:16.676496 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51794.5us
I20260709 12:25:16.629736 30136 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:16.669688 26051 rpcz_store.cc:276] 0709 12:25:15.572224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.597502 (+1025278us) service_pool.cc:224] Handling call
0709 12:25:16.609054 (+ 11552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.668498 (+ 59444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.668546 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.668613 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:16.668658 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.668752 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.668943 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20427,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3794,"thread_start_us":58965,"threads_started":1}
I20260709 12:25:16.677768 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.635818 30120 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.651188 29987 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:16.652554 29889 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.631850 30066 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.693343 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2736) took 1384 ms. Trace:
I20260709 12:25:16.693742 26052 rpcz_store.cc:276] 0709 12:25:15.309241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.474630 (+1165389us) service_pool.cc:224] Handling call
0709 12:25:16.474993 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.475210 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:16.565297 (+ 90087us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.636513 (+ 71216us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.691708 (+ 55195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.691768 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:16.693071 (+  1303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.693152 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.693289 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111907,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":2084,"raft.queue_time_us":77965,"raft.run_wall_time_us":4426,"spinlock_wait_cycles":11440896,"thread_start_us":189504,"threads_started":3}
I20260709 12:25:16.677372 30036 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.699261 26051 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:16.692029 30127 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 12:25:16.704998 30134 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:25:16.706862 29988 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.708609 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1993) took 1115 ms. Trace:
W20260709 12:25:16.696537 30097 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.859s	user 0.012s	sys 0.001s
I20260709 12:25:16.692447 30136 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:16.723466 26126 rpcz_store.cc:276] 0709 12:25:15.593164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.405816 (+812652us) service_pool.cc:224] Handling call
0709 12:25:16.406191 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.406459 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.406492 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.406944 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.552064 (+145120us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:16.552106 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.552187 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.708367 (+156180us) raft_consensus.cc:1685] finished
0709 12:25:16.708388 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.708529 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":144692,"threads_started":1,"wal-append.queue_time_us":226392}
I20260709 12:25:16.716255 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 450) took 1126 ms. Trace:
I20260709 12:25:16.716410 26049 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 12:25:16.717862 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1866) took 2922 ms (client timeout 2098 ms). Trace:
W20260709 12:25:16.721009 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71624.7us
I20260709 12:25:16.725078 26046 rpcz_store.cc:276] 0709 12:25:15.590143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.934224 (+344081us) service_pool.cc:224] Handling call
0709 12:25:16.715929 (+781705us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540006fc54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.716151 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494451584}
W20260709 12:25:16.725678 26196 rpcz_store.cc:269] 0709 12:25:13.794909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.075408 (+280499us) service_pool.cc:224] Handling call
0709 12:25:16.717588 (+2642180us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b5400336b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.717763 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2023442048}
I20260709 12:25:16.710636 30063 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:16.721305 30036 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.747193 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:16.692047 29987 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.762660 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:16.773133 26198 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:16.774072 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.774233 29987 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.775310 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:16.764688 30112 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.755s	user 0.004s	sys 0.000s
I20260709 12:25:16.764926 26195 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:16.739845 30145 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:16.768069 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:16.767985 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.797927 26194 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:16.798831 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.799698 26194 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:16.799974 26129 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:25:16.771943 30036 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.802019 30036 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.805557 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.806933 29976 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.778033 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.809106 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.810068 30141 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:16.810557 26198 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:16.740089 29794 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.777938 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.776067 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.813454 26202 raft_consensus.cc:2376] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10. Re-sending same reply.
I20260709 12:25:16.790944 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.796847 30063 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:16.806843 29956 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:16.786132 26020 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.571s (SENT)
I20260709 12:25:16.817696 26019 leader_election.cc:400] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:16.821233 26019 leader_election.cc:403] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:25:16.819152 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141473us
I20260709 12:25:16.822216 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:16.740389 30126 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:16.822613 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.728588 30097 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:16.823725 30087 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.002s
I20260709 12:25:16.836766 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.837316 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.837643 30149 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:16.838419 30122 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 12:25:16.838891 30145 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:16.839733 30040 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:16.840679 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2740) took 1591 ms. Trace:
I20260709 12:25:16.841050 26127 rpcz_store.cc:276] 0709 12:25:15.249504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.599482 (+349978us) service_pool.cc:224] Handling call
0709 12:25:15.599830 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.921575 (+321745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.921595 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.945983 (+ 24388us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.091189 (+145206us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:16.091237 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.091320 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.837842 (+746522us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:16.840303 (+  2461us) raft_consensus.cc:1685] finished
0709 12:25:16.840322 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.840613 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82940,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":96600832,"thread_start_us":451727,"threads_started":2,"wal-append.queue_time_us":235803}
I20260709 12:25:16.842948 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.843096 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.843582 26127 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.844429 30063 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:16.845497 30063 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:16.848779 30147 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:16.852288 30136 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.852372 30145 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:16.854058 30149 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
W20260709 12:25:16.854432 26756 connection.cc:256] RPC call timeout handler was delayed by -1.4820470869999554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 12:25:16.854928 30124 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.697s	user 0.002s	sys 0.000s
I20260709 12:25:16.818665 30113 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.858028 26129 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:16.860838 30078 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.780s	user 0.008s	sys 0.003s
I20260709 12:25:16.863164 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.864300 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.864720 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.865417 26782 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:16.865428 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.866474 26790 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:16.867074 26046 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:16.865752 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.868592 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.864073 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:16.869356 26791 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:16.867081 30087 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.881639 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.884069 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.884768 26787 raft_consensus.cc:2376] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9. Re-sending same reply.
I20260709 12:25:16.885298 29931 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:16.883230 30123 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.721s	user 0.002s	sys 0.000s
I20260709 12:25:16.886713 26126 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:16.888520 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:16.889361 26789 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:16.889631 26123 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:16.888614 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.893573 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:16.892328 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.900457 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:16.902273 26051 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:25:16.909044 30147 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:16.909703 30147 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:16.909794 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.910547 26757 leader_election.cc:400] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11.
I20260709 12:25:16.910926 26757 leader_election.cc:403] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.910804 30147 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:16.911755 30040 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.915661 30040 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:16.916949 29910 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:16.917394 29910 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:16.918375 29910 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:16.858165 30113 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:16.928268 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2733) took 1683 ms. Trace:
I20260709 12:25:16.928671 26053 rpcz_store.cc:276] 0709 12:25:15.244494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.415797 (+1171303us) service_pool.cc:224] Handling call
0709 12:25:16.416093 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.473389 (+ 57296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.928011 (+454622us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b5400228b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:16.928195 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93686144}
I20260709 12:25:16.931080 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 449) took 1341 ms. Trace:
I20260709 12:25:16.931464 26054 rpcz_store.cc:276] 0709 12:25:15.589846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.918518 (+328672us) service_pool.cc:224] Handling call
0709 12:25:16.930929 (+1012411us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400131354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.931026 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382619392}
I20260709 12:25:16.932334 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:16.932895 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1481) took 2115 ms. Trace:
I20260709 12:25:16.933360 26200 rpcz_store.cc:276] 0709 12:25:14.817537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.730016 (+912479us) service_pool.cc:224] Handling call
0709 12:25:16.486868 (+756852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.493845 (+  6977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.932722 (+438877us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b5400335750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.932823 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699389824}
W20260709 12:25:16.932845 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1484) took 2102 ms (client timeout 2175 ms). Trace:
I20260709 12:25:16.898828 29693 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:16.934593 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.937034 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:16.934702 26197 rpcz_store.cc:269] 0709 12:25:14.830190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.112389 (+282199us) service_pool.cc:224] Handling call
0709 12:25:16.932700 (+1820311us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b5400217ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.932788 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348816768}
I20260709 12:25:16.936957 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1885) took 2316 ms. Trace:
I20260709 12:25:16.938365 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.938261 26201 rpcz_store.cc:276] 0709 12:25:14.620910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.794913 (+174003us) service_pool.cc:224] Handling call
0709 12:25:15.124329 (+329416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.156150 (+ 31821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.932324 (+1776174us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54002b5050. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:16.936897 (+  4573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756801792}
I20260709 12:25:16.939261 26046 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:16.939507 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:16.920092 30149 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:16.922112 29910 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.921060 29779 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:16.940217 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:16.924299 30130 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:16.949801 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.890400 30129 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:16.946269 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.944842 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.940572 30150 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.957500 26046 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:16.966118 30142 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:25:16.966423 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:16.967700 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.969065 30136 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.970362 30136 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:25:16.956327 30080 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.873s	user 0.007s	sys 0.013s
W20260709 12:25:16.971894 30080 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.889s	user 0.007s	sys 0.013s
W20260709 12:25:16.960021 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.937014 29693 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.922544 30037 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:16.978377 26127 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:16.987542 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:16.943177 29779 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:17.021248 30082 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:17.026671 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:16.951716 29945 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:17.051330 26789 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:17.025012 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:17.052012 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.080825 26788 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.061728 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:17.076308 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 185676us
W20260709 12:25:17.116537 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 96170.1us
I20260709 12:25:16.950968 30130 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:17.092502 26789 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:17.020494 29693 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.024296 26050 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:17.025039 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:16.963006 29944 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:17.120888 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:17.132370 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 186949us
I20260709 12:25:17.019940 30141 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.075866 29931 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.022378 30066 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:17.126246 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:17.146330 26050 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.019735 30126 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:17.018627 30087 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:17.023357 30113 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:17.132103 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107189us
I20260709 12:25:17.116406 30025 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:17.119680 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:17.149158 26788 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:17.149078 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:17.119524 29889 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:17.124897 30130 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:17.161247 26120 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.084157 30036 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:25:17.062830 30082 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.155745 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:17.141251 29907 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:17.153199 29877 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:17.167409 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47889us
I20260709 12:25:17.175266 26094 leader_election.cc:400] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13.
I20260709 12:25:17.175721 26094 leader_election.cc:403] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:17.141530 29944 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:17.160259 30025 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:17.190359 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:17.174631 30076 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:17.160024 30106 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:25:17.147845 30141 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.162122 30113 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.189478 30037 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:25:17.187059 29907 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:17.222929 29837 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.003s
I20260709 12:25:17.225067 29987 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:17.133076 29779 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.154668 29931 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.156641 30126 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:17.213496 26120 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:17.190397 29988 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:17.232151 30076 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:17.230930 26127 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.214355 30087 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.205262 30081 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.188310 30147 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:17.192092 29889 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:17.232476 29794 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.256642 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:17.354286 26120 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:17.355262 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:17.253738 30129 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:17.333516 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 125989us
I20260709 12:25:17.294701 29779 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.381072 29889 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:17.385479 30097 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:17.386230 30157 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:25:17.390537 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 858) took 1209 ms. Trace:
I20260709 12:25:17.391088 26783 rpcz_store.cc:276] 0709 12:25:16.181056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.188165 (+  7109us) service_pool.cc:224] Handling call
0709 12:25:17.390260 (+1202095us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:17.390427 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354597120}
I20260709 12:25:17.393328 30113 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:17.394784 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 217939us
I20260709 12:25:17.397511 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:17.401090 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:17.335822 30087 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.394302 30169 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.313225 29976 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:17.344780 26127 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:17.252874 29837 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:17.338616 30076 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:17.432504 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2740) took 2085 ms (client timeout 1715 ms). Trace:
I20260709 12:25:17.422749 26784 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:25:17.433269 26047 rpcz_store.cc:269] 0709 12:25:15.334569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.381628 (+2047059us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:17.419916 (+ 38288us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:25:17.446828 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:17.447789 26783 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:17.446844 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:17.448563 26120 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.453027 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:17.421079 30087 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.303480 30025 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:17.434396 30164 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:25:17.419269 30113 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:17.460748 26046 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:17.357286 30081 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.466105 26053 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:17.470767 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:17.413719 30106 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:25:17.365692 29794 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.488826 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:17.471740 26050 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:17.499958 26120 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:17.455637 30035 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:25:17.524421 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 494288us
I20260709 12:25:17.435271 30169 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:17.476763 30039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.307s	user 0.007s	sys 0.000s
I20260709 12:25:17.518119 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:17.429783 29693 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.555285 29693 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:17.472625 30097 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.275063 30066 consensus_queue.cc:1175] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 10 status { last_received { term: 9 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:25:17.556823 30154 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:25:17.533216 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.517190 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 339690us
W20260709 12:25:17.557902 30172 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:25:17.502419 30026 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:17.475220 30113 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:17.558561 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:17.560624 30113 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:17.561630 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:17.563212 30026 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:17.524085 30025 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.479296 30082 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:17.514598 30031 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.009s	sys 0.002s
I20260709 12:25:17.556459 30097 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:17.558524 30166 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:25:17.564613 29987 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:17.562543 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:17.571907 29794 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:17.568504 29907 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:17.570617 29944 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:17.571980 26054 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:17.577227 29837 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:17.578529 30125 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.406s	user 0.001s	sys 0.003s
I20260709 12:25:17.582226 30081 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:17.459820 30165 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:25:17.466355 30160 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 12:25:17.584786 30106 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:17.523597 30035 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:17.606204 30126 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:17.594437 30169 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:17.542160 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:17.540361 29889 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:17.545611 30039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.008s	sys 0.000s
W20260709 12:25:17.608081 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45626us
I20260709 12:25:17.666812 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:17.563980 30026 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:17.604465 30031 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.010s	sys 0.002s
I20260709 12:25:17.600090 29877 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:17.604974 29910 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.000s
W20260709 12:25:17.671850 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 274396us
I20260709 12:25:17.605931 30107 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:17.592751 29931 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:17.631299 30150 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.000s
I20260709 12:25:17.662376 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.674574 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:17.674690 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 198767us
I20260709 12:25:17.649509 30163 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:17.679342 26054 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:17.677152 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:17.653584 29794 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.676515 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.656401 30106 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:17.688333 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:17.688256 26787 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.649803 29907 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:25:17.717587 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 1991) took 2125 ms. Trace:
I20260709 12:25:17.718312 26128 rpcz_store.cc:276] 0709 12:25:15.592643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.596558 (+  3915us) service_pool.cc:224] Handling call
0709 12:25:15.596926 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.788476 (+191550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.788515 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.801180 (+ 12665us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.055856 (+254676us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:16.055891 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.055952 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:17.577781 (+1521829us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:17.703709 (+125928us) raft_consensus.cc:1685] finished
0709 12:25:17.703743 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.717496 (+ 13753us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85858,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"spinlock_wait_cycles":566295808,"thread_start_us":445425,"threads_started":2,"wal-append.queue_time_us":326215}
I20260709 12:25:17.707437 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:17.679986 30107 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.681641 30026 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:17.700418 30150 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11.
I20260709 12:25:17.704797 30106 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.778115 26202 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:17.780755 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.781563 26198 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:17.707835 30191 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.780802 26128 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:17.785329 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.785423 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:17.787670 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97985.7us
I20260709 12:25:17.789439 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.790302 26198 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:25:17.791086 26197 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:17.788753 26052 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:17.793995 26787 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:17.797992 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:17.707219 30126 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:17.799564 26054 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.796525 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.789243 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.799914 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.708600 29794 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.805534 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.806560 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.806674 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.707563 30195 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.823870 30195 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.792511 30191 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:17.725927 30163 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:25:17.809723 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:17.820659 30161 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:25:17.834221 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.826828 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:17.827642 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:17.828282 26201 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:17.807165 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.938766 26202 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:17.832563 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:17.802387 30199 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.795398 30194 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:17.802953 30126 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.795989 30165 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:25:17.837010 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:17.724993 30035 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.833141 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:17.840315 30191 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.936386 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:17.937350 26203 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:17.944046 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116447us
I20260709 12:25:17.946242 26047 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.956845 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.957546 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:17.830063 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:17.949244 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 116168us
I20260709 12:25:17.949945 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:17.947278 30130 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.773s	user 0.012s	sys 0.010s
I20260709 12:25:17.961887 30206 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:17.961679 30130 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.962463 30206 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:17.963397 30206 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:17.959698 26048 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:17.967844 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:17.969120 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:17.865993 29877 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
W20260709 12:25:17.949172 29988 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.003s
W20260709 12:25:17.960721 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 131634us
W20260709 12:25:17.941361 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 110038us
I20260709 12:25:17.941967 30081 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:17.866287 29931 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:25:17.976846 26782 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.978152 29794 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:17.979537 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:17.979905 26047 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:17.975029 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:17.946259 30126 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:18.021875 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46939.5us
W20260709 12:25:18.027577 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48114.2us
I20260709 12:25:18.046877 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.047746 26048 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:18.047142 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.049811 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:18.051149 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:17.981710 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 54301us
I20260709 12:25:18.019630 29988 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13.
I20260709 12:25:18.053620 29944 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:17.973387 30194 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:17.956501 30097 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:17.954923 29987 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 12:25:18.068732 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.069694 26054 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:18.069924 29944 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:18.070647 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.896410 30026 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:17.949414 30199 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.075157 30183 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:18.075646 30183 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:18.077543 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.078819 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:17.981338 30196 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:18.080649 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.981624 30081 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.081714 30081 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.063771 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.085135 26048 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:18.085816 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.070484 29944 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.086936 26194 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:18.085884 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:18.094000 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.080677 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.079911 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.090261 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1914) took 1178 ms. Trace:
I20260709 12:25:18.091300 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.095566 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:18.097321 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.102222 26120 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
W20260709 12:25:18.098924 29945 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.005s	sys 0.022s
I20260709 12:25:18.095315 26196 rpcz_store.cc:276] 0709 12:25:16.912149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.919780 (+  7631us) service_pool.cc:224] Handling call
0709 12:25:16.920190 (+   410us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:17.162093 (+241903us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:17.242463 (+ 80370us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.456276 (+213813us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.677998 (+221722us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:17.709319 (+ 31321us) log.cc:844] Serialized 78 byte log entry
0709 12:25:17.709514 (+   195us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:17.709556 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.709620 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.089960 (+380340us) raft_consensus.cc:1685] finished
0709 12:25:18.089987 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.090183 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139976,"prepare.queue_time_us":45321,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":474213,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":6434,"thread_start_us":584895,"threads_started":4,"wal-append.queue_time_us":268334}
I20260709 12:25:18.111065 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.111768 26194 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:18.112479 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.052986 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.125061 26202 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:18.126006 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.127790 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:18.080703 30159 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:18.075639 30199 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.137188 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.137744 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.138818 26048 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:18.111129 30076 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.000s
W20260709 12:25:18.131132 26018 connection.cc:256] RPC call timeout handler was delayed by -1.6778523040002256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 12:25:18.124496 29945 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.006s	sys 0.022s
W20260709 12:25:18.144649 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1036523040002066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
I20260709 12:25:18.136157 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.091828 30097 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:18.140491 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:18.123325 30182 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.139696 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.123185 29988 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.152259 26047 raft_consensus.cc:2376] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9. Re-sending same reply.
I20260709 12:25:18.130573 26203 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:18.153364 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.155833 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.154979 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.132566 30159 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.140594 29877 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.164572 30159 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:18.180465 30189 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:25:18.178049 29882 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.184463 30169 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:18.161873 29988 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.092581 30194 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:18.122928 30196 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.150554 30076 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.159356 30097 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:18.203042 30087 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.727s	user 0.022s	sys 0.000s
I20260709 12:25:18.207927 30195 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.163143 30182 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:18.224318 30103 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 2.172s	user 0.008s	sys 0.011s
I20260709 12:25:18.133080 30183 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.211602 30196 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.250072 30160 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:18.257352 30160 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:18.258285 30160 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.259585 30160 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:18.239490 30204 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:25:18.236555 30158 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.941s	user 0.004s	sys 0.000s
W20260709 12:25:18.237815 30168 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.796s	user 0.001s	sys 0.000s
I20260709 12:25:18.249935 30130 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.266721 30169 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:18.267282 30169 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:18.267347 30182 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.268662 30126 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.269896 30141 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.270795 30126 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.264298 30183 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.269903 30196 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.233222 30194 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.276367 30159 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.277413 30191 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.285478 30191 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.279771 30169 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.145534 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:18.322954 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 177263us
I20260709 12:25:18.323416 26047 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:18.325866 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1476) took 3294 ms. Trace:
I20260709 12:25:18.326644 26045 rpcz_store.cc:276] 0709 12:25:15.031477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.038287 (+  6810us) service_pool.cc:224] Handling call
0709 12:25:15.041225 (+  2938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.915941 (+874716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.324945 (+2409004us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b540022ae54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:18.325518 (+   573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995163520}
I20260709 12:25:18.330343 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:16.798224 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:18.338064 26045 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:18.342914 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1507) took 1410 ms. Trace:
I20260709 12:25:18.344619 26046 rpcz_store.cc:276] 0709 12:25:16.932386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.967449 (+ 35063us) service_pool.cc:224] Handling call
0709 12:25:18.342331 (+1374882us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.342645 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132102016}
I20260709 12:25:18.346843 26046 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:25:18.337145 26047 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:18.351351 26045 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:18.351712 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.352906 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2732) took 3126 ms. Trace:
I20260709 12:25:18.354588 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:18.144731 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.190042 26121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1990) took 3300 ms (client timeout 2856 ms). Trace:
W20260709 12:25:18.357409 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 1996) took 2756 ms (client timeout 3199 ms). Trace:
I20260709 12:25:18.358822 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:18.361419 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.149894 26196 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:18.363150 26195 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:18.363610 29837 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:18.358886 26122 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.368192 26049 rpcz_store.cc:276] 0709 12:25:15.226835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.970615 (+743780us) service_pool.cc:224] Handling call
0709 12:25:16.716094 (+745479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.725283 (+  9189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.352391 (+1627108us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b540006fc50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.352654 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1671090048}
W20260709 12:25:18.383854 26125 rpcz_store.cc:269] 0709 12:25:15.601044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.187025 (+585981us) service_pool.cc:224] Handling call
0709 12:25:18.356726 (+2169701us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54001c3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.357012 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774377856}
W20260709 12:25:18.391894 30113 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.002s
W20260709 12:25:18.402397 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2743) took 2708 ms (client timeout 2635 ms). Trace:
W20260709 12:25:18.403095 26123 rpcz_store.cc:269] 0709 12:25:15.693775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.180175 (+486400us) service_pool.cc:224] Handling call
0709 12:25:18.401725 (+2221550us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54001f54d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.402103 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604754176}
I20260709 12:25:18.249348 30106 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.406438 29961 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:18.406589 26129 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.394044 26051 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:18.407241 29961 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:18.393996 26121 rpcz_store.cc:269] 0709 12:25:14.884534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.046196 (+161662us) service_pool.cc:224] Handling call
0709 12:25:18.184356 (+3138160us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.184526 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1570770688}
I20260709 12:25:18.426491 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2750) took 1467 ms. Trace:
I20260709 12:25:18.426952 26121 rpcz_store.cc:276] 0709 12:25:16.959308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.425098 (+1465790us) service_pool.cc:224] Handling call
0709 12:25:18.425731 (+   633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.426107 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.426131 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.426187 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:18.426222 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.426275 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.426384 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:18.428476 29877 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:18.430426 29877 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.433488 29944 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.436446 30167 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:18.437520 30167 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.443610 30097 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.445820 29988 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.446308 26125 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:18.446408 30143 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:18.448820 30169 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.452419 26128 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.454957 30033 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.224032 30107 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.456939 30113 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.458905 30035 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:18.461206 30036 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:18.464668 30191 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:18.465541 30160 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:18.470499 30176 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:18.556294 30176 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.472451 30076 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.473151 30080 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:18.475749 29938 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.560708 30080 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:18.562559 30080 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.478511 29837 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.564064 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:18.565130 30080 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.564914 29837 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.566200 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:18.567059 26789 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:18.567166 30107 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.480521 29931 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.568749 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:18.569708 26784 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:18.570950 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:18.571350 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1504) took 1641 ms. Trace:
I20260709 12:25:18.571961 26787 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:18.571882 26053 rpcz_store.cc:276] 0709 12:25:16.929791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.935431 (+  5640us) service_pool.cc:224] Handling call
0709 12:25:16.935747 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.474097 (+538350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.570794 (+1096697us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:18.571112 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199852288}
I20260709 12:25:18.490759 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 956) took 4427 ms. Trace:
I20260709 12:25:18.494300 29961 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.573803 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:18.574424 30113 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:18.493368 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2746) took 1535 ms (client timeout 1518 ms). Trace:
W20260709 12:25:18.575065 26127 rpcz_store.cc:269] 0709 12:25:16.958030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.023891 (+ 65861us) service_pool.cc:224] Handling call
0709 12:25:18.492921 (+1469030us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54001c5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.493156 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398955264}
I20260709 12:25:18.496696 29707 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.576622 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:18.576927 30210 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:18.218498 30087 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:18.501677 26121 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:18.573835 29961 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.578128 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 599) took 1262 ms. Trace:
I20260709 12:25:18.578649 26121 rpcz_store.cc:276] 0709 12:25:17.315543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.427929 (+1112386us) service_pool.cc:224] Handling call
0709 12:25:18.501150 (+ 73221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.577804 (+ 76654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.578044 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:18.581138 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:18.581295 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:18.581462 30129 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:18.582568 26200 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:18.583135 30169 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:18.582530 30129 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.583546 29907 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:18.500270 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 982) took 1634 ms. Trace:
I20260709 12:25:18.584410 26788 rpcz_store.cc:276] 0709 12:25:16.865423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.866285 (+   862us) service_pool.cc:224] Handling call
0709 12:25:18.499704 (+1633419us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.499935 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435098368}
I20260709 12:25:18.573271 26786 rpcz_store.cc:276] 0709 12:25:14.063047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.063330 (+   283us) service_pool.cc:224] Handling call
0709 12:25:14.801073 (+737743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.959324 (+158251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.959358 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.959900 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.073635 (+113735us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:15.078133 (+  4498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.078213 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.860314 (+1782101us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:16.883636 (+ 23322us) raft_consensus.cc:1685] finished
0709 12:25:16.883658 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.490230 (+1606572us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54006bf8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.490484 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75381,"raft.queue_time_us":76235,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":2557,"spinlock_wait_cycles":1707089152,"thread_start_us":195753,"threads_started":2,"wal-append.queue_time_us":120241,"wal-append.run_cpu_time_us":18104,"wal-append.run_wall_time_us":2847527}
I20260709 12:25:18.512878 30167 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.529023 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2002) took 1754 ms. Trace:
W20260709 12:25:18.523571 26052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1505) took 1593 ms (client timeout 1649 ms). Trace:
I20260709 12:25:18.586418 26129 rpcz_store.cc:276] 0709 12:25:16.774246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.858938 (+ 84692us) service_pool.cc:224] Handling call
0709 12:25:18.405968 (+1547030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.409640 (+  3672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.528690 (+119050us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54001f54d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.528917 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3553945984}
W20260709 12:25:18.586756 26052 rpcz_store.cc:269] 0709 12:25:16.930145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.930671 (+   526us) service_pool.cc:224] Handling call
0709 12:25:18.522894 (+1592223us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540008e3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.523250 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713508096}
I20260709 12:25:18.536638 26051 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:18.541718 29987 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:18.483470 29910 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:25:18.577929 30210 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.588411 29987 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.542759 30106 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.463836 30159 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.591168 29987 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.591375 30210 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.591732 30160 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.592151 29907 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:18.593104 29907 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:18.543435 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 594) took 2013 ms. Trace:
I20260709 12:25:18.582679 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2749) took 1623 ms. Trace:
I20260709 12:25:18.572821 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:18.507122 29693 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:18.599006 29693 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:18.595166 26128 rpcz_store.cc:276] 0709 12:25:16.959019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.787392 (+828373us) service_pool.cc:224] Handling call
0709 12:25:17.805447 (+ 18055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.581128 (+775681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.582291 (+  1163us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54001f95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:18.582580 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385268992}
I20260709 12:25:18.600602 29693 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.602020 29693 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:18.603004 25952 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:18.604849 25952 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:18.606552 25952 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 5 to 8, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 8 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.553491 30036 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:18.612053 30075 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.597472 26054 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:18.568779 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 593) took 2150 ms. Trace:
W20260709 12:25:18.560024 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.549314 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 597) took 1586 ms. Trace:
I20260709 12:25:18.614670 26125 rpcz_store.cc:276] 0709 12:25:16.959597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.445364 (+1485767us) service_pool.cc:224] Handling call
0709 12:25:18.445853 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.543908 (+ 98055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.546244 (+  2336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:18.569278 29931 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.598064 30191 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.577471 26785 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:18.595062 26122 rpcz_store.cc:276] 0709 12:25:16.530428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.848275 (+317847us) service_pool.cc:224] Handling call
0709 12:25:18.358236 (+1509961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.396990 (+ 38754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.543224 (+146234us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54001c3c50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.543373 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3471412608}
I20260709 12:25:18.613299 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2745) took 1655 ms. Trace:
I20260709 12:25:18.614403 26126 rpcz_store.cc:276] 0709 12:25:16.418683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.724782 (+306099us) service_pool.cc:224] Handling call
0709 12:25:16.742780 (+ 17998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.887120 (+144340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.568104 (+1680984us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54001da954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:18.568586 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320250112}
I20260709 12:25:18.579984 30087 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.638692 26120 rpcz_store.cc:276] 0709 12:25:16.957755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.517786 (+560031us) service_pool.cc:224] Handling call
0709 12:25:18.612845 (+1095059us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54000b95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.613094 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199814400}
I20260709 12:25:18.614078 30143 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:18.594226 29907 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.672086 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:18.673910 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.612169 30036 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.676775 29907 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.677368 29511 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.677484 30036 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.678068 30081 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.679000 30165 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.681643 30087 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.682297 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 405924us
I20260709 12:25:18.682615 26199 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.681577 30033 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.682868 29987 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.683545 26790 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:18.684789 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 255346us
I20260709 12:25:18.689802 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.691092 30081 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.691284 30129 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.669895 30160 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.697873 30003 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:18.700511 30080 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:18.701673 29931 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.701426 30165 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.703142 30004 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:18.704711 30066 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:18.704499 30087 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.707273 29882 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:18.707926 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.709345 30143 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.712358 30203 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:18.715054 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:18.747157 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59226.1us
I20260709 12:25:18.745419 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.678112 29511 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.767057 30087 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.768002 29882 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:18.768579 29882 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:18.768447 30004 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.770046 29882 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:18.772447 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.773033 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.785511 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 161513us
I20260709 12:25:18.771987 30004 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.791930 30159 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:18.783299 25952 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:18.794695 29511 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.795496 29511 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:18.797283 26785 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:18.798480 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:18.799690 30076 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:18.801783 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:18.802729 26786 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 7: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:18.802606 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.803452 26200 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:18.783733 29987 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.805823 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:18.806401 29987 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.787191 30203 consensus_queue.cc:1243] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:18.785275 30033 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.811388 30033 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:18.812825 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 97929us
W20260709 12:25:18.815622 30025 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.209s	user 0.012s	sys 0.008s
I20260709 12:25:18.817101 30191 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:18.818574 26790 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:18.820439 30080 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.821132 30080 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:18.821645 29779 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.410s	user 0.005s	sys 0.016s
I20260709 12:25:18.823206 26201 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:18.823827 26126 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.824086 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:18.815454 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:18.824805 26786 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:18.838272 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.841432 26198 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:18.841688 26128 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.827028 29882 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:18.844219 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.846365 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.847707 26129 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:18.834172 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:18.824800 30163 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:18.820106 30035 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:18.833869 30081 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.858279 30225 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.858978 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53411.6us
I20260709 12:25:18.859375 30225 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:18.858873 30081 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.856514 30087 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:18.861876 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:18.865082 30224 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.866816 30147 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.868499 30184 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.875715 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51811.2us
I20260709 12:25:18.881199 26785 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:18.882089 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:18.881445 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:18.887686 30080 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.885601 26120 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:18.854928 30163 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:18.890926 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:18.894615 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 79278.8us
I20260709 12:25:18.895394 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1513) took 1093 ms. Trace:
W20260709 12:25:18.897222 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.905227 30081 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.888216 29779 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.906176 30176 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.909693 30165 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.923323 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.920055 26198 rpcz_store.cc:276] 0709 12:25:17.801646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.804181 (+  2535us) service_pool.cc:224] Handling call
0709 12:25:18.895074 (+1090893us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002b5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:18.895262 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316179200}
I20260709 12:25:18.923498 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.918685 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:18.927214 26128 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:18.929492 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.931334 26126 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:18.933293 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.934213 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:18.935334 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.940744 26052 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:18.946583 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.928874 29961 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:18.949347 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:18.950790 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:18.949560 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.953980 26195 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 7: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:18.949337 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.959168 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.960525 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:18.959410 26051 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:18.962385 26129 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:18.967911 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:18.968703 26126 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:18.968907 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:18.970428 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:18.971316 30033 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:18.971704 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:18.971990 29877 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:18.972441 29877 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.972818 29877 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:18.973135 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.974915 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.975605 26124 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:18.976588 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.977389 26123 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:18.982019 26128 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:18.982733 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:18.986666 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.953362 30147 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.950440 30224 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:18.993276 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:18.996707 29837 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:18.997087 26124 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:25:18.997553 26124 pending_rounds.cc:85] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 6
I20260709 12:25:18.997874 26124 pending_rounds.cc:107] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:25:18.998173 26124 raft_consensus.cc:2889] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.998450 30225 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:18.996780 29961 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.999413 26200 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:18.999617 26045 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:18.999454 29961 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.002009 29961 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.002144 30147 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.002859 30160 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:19.003355 30160 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:19.004537 26123 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:19.004398 30160 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.005789 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.006313 30160 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.006547 26123 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:19.007704 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:18.938072 30184 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.010833 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:18.926241 30035 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.011616 26128 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:19.012100 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:19.013319 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.015033 26129 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:19.015996 26120 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:19.018049 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.012997 26126 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 7.
I20260709 12:25:19.027998 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:19.029206 26045 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:19.032465 26050 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:19.036060 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:19.035778 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 174250us
I20260709 12:25:19.008620 29931 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:19.042284 26123 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:19.034716 25952 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 2 to 5, term changed from 5 to 6, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:19.032491 30224 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.032557 30184 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.048808 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:19.050277 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:19.051183 26123 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:19.052160 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.042774 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:19.055505 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.057718 26129 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:19.057665 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 101068us
I20260709 12:25:19.024879 30036 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:19.061169 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:19.063647 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:19.070891 26128 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:19.071722 26200 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:19.057607 29961 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:19.076514 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:19.057834 29944 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:19.072994 25952 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 2 to 9, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 9 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.060878 30106 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:19.087787 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:19.091230 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98148.9us
I20260709 12:25:19.092188 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.092916 26128 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:19.094491 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:19.095713 30026 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:19.096421 30026 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:19.095938 25952 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:19.097597 30026 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.099857 26129 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:19.100628 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:19.100752 29511 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:19.101351 26050 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.101825 26125 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:19.103130 26125 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:19.104370 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 132750us
I20260709 12:25:19.105543 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.106794 30219 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:19.107228 30219 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.107623 30219 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.112768 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:19.113408 30227 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:25:19.113806 30036 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:19.081012 29944 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:19.094025 26123 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:19.118232 26125 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:19.036451 30163 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.093472 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:19.140516 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.080871 29961 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:19.141285 26201 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.121783 30036 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.090492 30004 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.144047 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.097617 25952 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:25:19.143131 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.144608 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.149755 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:19.143574 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.153513 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:19.153868 26128 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:19.120208 26126 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:19.149353 26197 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:19.157756 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:19.085528 29877 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.153323 26786 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
W20260709 12:25:19.148591 29693 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.546s	user 0.015s	sys 0.000s
I20260709 12:25:19.146813 30036 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.174758 26123 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:25:19.177366 26123 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 7
I20260709 12:25:19.146261 29961 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.180512 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:19.181342 26789 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 12:25:19.181740 26789 pending_rounds.cc:85] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 10
I20260709 12:25:19.181948 26789 pending_rounds.cc:107] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 12:25:19.182196 26789 raft_consensus.cc:2889] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.143028 29511 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:25:19.159813 30183 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.007s
I20260709 12:25:19.186548 26784 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:19.190614 26128 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:19.194190 26124 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:19.183308 25952 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 9, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 9 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:19.193429 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:19.195914 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 186945us
W20260709 12:25:19.196524 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52545us
I20260709 12:25:19.197584 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:19.201050 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62887.6us
I20260709 12:25:19.203513 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:19.177634 26123 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.8
I20260709 12:25:19.205619 26123 raft_consensus.cc:2917] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Aborting config change with OpId 10.8: Aborted: Op aborted by new leader
I20260709 12:25:19.206073 26123 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.178037 30081 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:19.196157 25949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 285) took 25087 ms (client timeout 15000 ms). Trace:
I20260709 12:25:19.215939 26195 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:19.192453 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:19.149310 29931 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:25:19.177489 30129 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.222158 29511 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:19.160882 30075 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 12:25:19.224871 30229 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:25:19.225737 29907 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:19.226457 30037 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.227308 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:19.229499 30033 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:19.231349 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:19.236374 30040 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:25:19.236779 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 611) took 1148 ms. Trace:
I20260709 12:25:19.238946 26129 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:19.241986 29693 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:19.244743 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:19.247758 29877 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.248807 26200 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.255065 26052 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:19.216033 25949 rpcz_store.cc:269] 0709 12:24:54.108863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.109153 (+   290us) service_pool.cc:224] Handling call
0709 12:25:19.195846 (+25086693us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:10.417514 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:10.418398 (+   884us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:10.418456 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:25:10.418481 (+    25us) tablet.cc:662] Decoding operations
0709 12:25:10.419707 (+  1226us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:10.419795 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:10.419821 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:25:10.564777 (+144956us) tablet.cc:701] Row locks acquired
0709 12:25:10.564828 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:25:10.565021 (+   193us) write_op.cc:270] Start()
0709 12:25:10.565188 (+   167us) write_op.cc:276] Timestamp: P: 1783599910564941 usec, L: 0
0709 12:25:10.565210 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:25:10.566079 (+   869us) log.cc:844] Serialized 5294 byte log entry
0709 12:25:11.448554 (+882475us) op_driver.cc:464] REPLICATION: finished
0709 12:25:12.454657 (+1006103us) write_op.cc:301] APPLY: starting
0709 12:25:12.454801 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:12.455212 (+   411us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:12.455390 (+   178us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:12.457557 (+  2167us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:12.458056 (+   499us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:25:12.458139 (+    83us) write_op.cc:312] APPLY: finished
0709 12:25:12.458685 (+   546us) log.cc:844] Serialized 95 byte log entry
0709 12:25:13.690369 (+1231684us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:13.690512 (+   143us) write_op.cc:454] Released schema lock
0709 12:25:13.690587 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:13.691040 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:13.691068 (+    28us) write_op.cc:454] Released schema lock
0709 12:25:14.863816 (+1172748us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
Metrics: {"mutex_wait_us":3262206,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":1005724,"apply.run_cpu_time_us":45673,"apply.run_wall_time_us":1236395,"num_ops":13,"prepare.queue_time_us":524529,"prepare.run_cpu_time_us":35987,"prepare.run_wall_time_us":303625,"raft.queue_time_us":429507,"raft.run_cpu_time_us":59129,"raft.run_wall_time_us":1344339,"replication_time_us":883131,"spinlock_wait_cycles":11793152,"thread_start_us":3114248,"threads_started":5,"wal-append.queue_time_us":1456468,"wal-append.run_cpu_time_us":51689,"wal-append.run_wall_time_us":3783457}]]}
I20260709 12:25:19.268771 26122 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:25:19.249469 30035 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.236083 30194 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.293651 26124 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:19.269464 25952 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 5 to 8, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 8 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:19.294370 26123 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:19.322921 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 95734.4us
I20260709 12:25:19.324218 26196 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.325649 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.330039 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.332347 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.285578 26050 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:19.296301 26197 rpcz_store.cc:276] 0709 12:25:18.088441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.093694 (+  5253us) service_pool.cc:224] Handling call
0709 12:25:19.236521 (+1142827us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54002b4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:19.236701 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274258176}
W20260709 12:25:19.280946 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.309897 30176 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:19.308594 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108617us
I20260709 12:25:19.253087 30143 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.312314 30040 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:19.315622 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:19.316468 30224 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:19.315900 30147 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:19.276227 30163 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.335429 25952 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 8, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 8 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:25:19.339983 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.345634 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:19.347981 26788 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.348627 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:19.349258 26124 raft_consensus.cc:2376] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10. Re-sending same reply.
I20260709 12:25:19.330470 30087 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:19.389189 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.352407 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.389784 25952 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.398212 26122 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:19.413627 25952 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.310261 30037 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.343396 29907 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.341434 30081 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:19.339833 30129 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:19.434556 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 203268us
I20260709 12:25:19.339023 30040 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:19.330188 29693 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.455230 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:19.457640 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.345633 30035 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:19.324950 30194 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.468986 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:19.462586 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 147135us
I20260709 12:25:19.322031 29944 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.417443 30176 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.341256 29877 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 12:25:19.416920 30184 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:19.475519 30050 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:19.516662 30050 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:19.517939 30050 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.452951 30081 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.519003 30081 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:19.519829 29976 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:19.482583 30184 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:19.520816 30184 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.526132 30066 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:19.526620 30066 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:19.528626 29907 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.531651 29907 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:19.532637 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 335133us
I20260709 12:25:19.485984 30176 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.533718 26123 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:19.486421 29837 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.487452 30040 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.489706 29931 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.491238 29961 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.493974 26203 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:19.495360 29877 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.417069 30143 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.506129 29693 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:19.475914 30033 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:25:19.472983 30183 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:19.507442 30160 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
I20260709 12:25:19.509677 26201 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:19.397684 30087 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.571422 30087 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.573553 29961 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.575536 30242 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:19.512569 30224 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:19.576551 30242 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:19.576924 30242 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:19.512770 30147 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.577945 30242 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.578763 30147 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:19.579289 30242 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.515156 30037 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.579259 30147 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:19.520275 29976 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:19.482470 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.533059 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.458423 30129 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:19.567260 30080 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:19.583065 30147 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.584408 30129 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.584909 30080 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:19.386541 30075 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:25:19.585644 30080 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.586257 30080 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.527557 30066 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:19.551275 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 367030us
I20260709 12:25:19.586853 30040 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:19.587913 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.591511 30087 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.555572 26788 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:19.563879 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:19.565739 26196 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:19.561527 25952 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.547880 30176 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:25:19.585389 30147 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.586601 30037 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:19.592370 30066 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.563359 30225 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:19.595830 26200 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 12:25:19.599129 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44861.5us
I20260709 12:25:19.596800 30037 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:19.570554 29693 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:19.591641 30219 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.568639 30183 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.585660 30129 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.559903 29877 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.558868 29931 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.643885 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:19.633966 25952 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 4 to 7, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:19.647249 26125 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:19.598721 30225 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:19.648326 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:19.632095 29907 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:19.631116 30004 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:19.655226 29907 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.655941 30243 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:19.583009 29976 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.660223 30143 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.647521 30037 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.594923 29837 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.651302 30176 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:19.663287 29931 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.666352 26122 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 12:25:19.642123 29961 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:19.678519 29976 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.670626 29907 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.681886 30225 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:19.683372 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.686211 29837 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.687265 29511 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:19.687820 29511 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:19.653545 25952 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 3 to 7, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 7 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.642450 30194 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.631989 30183 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.680830 26122 pending_rounds.cc:85] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 10
I20260709 12:25:19.693982 26122 pending_rounds.cc:107] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 12:25:19.694308 26122 raft_consensus.cc:2889] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:19.692467 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51064.3us
I20260709 12:25:19.695297 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.688334 29511 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:19.671684 30176 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:19.702582 30225 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.704494 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:19.694169 30243 consensus_queue.cc:1243] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:19.715513 26018 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.641s (SENT)
I20260709 12:25:19.720781 30087 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.705114 30033 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.724278 30033 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.726441 30033 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.697607 29907 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.723558 29877 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.734301 25952 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 4 to 9, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 9 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:19.707684 30036 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.737094 30037 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:19.759115 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 265474us
W20260709 12:25:19.765347 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65464.4us
W20260709 12:25:19.760408 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 248286us
I20260709 12:25:19.766271 26128 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:19.770556 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.765681 25952 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 6 to 9, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 9 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:19.789819 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 449922us
W20260709 12:25:19.792733 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 310473us
I20260709 12:25:19.809497 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.757682 30184 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:19.820168 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.821183 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 341136us
I20260709 12:25:19.779076 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:19.745222 30176 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.771735 30219 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.824138 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.790913 29877 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.833564 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:19.839747 26785 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.839514 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:19.840412 26789 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:19.836318 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.842175 26788 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:19.843418 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:19.844219 26786 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:19.810524 30035 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:19.850376 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 204680us
I20260709 12:25:19.849749 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.832583 30033 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.792866 30143 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 12:25:19.860186 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 82820.7us
I20260709 12:25:19.861943 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:19.845611 30194 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.863511 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 517700us
I20260709 12:25:19.867404 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:19.867798 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.811187 29837 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:25:19.858235 30250 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.854840 30075 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:19.894917 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.896381 30248 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:25:19.900720 30035 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.904388 30251 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:25:19.904507 30040 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:19.905208 30251 consensus_queue.cc:1243] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:19.906728 30147 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.907361 30147 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.868366 30176 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.908177 29976 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:19.908797 29976 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.910938 29976 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:19.911458 29976 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:19.911801 29976 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:19.912463 30249 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:19.913779 30219 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:19.914373 30219 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:19.915150 30081 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:19.918504 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107291us
I20260709 12:25:19.921584 26789 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:19.920280 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.851353 29877 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:19.924435 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44551.7us
I20260709 12:25:19.926049 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:19.870779 30050 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:19.872489 29961 consensus_queue.cc:1243] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:19.853122 30252 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:25:19.933858 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 229441us
W20260709 12:25:19.939035 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.915489 30219 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.865469 30033 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:19.946008 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 89527us
I20260709 12:25:19.931227 30004 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.948583 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:19.871496 30194 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.947016 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:19.932950 30026 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.959095 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:19.932618 30050 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:19.949052 29961 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.965427 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:19.971336 29511 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:19.985528 29511 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:19.965691 29976 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.042009 29976 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:20.042083 30143 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:19.968420 30004 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.045046 30240 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.552s	user 0.006s	sys 0.000s
I20260709 12:25:20.047221 30004 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:20.048209 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 168041us
I20260709 12:25:20.049973 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:20.051013 30147 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.051126 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:19.957437 30036 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.048702 30254 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:20.055696 30251 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:20.059968 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.970355 30050 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.929525 30250 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:19.962550 30253 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.067919 26125 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.068460 30249 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:20.049024 29931 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:20.076185 29931 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:20.075942 30266 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:20.053527 30252 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:20.069901 29511 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 12:25:20.079821 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.070629 30251 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:20.066093 26126 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:20.056847 30075 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:20.070190 29693 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:20.070051 30255 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:20.077857 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 112670us
I20260709 12:25:20.071902 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1523) took 1112 ms. Trace:
I20260709 12:25:20.085927 30225 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:20.089282 30260 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:20.090991 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.077329 29837 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.071131 30269 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:25:19.972165 30087 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.095587 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.059942 30254 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:20.101019 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 149351us
I20260709 12:25:20.077754 30184 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.079742 30252 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.126060 30252 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.128422 30267 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:20.128894 30267 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:20.129335 29944 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.130234 30250 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.105391 30160 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:20.130932 30160 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:20.122787 30254 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.131923 30160 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.133054 30160 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.122799 29837 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.136998 30252 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.138860 29837 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.120383 30087 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:20.109743 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:20.140374 30255 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.141227 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:20.141975 26120 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:20.141490 29931 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.139222 30252 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.142706 30255 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.142834 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:20.143195 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.143847 26782 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:20.110685 30251 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.111693 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:19.971891 30026 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.147354 30080 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
I20260709 12:25:20.147922 26126 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:20.149142 30026 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.149533 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.150269 26787 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:20.150406 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2018) took 1099 ms. Trace:
I20260709 12:25:20.150816 26128 rpcz_store.cc:276] 0709 12:25:19.051366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.216208 (+164842us) service_pool.cc:224] Handling call
0709 12:25:19.228130 (+ 11922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.766790 (+538660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.150161 (+383371us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:20.150346 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237039488}
I20260709 12:25:20.151552 26787 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:20.151495 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.152608 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.152783 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.153247 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.153591 26791 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:20.154686 26791 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:20.119031 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 175581us
I20260709 12:25:20.155812 26791 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEARNER]: Advancing to term 10
I20260709 12:25:20.140077 30087 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.158357 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:20.158993 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:20.119086 26784 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.160703 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.164471 30066 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:20.164530 26786 raft_consensus.cc:2376] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8. Re-sending same reply.
I20260709 12:25:20.166214 26786 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:20.166074 30066 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:20.167486 30066 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:20.081779 30266 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:20.107048 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 249019us
I20260709 12:25:20.099306 26199 rpcz_store.cc:276] 0709 12:25:18.954377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.957085 (+  2708us) service_pool.cc:224] Handling call
0709 12:25:19.045951 (+ 88866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.046334 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.046359 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:19.046955 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:25:19.253716 (+206761us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:19.253752 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.253820 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.888345 (+634525us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:20.052146 (+163801us) raft_consensus.cc:1685] finished
0709 12:25:20.052164 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.060576 (+  8412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94400256,"thread_start_us":206388,"threads_started":1,"wal-append.queue_time_us":341926}
I20260709 12:25:20.109130 30081 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.129791 30267 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.113472 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.074713 30253 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:20.142846 29931 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.174052 30033 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.152791 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.169135 30249 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.215806 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 102464us
I20260709 12:25:20.217409 26787 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:20.218951 26200 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:20.121251 29693 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:20.121575 30272 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:20.199772 30269 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.104861 30225 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:20.237258 30225 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.198920 26784 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:20.239370 26788 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:20.206779 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.249001 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:20.251423 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:20.249020 30066 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.213574 26782 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:20.253731 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.198655 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:20.226634 26020 leader_election.cc:400] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:20.254474 26020 leader_election.cc:403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.199461 26785 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:20.255327 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:20.227170 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.127454 30194 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:20.147810 30080 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.208674 30183 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.003s
I20260709 12:25:20.264729 30183 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.231751 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:20.268373 30250 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:20.269487 26782 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:20.235527 30050 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.205410 30075 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.256138 29511 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:20.271584 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:20.260758 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 63623us
I20260709 12:25:20.252701 30066 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.261408 26784 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:20.262220 26789 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.262754 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:20.262919 30080 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.263563 26124 raft_consensus.cc:2368] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10.
I20260709 12:25:20.268468 30262 raft_consensus.cc:2764] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:20.326295 26199 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.270155 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:20.324407 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.332911 26126 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
W20260709 12:25:20.298560 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.268401 30183 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.268139 26122 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:25:20.232695 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 172226us
I20260709 12:25:20.201390 30266 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.208945 30253 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.294533 29511 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:20.259639 26790 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:20.327929 30087 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.342020 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.345191 26126 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:25:20.365130 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.359089 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 609) took 1033 ms. Trace:
I20260709 12:25:20.351964 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:20.269569 29837 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.232507 30033 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:20.366562 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.417708 26784 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.326339 29961 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:20.418599 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.328150 30250 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:20.229326 30272 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:20.356525 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62327.5us
I20260709 12:25:20.297801 30081 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:20.368542 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81701us
I20260709 12:25:20.446743 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.394474 29511 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.395591 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.399015 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.293558 30106 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:20.453094 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:20.274847 30163 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.923s	user 0.013s	sys 0.000s
I20260709 12:25:20.409118 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.411233 26789 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:25:20.235899 29693 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.329879 29907 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:20.455711 26789 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 7
I20260709 12:25:20.331120 30267 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.460870 26789 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.8
I20260709 12:25:20.350263 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.327693 30050 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:20.403218 26123 rpcz_store.cc:276] 0709 12:25:19.325847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.548774 (+222927us) service_pool.cc:224] Handling call
0709 12:25:19.563571 (+ 14797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.874083 (+310512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.874149 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:19.922944 (+ 48795us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.051641 (+128697us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:20.051682 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.051750 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.358804 (+307054us) raft_consensus.cc:1685] finished
0709 12:25:20.358830 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.359010 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14351,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":329494,"threads_started":2,"wal-append.queue_time_us":325669}
I20260709 12:25:20.416366 26196 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:20.464994 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.368253 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:20.466892 26121 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:20.419286 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 382366us
I20260709 12:25:20.419464 26785 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:20.422179 30087 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.422499 30066 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:20.467873 26757 leader_election.cc:400] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10.
I20260709 12:25:20.468498 26757 leader_election.cc:403] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.428473 30033 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:20.428764 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76901us
I20260709 12:25:20.415019 26791 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:20.431689 30225 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.415853 30274 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:20.393270 29931 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.432461 30026 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:20.436957 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2768) took 1113 ms. Trace:
W20260709 12:25:20.440416 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.451098 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.364914 30266 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.453073 26127 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.338353 30255 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:25:20.457638 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.413288 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:20.461498 26782 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:20.461726 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:20.452310 30242 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.873s	user 0.011s	sys 0.004s
I20260709 12:25:20.461189 26789 raft_consensus.cc:2917] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Aborting config change with OpId 10.8: Aborted: Op aborted by new leader
I20260709 12:25:20.521735 26789 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:20.452536 30272 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.524649 30250 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.293630 29877 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.459509 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:20.460638 29907 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:20.483397 26784 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:20.528352 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70833.9us
I20260709 12:25:20.274178 30269 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.499027 26129 rpcz_store.cc:276] 0709 12:25:19.323606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.323763 (+   157us) service_pool.cc:224] Handling call
0709 12:25:20.436744 (+1112981us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002c9cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:20.436901 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458729216}
I20260709 12:25:20.456005 30080 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:20.463182 30249 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:20.532356 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:20.483345 30160 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:20.517443 26196 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:20.420156 30254 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.528465 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58703.3us
W20260709 12:25:20.538147 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.538213 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:20.445765 29961 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.528411 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44362.2us
I20260709 12:25:20.518210 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.484596 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:20.540980 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.530185 26199 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:20.473803 30081 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:20.536975 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:20.538841 26785 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.539312 26200 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:20.492358 30274 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:20.541405 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.543628 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.544194 26201 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:20.545089 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:20.545568 26054 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:20.497885 30026 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:20.547230 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:20.547392 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:20.551126 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.542273 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:20.551216 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.548041 30274 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:20.555248 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.554510 26053 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:20.553138 26198 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:20.556376 26120 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:20.537439 30087 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:20.557971 26195 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:20.552039 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:20.557657 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:20.560830 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:20.549861 30129 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.916s	user 0.013s	sys 0.000s
I20260709 12:25:20.562351 26047 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:25:20.563704 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 118768us
I20260709 12:25:20.564674 26126 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:20.557734 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:20.531934 30263 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10.
I20260709 12:25:20.567102 26046 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:20.563632 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:20.491003 30270 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:25:20.565171 26196 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:20.574631 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:20.548391 30249 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:20.577567 26199 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:20.530541 29931 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.578572 26049 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:20.553536 30080 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:20.552655 30276 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:20.555051 30160 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:20.610759 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.610157 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:20.614228 26203 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:25:20.590631 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.582026 26783 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:20.616118 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:20.621574 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 152399us
W20260709 12:25:20.602927 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.098s (SENT) [suppressed 28 similar messages]
I20260709 12:25:20.625183 26791 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:20.621763 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:20.573586 30087 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:20.577149 30037 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.816s	user 0.010s	sys 0.001s
W20260709 12:25:20.638572 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57743.5us
I20260709 12:25:20.661217 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:20.646771 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.652590 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:20.653860 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.663368 26203 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:20.560213 30278 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:20.656778 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.664374 26196 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:20.608779 30035 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:20.583670 30194 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.579602 29907 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.666069 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.673491 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:20.688742 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:20.644490 29944 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:20.637926 30129 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:20.678731 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:20.703263 26203 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:20.654145 30176 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:20.584051 30075 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.622985 30276 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:20.650374 30260 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.658383 30270 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.706228 26196 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:20.670099 30035 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:20.693722 30037 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:20.714290 30129 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.729975 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:20.704684 29944 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:20.614730 30033 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.696777 30264 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:20.732324 30066 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:20.732692 30287 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:25:20.733618 29961 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.708806 30176 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:20.706353 30194 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.700661 30081 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:20.737187 30269 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:20.738593 30289 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:20.739876 30254 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:20.655071 30253 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.667117 29877 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:20.731567 29931 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:20.722523 30050 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:25:20.696511 30080 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:20.848992 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.749745 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:20.861121 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.700109 29837 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
I20260709 12:25:20.887557 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:20.930001 26121 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
W20260709 12:25:20.839924 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.907217 26198 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:20.907018 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:20.901911 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.897624 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 175092us
W20260709 12:25:20.933636 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.933772 26196 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
W20260709 12:25:20.934321 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.935775 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.940791 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 244296us
W20260709 12:25:20.942070 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 303815us
I20260709 12:25:20.942950 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:20.948422 30219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.000s	sys 0.013s
W20260709 12:25:20.948869 30219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.000s	sys 0.013s
I20260709 12:25:20.947055 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:20.991901 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.997963 26121 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:25:20.938280 26194 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:20.723152 30250 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:21.001349 26050 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:21.001118 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:20.804044 30147 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:20.710405 30276 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:20.804072 30283 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 12:25:21.005267 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 316612us
W20260709 12:25:20.999718 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57918.6us
I20260709 12:25:21.011816 26123 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:20.841162 30281 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.257s	user 0.001s	sys 0.001s
I20260709 12:25:20.808651 29693 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.015136 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.016263 26129 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:21.011816 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:21.021746 29511 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:21.022310 29511 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.950119 29976 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:21.023357 29511 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.022957 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:21.001451 30183 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.007s	sys 0.010s
I20260709 12:25:20.999491 30066 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:20.998494 30254 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.950165 30040 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.027897 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.004973 30289 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:20.842737 30251 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:21.034703 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91853.5us
I20260709 12:25:21.081040 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:20.803134 30037 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.728066 30035 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.037827 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:20.930361 29944 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.722877 30160 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:21.042322 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 340204us
I20260709 12:25:21.007993 30147 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:21.050637 26129 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:21.029996 26127 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:20.725642 30270 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:21.029577 30267 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.002s
I20260709 12:25:20.805375 30272 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.949848 30268 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.993188 30252 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:21.077937 26196 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:21.077956 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:21.040856 30183 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.007s	sys 0.011s
I20260709 12:25:20.778565 30075 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:20.722570 30087 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:21.386448 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 305457us
I20260709 12:25:21.081296 30260 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.122874 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 175873us
W20260709 12:25:21.227475 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.429770 29976 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:21.080498 30250 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:21.265340 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 255863us
W20260709 12:25:21.417912 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.233300 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:21.423942 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 208427us
I20260709 12:25:21.480305 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:21.027289 29693 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.113430 30251 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.043834 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:20.937945 30264 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.081275 30040 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:21.340425 30271 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:25:21.042364 30176 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.043231 30219 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:20.792990 30249 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.433634 29511 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:21.449180 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.450111 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2039) took 1296 ms. Trace:
W20260709 12:25:21.453090 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.458529 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.521085 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.521488 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.521835 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.459784 26129 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:21.461614 26120 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:25:21.433492 30302 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.320s	user 0.002s	sys 0.005s
I20260709 12:25:21.463636 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:21.466501 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.528903 26198 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:21.039834 30066 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:21.428580 30036 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:21.474296 26783 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:21.484406 26121 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.492797 30255 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.487541 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 443759us
I20260709 12:25:21.534502 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:21.534971 29511 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.535130 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:21.535738 29511 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.536298 29511 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.453691 30294 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.461272 29837 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.537434 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:21.538249 26783 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:21.537807 29837 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.538888 26784 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:21.474892 29961 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:21.428917 30037 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.540560 29961 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.542587 30037 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:21.543131 29961 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.500716 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:21.543869 29961 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:21.473979 29976 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:21.482782 30252 consensus_queue.cc:1243] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:21.524682 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.518393 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:21.519841 26125 rpcz_store.cc:276] 0709 12:25:20.154210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.345989 (+191779us) service_pool.cc:224] Handling call
0709 12:25:20.346299 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.350090 (+  3791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:20.350121 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.350600 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.437112 (+ 86512us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:20.437153 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.437220 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.449860 (+1012640us) raft_consensus.cc:1685] finished
0709 12:25:21.449886 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.450045 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3315,"raft.queue_time_us":13129,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":151,"thread_start_us":86097,"threads_started":1,"wal-append.queue_time_us":139343}
I20260709 12:25:21.497803 30269 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:21.507975 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.560071 26196 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:21.560115 29511 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.537746 30294 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.546540 29837 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.549763 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:21.432348 30254 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.586885 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.551659 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:21.555876 26789 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:21.503683 30267 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:25:21.588707 29511 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.589154 29511 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.507519 30253 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:21.556730 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:21.512666 30295 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.466631 30268 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.563444 26128 pending_rounds.cc:85] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 5
W20260709 12:25:21.570612 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 88001.6us
I20260709 12:25:21.567551 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:21.568356 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:21.595899 26128 pending_rounds.cc:107] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:25:21.596058 26790 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:21.596457 26128 raft_consensus.cc:2889] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.479445 30260 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.584334 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.558102 29976 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.600047 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.616024 26120 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:21.562119 30036 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:21.563174 30269 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.576967 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:21.564380 30298 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:25:21.571579 30252 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.497153 30304 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:25:21.608139 30037 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:25:21.554198 30106 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:21.631490 30106 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.634555 30106 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.556406 30256 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.556051 29961 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.636556 29961 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:21.636139 30256 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.637526 26129 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:21.619129 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:21.560973 30129 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.560297 30317 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.640023 30129 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.640245 30317 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.640676 30129 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.622045 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:21.626443 26121 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:21.595248 30262 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.642220 30262 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.643383 30129 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.644716 30262 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.645357 30262 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:21.645735 30262 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:21.601361 30295 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.646145 30262 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:21.646445 30295 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.550288 30255 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.648888 30255 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.649516 30255 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:21.650004 30255 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:21.651222 30255 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.657200 29961 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.658959 26791 raft_consensus.cc:2368] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:21.660188 29961 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.665580 30262 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.668460 30262 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.644021 30129 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.670455 30129 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:21.679601 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.602972 30194 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:21.532656 30147 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.683686 26127 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 12:25:21.680191 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.624691 30269 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:21.622730 30253 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.683029 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 296595us
W20260709 12:25:21.696952 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75020.6us
I20260709 12:25:21.624411 30183 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:21.710835 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 210209us
I20260709 12:25:21.733279 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.734133 26195 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
W20260709 12:25:21.736248 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 272729us
I20260709 12:25:21.733266 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:21.737012 26201 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.742605 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:21.718753 30266 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.186s	user 0.009s	sys 0.007s
I20260709 12:25:21.550237 30289 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.761970 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:21.743716 30274 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.171s	user 0.009s	sys 0.016s
I20260709 12:25:21.716255 30129 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:21.767714 26046 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:21.743964 29944 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.557s	user 0.019s	sys 0.005s
I20260709 12:25:21.742455 30106 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.769495 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:21.769631 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:21.784632 26050 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:21.770187 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:21.773324 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:21.769207 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:21.774914 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 240477us
I20260709 12:25:21.752245 30147 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:21.781683 26049 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:21.629558 30252 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.630115 30066 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.762543 30194 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.802678 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75320.9us
I20260709 12:25:21.827420 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.626320 30250 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.774948 30309 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:21.805652 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.811112 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:21.825263 30308 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:21.825673 30288 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.826030 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:21.826400 30303 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:21.826783 30324 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:21.832674 30262 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.836563 30290 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
I20260709 12:25:21.836705 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:21.837144 30318 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
W20260709 12:25:21.837282 30293 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.349s	user 0.001s	sys 0.001s
I20260709 12:25:21.836440 30194 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:21.837632 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:21.838603 30320 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 12:25:22.012843 26053 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:22.013444 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 472) took 1478 ms. Trace:
I20260709 12:25:22.013928 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:21.841557 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55039.5us
I20260709 12:25:22.014842 26053 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:21.803838 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.015985 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:21.781057 30075 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:21.889075 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 120073us
I20260709 12:25:22.021297 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:21.916265 26194 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:21.797032 30267 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.913400 26046 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:22.027424 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.034543 30264 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.036914 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:22.037317 30339 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:25:22.039198 30035 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.927s	user 0.009s	sys 0.003s
I20260709 12:25:22.040676 30336 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.041318 30336 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.047547 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 625) took 1332 ms. Trace:
I20260709 12:25:21.925563 26048 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:21.753396 30253 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:21.927312 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.057622 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.058095 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.804893 30129 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEARNER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:21.927664 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 225866us
I20260709 12:25:21.911280 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.781876 30272 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:21.765317 30266 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:21.851572 29907 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:21.932801 26122 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:21.934432 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:21.781642 30274 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.934984 26050 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:21.935129 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.935555 30268 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.072950 26197 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:21.788113 29944 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.927482 30251 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:21.909315 29693 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.104501 30268 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:22.107147 30325 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:22.047051 30264 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.842149 30270 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.785640 26201 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:21.986438 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:21.957314 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.797479 30106 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:22.007149 26200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.005s	sys 0.008s
W20260709 12:25:22.113344 26200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.005s	sys 0.008s
W20260709 12:25:21.928442 26094 connection.cc:256] RPC call timeout handler was delayed by -2.36050367200005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:25:22.115373 30325 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:22.116758 30325 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:22.117159 30325 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:22.117617 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:21.930882 30219 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:22.118865 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.119761 30219 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.121374 30274 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:21.933342 30308 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.013971 26052 rpcz_store.cc:276] 0709 12:25:20.535489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.535648 (+   159us) service_pool.cc:224] Handling call
0709 12:25:20.535940 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.547628 (+ 11688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:20.547662 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.559679 (+ 12017us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.968654 (+408975us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:20.968709 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.968811 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.013178 (+1044367us) raft_consensus.cc:1685] finished
0709 12:25:22.013207 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.013364 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4203,"raft.queue_time_us":4422,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":16103,"thread_start_us":340041,"threads_started":1,"wal-append.queue_time_us":464819}
I20260709 12:25:21.793427 30036 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.053323 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:25:22.055835 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 358686us
I20260709 12:25:22.128963 30308 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:22.014211 30194 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.059664 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:22.023811 30255 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:22.132634 30255 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.132676 29931 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:22.133178 30255 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:22.050585 26203 rpcz_store.cc:276] 0709 12:25:20.715496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.725948 (+ 10452us) service_pool.cc:224] Handling call
0709 12:25:20.726434 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.023617 (+297183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.023655 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.125277 (+101622us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.536433 (+411156us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:21.536588 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.536850 (+   262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.047283 (+510433us) raft_consensus.cc:1685] finished
0709 12:25:22.047308 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.047481 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67632,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":10454,"thread_start_us":311724,"threads_started":2,"wal-append.queue_time_us":489856}
I20260709 12:25:22.134749 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:22.135108 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81857.5us
I20260709 12:25:22.135541 26203 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:22.039695 30035 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.068104 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.077437 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.140058 26785 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:22.140151 26194 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:25:22.079041 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.044102 30336 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.083346 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.141853 26195 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:22.084534 29944 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.047940 30254 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.085141 30307 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.085785 30323 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:25:22.161270 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.087430 30335 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:22.090324 30266 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.052577 30262 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.091939 26045 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:22.092610 30317 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.093653 30252 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.094594 30253 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:22.095458 30276 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.071s	user 0.005s	sys 0.019s
I20260709 12:25:22.095582 30309 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:22.097410 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:21.994553 30303 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:21.956818 30288 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.102594 30295 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.063323 29961 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:22.108552 26786 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:22.108899 26046 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:21.809361 30183 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 12:25:22.119952 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.122735 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.126971 26054 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:22.363140 26201 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:22.123924 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:22.130710 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.364114 26052 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:25:22.132167 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72640.9us
I20260709 12:25:22.366760 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.060775 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.050164 30267 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.124334 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.136982 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.117902 30325 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.138945 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1931) took 1252 ms. Trace:
I20260709 12:25:22.167248 30307 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.086651 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.141811 30274 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:22.191906 26050 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.129457 30264 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.194926 30331 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 12:25:22.395395 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.395709 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.199118 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.396493 26782 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
W20260709 12:25:22.200151 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57775.1us
W20260709 12:25:22.303339 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.171371 29931 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.295713 30266 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.331845 30279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.498s	user 0.018s	sys 0.000s
I20260709 12:25:22.333026 30303 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.198434 30276 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.338426 30288 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.148075 30033 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:25:22.363233 26197 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:22.372354 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.384208 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:22.389623 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 266071us
I20260709 12:25:22.391173 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.362273 29961 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.391706 26787 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:22.391232 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.386994 26200 rpcz_store.cc:276] 0709 12:25:20.886806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.886998 (+   192us) service_pool.cc:224] Handling call
0709 12:25:20.998667 (+111669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.008781 (+ 10114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.008834 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.027742 (+ 18908us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.113828 (+1086086us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:22.138364 (+ 24536us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.138622 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.138704 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.138753 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:22.138769 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.138893 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24097,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":5029,"raft.queue_time_us":125653,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":34156,"thread_start_us":659531,"threads_started":1,"wal-append.queue_time_us":932328}
I20260709 12:25:22.197213 30262 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.396673 26194 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:22.391966 29976 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.705s	user 0.011s	sys 0.000s
I20260709 12:25:22.188421 30268 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.208s
I20260709 12:25:22.388500 26202 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:22.340534 30129 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.361837 30289 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.417755 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.410580 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.397791 29907 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:22.426280 26783 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:22.427012 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.394670 30274 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:22.403836 30279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.758s	user 0.019s	sys 0.000s
I20260709 12:25:22.398757 29931 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.432685 26203 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:22.406175 30331 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:22.441166 26126 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:22.173553 30335 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.415323 30106 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.443207 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:22.443728 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.425066 29961 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.443208 26050 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.446293 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.450177 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.453078 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.453706 26050 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.451885 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.452756 26783 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:22.435561 29907 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:22.459822 29907 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:22.461359 29931 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.464020 30330 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:22.464584 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.464704 30330 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.465603 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.466791 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.466988 30330 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.467351 26200 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:22.468585 26782 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.469491 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.469728 30335 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.470516 30329 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.443624 30274 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.471961 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.473100 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.457370 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.434948 30325 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.405865 30266 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.458585 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.475387 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.472010 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:22.477501 26203 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.478986 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:22.489003 26790 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:22.485994 26050 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:22.495177 30106 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:22.495450 30347 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:25:22.495709 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.496589 26201 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:22.446233 30331 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:22.497331 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:22.495462 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.498742 26054 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:22.501837 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.502097 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.502602 26784 raft_consensus.cc:2368] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:22.502766 26783 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:22.471069 30329 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:22.456058 30252 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:22.434260 29976 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.503679 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.519443 26050 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.520545 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.520782 29976 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.521155 26050 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:22.505641 29931 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:25:22.522101 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.523873 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:22.524708 26120 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:22.524863 30331 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.525743 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:22.525981 30331 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.526553 26051 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:22.471096 30267 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.529014 30295 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.529129 30035 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.505820 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.529937 26054 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.530619 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 897) took 2258 ms. Trace:
I20260709 12:25:22.530771 30276 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.530881 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.531558 26054 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:22.503861 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.531977 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:22.440840 30129 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.532655 26125 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:25:22.533582 30348 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:25:22.534504 30129 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.534554 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.534907 30307 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.508888 30335 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:22.535483 30335 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:22.535861 30335 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.502811 30340 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:22.536578 30340 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:22.536947 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:22.537460 30315 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.537570 29907 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.538199 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:22.538857 26124 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:22.502825 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.504451 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.510542 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:22.539681 26201 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:22.448689 30289 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.516315 30266 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.532805 26784 raft_consensus.cc:2410] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:22.538975 26789 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:22.495741 30106 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.517913 30270 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:22.504868 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.528610 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.533595 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:22.523800 30345 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:22.535801 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.555943 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.542737 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.557165 26790 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:22.544620 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.550853 26125 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:22.557574 26054 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:22.552290 26051 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:25:22.556051 26198 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.541960 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.557987 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.541855 26788 rpcz_store.cc:276] 0709 12:25:20.272154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.531777 (+259623us) service_pool.cc:224] Handling call
0709 12:25:20.547579 (+ 15802us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.553004 (+  5425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:20.553038 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.553466 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.969097 (+415631us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:20.969149 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.530207 (+1561058us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b540057a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:22.530267 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.530310 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:22.530323 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.530540 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35353,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2492,"spinlock_wait_cycles":11684096,"thread_start_us":388686,"threads_started":1,"wal-append.queue_time_us":1335487}
I20260709 12:25:22.557945 26054 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:22.561069 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.562345 26120 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:22.558725 26783 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:22.563520 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 618) took 1588 ms. Trace:
I20260709 12:25:22.570123 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.615533 26201 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.616724 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 617) took 1641 ms. Trace:
W20260709 12:25:22.570689 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.537015 30340 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.540278 30330 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:22.532660 30264 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.561307 26054 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.625290 26054 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.541335 30163 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:25:22.626375 26120 rpcz_store.cc:276] 0709 12:25:20.975054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.544288 (+1569234us) service_pool.cc:224] Handling call
0709 12:25:22.562159 (+ 17871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.614897 (+ 52738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.616627 (+  1730us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:22.629328 26051 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:22.615314 26125 rpcz_store.cc:276] 0709 12:25:20.975370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.562411 (+1587041us) service_pool.cc:224] Handling call
0709 12:25:22.562818 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.563080 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.563120 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.563179 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:22.563239 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.563315 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.563446 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:22.630705 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.631274 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2046) took 1655 ms. Trace:
I20260709 12:25:22.569451 26047 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:22.631870 26125 rpcz_store.cc:276] 0709 12:25:20.976045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.630301 (+1654256us) service_pool.cc:224] Handling call
0709 12:25:22.630669 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.630882 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.630921 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.630977 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:22.631020 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.631084 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.631204 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:22.559252 30037 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:22.643807 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.645454 26201 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:22.634356 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2047) took 1658 ms. Trace:
I20260709 12:25:22.557377 30106 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:22.564821 30269 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.812s	user 0.010s	sys 0.002s
I20260709 12:25:22.617885 29693 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.541584 30317 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:25:22.563472 30303 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:22.668576 26124 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:22.670629 26124 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:25:22.670689 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 227588us
W20260709 12:25:22.649767 30036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.031s	sys 0.049s
I20260709 12:25:22.618435 30004 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.618187 30251 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.625732 30307 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.538013 30315 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.656546 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.644153 30330 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:22.547093 29961 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.665584 26054 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:25:22.622217 30194 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:22.638370 26120 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:22.689700 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2045) took 1714 ms. Trace:
I20260709 12:25:22.690052 26120 rpcz_store.cc:276] 0709 12:25:20.975706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.629952 (+1654246us) service_pool.cc:224] Handling call
0709 12:25:22.638131 (+  8179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.689441 (+ 51310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.689642 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:22.648447 26125 rpcz_store.cc:276] 0709 12:25:20.989766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.633370 (+1643604us) service_pool.cc:224] Handling call
0709 12:25:22.633736 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.633942 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.633994 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.634048 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:22.634087 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.634158 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.634290 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:22.690675 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.671389 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 619) took 1654 ms. Trace:
I20260709 12:25:22.691414 26054 pending_rounds.cc:85] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 8
I20260709 12:25:22.691839 26054 pending_rounds.cc:107] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:25:22.692283 26054 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.645870 30037 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:22.674981 30036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.031s	sys 0.049s
I20260709 12:25:22.679247 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.680811 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.681514 30303 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.678129 30004 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:22.700196 30184 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:25:22.701751 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.702013 30004 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.702674 30004 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:22.703087 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.703749 26052 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:22.687742 30307 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.704636 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.705370 26052 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:22.706220 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.706543 30307 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.707340 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.708108 26052 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:22.708868 30303 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.709005 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.711596 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2049) took 1241 ms. Trace:
I20260709 12:25:22.712007 26125 rpcz_store.cc:276] 0709 12:25:21.469811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.710758 (+1240947us) service_pool.cc:224] Handling call
0709 12:25:22.711078 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.711270 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.711297 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.711338 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:22.711369 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.711424 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.711541 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:22.712800 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.713546 26125 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:22.687345 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 126397us
I20260709 12:25:22.714197 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.691183 26124 rpcz_store.cc:276] 0709 12:25:21.016898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.669626 (+1652728us) service_pool.cc:224] Handling call
0709 12:25:22.670291 (+   665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.671130 (+   839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.671338 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:22.719141 26198 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:22.720045 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.720480 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2050) took 1250 ms. Trace:
I20260709 12:25:22.720788 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.721941 26198 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:22.722054 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.722775 26203 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:22.718261 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.722724 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1004) took 1171 ms. Trace:
I20260709 12:25:22.717669 26120 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:25:22.725296 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.724584 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:22.726090 26052 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.682861 30317 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.724433 26791 rpcz_store.cc:276] 0709 12:25:21.551099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.551359 (+   260us) service_pool.cc:224] Handling call
0709 12:25:22.722530 (+1171171us) spinlock_profiling.cc:243] Waited 71 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:22.722663 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163225984}
I20260709 12:25:22.728794 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2048) took 1259 ms. Trace:
I20260709 12:25:22.732827 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.720954 26125 rpcz_store.cc:276] 0709 12:25:21.470097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.713037 (+1242940us) service_pool.cc:224] Handling call
0709 12:25:22.713353 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.720181 (+  6828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.720422 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:22.746018 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.711112 30037 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:22.740154 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:22.729558 26120 rpcz_store.cc:276] 0709 12:25:21.469448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.716887 (+1247439us) service_pool.cc:224] Handling call
0709 12:25:22.717330 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.728395 (+ 11065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.728712 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:22.790985 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.786111 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:22.792588 26198 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:22.792888 26203 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.670274 30269 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.685449 30354 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.793968 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53931us
I20260709 12:25:22.669088 29693 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:22.734905 26787 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 12:25:22.799471 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 73333.4us
I20260709 12:25:22.734171 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.730998 30303 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.800637 26201 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:22.688058 30251 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.804956 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.735908 29907 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.810712 26047 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:22.810249 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.807047 30184 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.817245 30026 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.822167 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:22.817938 30184 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:22.825480 26052 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:22.825863 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.831530 30026 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:22.835328 30315 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.840653 30004 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:22.841209 30004 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.841589 30004 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.844518 26051 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:22.846662 29961 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.847810 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:22.848173 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.848583 26790 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:22.846768 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:22.849579 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:22.850201 26791 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:22.844787 26201 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:22.794044 30317 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:22.852604 26018 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.810s (SENT)
I20260709 12:25:22.853331 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.854867 30317 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:22.855754 26018 leader_election.cc:400] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:22.856415 26018 leader_election.cc:403] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:22.857318 30356 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:22.857805 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.860657 26201 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:22.863796 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.859802 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:22.867755 26194 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:22.807390 30329 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.880034 30184 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.881486 30004 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.882246 26203 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:22.884603 26047 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:22.886217 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.810184 30303 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:22.890772 26019 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.037s (SENT)
I20260709 12:25:22.893031 30329 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:22.893836 30269 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.894399 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:22.967492 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73212.9us
I20260709 12:25:22.882624 30330 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.898909 29693 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:22.981664 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 132167us
W20260709 12:25:22.982095 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91360.8us
I20260709 12:25:22.982673 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:22.983525 26019 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.098s (SENT)
I20260709 12:25:22.900714 30359 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:22.904759 30254 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:22.907861 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 623) took 1262 ms. Trace:
I20260709 12:25:22.984841 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.985131 26126 rpcz_store.cc:276] 0709 12:25:21.645559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.928804 (+283245us) service_pool.cc:224] Handling call
0709 12:25:22.907631 (+978827us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54001f77d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:22.907784 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848591104}
I20260709 12:25:22.908792 30289 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:22.910682 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:22.919224 30066 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:22.921872 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.925834 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:22.931115 30349 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.426s	user 0.003s	sys 0.000s
I20260709 12:25:22.933800 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:22.934473 30255 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.001s
I20260709 12:25:22.936585 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:22.803395 30354 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.940384 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.016444 26198 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:22.942014 26120 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:23.018249 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:22.943044 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:22.944779 26124 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:23.019090 26125 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:22.946043 30251 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:22.953076 30296 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:22.960541 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 165483us
I20260709 12:25:23.030652 30296 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:22.987366 30315 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.031795 30296 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:23.033021 30296 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.021020 30184 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.034394 30184 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:22.968878 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.036587 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2774) took 2095 ms. Trace:
W20260709 12:25:22.944871 30308 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.813s	user 0.008s	sys 0.003s
I20260709 12:25:23.037155 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:22.989416 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:22.981177 30252 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:23.013240 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79591.6us
W20260709 12:25:23.013475 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76987us
I20260709 12:25:23.045080 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.038669 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:23.047430 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.047683 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.013972 30289 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.014782 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.015398 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.048842 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:23.019886 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.049453 26203 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.023257 26120 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:23.032614 30315 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:23.037034 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68237.1us
I20260709 12:25:23.049487 26048 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:23.012686 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.039503 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:23.036994 26128 rpcz_store.cc:276] 0709 12:25:20.941039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.993947 (+ 52908us) service_pool.cc:224] Handling call
0709 12:25:21.562935 (+568988us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:25:21.563173 (+   238us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:21.928188 (+365015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.928223 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:21.950829 (+ 22606us) log.cc:844] Serialized 78 byte log entry
0709 12:25:22.163418 (+212589us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:22.163453 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.163768 (+   315us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.930360 (+766592us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:23.036323 (+105963us) raft_consensus.cc:1685] finished
0709 12:25:23.036347 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.036525 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18964,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":60,"spinlock_wait_cycles":74753920,"thread_start_us":448458,"threads_started":2,"wal-append.queue_time_us":426779}
I20260709 12:25:23.041954 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.012060 26194 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.051406 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.049340 26126 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:23.052337 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:23.052500 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.054744 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:23.055611 26790 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:23.010246 29693 raft_consensus.cc:2764] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:23.057216 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:23.022122 30268 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.060330 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.060362 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:23.062012 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.068564 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.058367 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.066022 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.070958 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:23.073591 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:23.069229 26127 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:23.075109 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.075749 26120 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:23.078908 26203 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:23.079766 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.077761 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.081911 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.082152 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.076802 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.037706 30308 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:23.083159 26125 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:23.083683 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.046034 30329 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.086120 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.088449 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.052780 30289 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.050670 30276 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.060662 30264 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.091905 26125 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:23.091739 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.096148 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.062554 30288 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.094386 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.098743 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:23.070192 30183 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.103788 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.104074 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:23.022480 30255 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.142494 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:23.151310 26203 raft_consensus.cc:2368] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:23.144716 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.153210 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:23.153676 26194 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:23.153818 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57840.1us
I20260709 12:25:23.046653 30251 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:25:23.155992 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:23.156917 26199 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:23.140723 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.159927 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.093788 29907 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:23.098585 30252 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.107991 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:23.087212 30335 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 12:25:23.110107 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:23.169878 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:23.161747 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:23.181018 26124 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.093082 30275 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:23.089986 30366 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.185191 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.160022 30268 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:23.137840 30354 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:23.137913 30004 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.187037 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.187049 26203 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:23.098889 30269 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:25:23.181556 26054 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:23.102861 30289 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.160362 30331 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.221045 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:23.191557 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 140599us
I20260709 12:25:23.167480 30264 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.138617 30364 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.199259 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.168119 30254 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:23.136713 30184 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.255620 26124 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:23.257006 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:23.170392 30355 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 12:25:23.265022 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96983us
W20260709 12:25:23.243322 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60113.6us
W20260709 12:25:23.265702 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 213253us
I20260709 12:25:23.247255 30004 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:23.221271 26126 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:23.268391 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.221001 30331 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.185258 29907 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:23.190025 30268 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.164492 30356 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.269824 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.270617 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.225713 30251 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.228174 30264 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.290019 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:23.189420 30275 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.271938 30288 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:23.283258 30274 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.774s	user 0.014s	sys 0.000s
I20260709 12:25:23.279330 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:23.280823 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 189157us
I20260709 12:25:23.182668 30194 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:25:23.257732 30035 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.282335 29961 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.320276 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:23.288617 30268 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:23.305747 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.187148 30335 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:23.296624 26167 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.644s (SENT)
I20260709 12:25:23.324870 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:23.282163 30366 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.282338 30331 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:23.303280 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.257983 30183 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:23.288862 30356 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.289244 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.326066 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.323563 26048 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:23.332376 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 622) took 1778 ms. Trace:
I20260709 12:25:23.332975 26129 rpcz_store.cc:276] 0709 12:25:21.553526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.586597 (+ 33071us) service_pool.cc:224] Handling call
0709 12:25:23.332182 (+1745585us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:23.332314 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114699136}
W20260709 12:25:23.333303 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44141.1us
I20260709 12:25:23.302138 30262 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.299238 29693 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.297461 30036 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:23.333693 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 119816us
I20260709 12:25:23.334424 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:23.290586 30037 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.311998 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.340734 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:23.342238 26785 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:23.343403 26018 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:23.343806 26018 leader_election.cc:403] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:23.343746 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:23.344936 26129 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.347621 26203 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:25:23.348315 26203 pending_rounds.cc:85] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 4
I20260709 12:25:23.348639 26203 pending_rounds.cc:107] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:25:23.348949 26203 raft_consensus.cc:2889] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:23.351891 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.352850 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.353654 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:23.354427 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:23.327785 30366 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.359534 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.360157 26123 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:23.326982 30308 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:23.341553 30183 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:23.327173 30373 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:23.372112 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:23.304644 30303 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:23.327613 30374 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.305205 30274 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.342747 30262 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.375047 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:23.366107 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.326364 30035 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.334363 30356 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.386006 26201 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:23.352769 30252 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.347860 30269 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.388859 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.404042 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.341857 30276 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.370599 30255 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.341820 26126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:23.416893 26126 raft_consensus.cc:606] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.417322 26126 raft_consensus.cc:614] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:23.372661 29907 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.368842 30160 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.372910 30366 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.392041 30275 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:23.424988 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82603.6us
I20260709 12:25:23.393060 30183 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:23.352002 30315 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:23.399020 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 102461us
I20260709 12:25:23.351747 30288 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:23.402251 26129 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:23.431279 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.432001 26129 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:23.366758 30278 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.410902 26052 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:23.398550 30106 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.710s	user 0.008s	sys 0.004s
I20260709 12:25:23.379582 30376 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.419838 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.394147 30262 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:23.402208 30340 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.002s
W20260709 12:25:23.422926 26019 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.487s (SENT)
I20260709 12:25:23.361399 30004 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.401643 30252 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.399893 30036 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:23.364192 30385 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.384860 30309 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:25:23.429582 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 95191.7us
I20260709 12:25:23.457399 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:23.434439 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:23.458375 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.458797 26129 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:23.417452 30269 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.468349 26126 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:23.447986 30367 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 12:25:23.442711 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92441.4us
I20260709 12:25:23.448421 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:23.449286 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.479475 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.442569 30026 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.008s
I20260709 12:25:23.460088 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.441321 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.435547 30278 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.464843 29511 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:23.440079 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.503979 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:23.504875 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44877.9us
W20260709 12:25:23.505163 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 63927.3us
I20260709 12:25:23.505927 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.506464 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.503257 30278 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.513288 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:23.514117 26789 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.473248 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:25:23.502810 30377 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:25:23.510183 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:23.491006 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:23.513345 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 73326.1us
I20260709 12:25:23.457226 30252 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.519832 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.469637 26129 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:23.457491 30385 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.457397 30036 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:23.561570 26129 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:23.561525 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:23.525249 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 102343us
I20260709 12:25:23.563278 26129 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:23.564219 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56060.8us
I20260709 12:25:23.565270 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.566116 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.444917 30106 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:23.580401 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70277.1us
I20260709 12:25:23.582707 29511 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:23.583307 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 92443.5us
I20260709 12:25:23.504436 30075 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.585146 26094 leader_election.cc:400] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:23.590796 26094 leader_election.cc:403] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:23.456328 30340 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.532879 26783 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9.
W20260709 12:25:23.586992 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.587018 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:23.588726 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 115618us
I20260709 12:25:23.471427 30384 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.512720 30382 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:23.562279 30365 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.593961 30276 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:23.524830 30315 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:23.564855 30373 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:23.603610 26127 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:23.565222 30385 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.587837 30370 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:23.565798 30036 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.515419 30004 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:23.592111 30026 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.608633 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1546) took 1407 ms. Trace:
I20260709 12:25:23.609149 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.536435 29877 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:23.599292 30303 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:23.582620 30037 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.621335 26050 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
W20260709 12:25:23.620926 30066 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.001s
I20260709 12:25:23.624267 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:23.624421 26789 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:23.596107 30386 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.630601 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.601029 30384 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.627343 26197 rpcz_store.cc:276] 0709 12:25:22.201623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.016018 (+814395us) service_pool.cc:224] Handling call
0709 12:25:23.056342 (+ 40324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.077041 (+ 20699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.077070 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.093131 (+ 16061us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.297570 (+204439us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:23.297666 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.299873 (+  2207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.608409 (+308536us) raft_consensus.cc:1685] finished
0709 12:25:23.608429 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.608574 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10161,"raft.queue_time_us":25845,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":40153,"thread_start_us":121409,"threads_started":1,"wal-append.queue_time_us":174744}
I20260709 12:25:23.562512 30255 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.657833 26200 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:23.610037 30308 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.620118 30274 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.620707 30183 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.622992 30373 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:23.599326 30075 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.636133 30309 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:23.679109 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.665856 30163 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.600406 30288 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.658474 30384 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.660385 30386 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.681380 30035 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.687634 30309 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.726881 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.724632 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1957) took 1034 ms. Trace:
I20260709 12:25:23.733217 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.734848 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.725606 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2043) took 3386 ms. Trace:
I20260709 12:25:23.728075 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2778) took 1105 ms. Trace:
I20260709 12:25:23.726825 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 482) took 1990 ms. Trace:
W20260709 12:25:23.727391 26046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1542) took 1705 ms (client timeout 1892 ms). Trace:
I20260709 12:25:23.733884 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 707) took 3581 ms. Trace:
I20260709 12:25:23.789280 26194 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
W20260709 12:25:23.788987 26046 rpcz_store.cc:269] 0709 12:25:22.021723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.036564 (+ 14841us) service_pool.cc:224] Handling call
0709 12:25:23.727178 (+1690614us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:23.727328 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163047680}
I20260709 12:25:23.726249 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1937) took 2128 ms. Trace:
I20260709 12:25:23.689502 30255 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:23.790771 26203 rpcz_store.cc:276] 0709 12:25:22.689841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.358317 (+668476us) service_pool.cc:224] Handling call
0709 12:25:23.382773 (+ 24456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.399412 (+ 16639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.399446 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.404144 (+  4698us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.562103 (+157959us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:23.562149 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.562220 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.724404 (+162184us) raft_consensus.cc:1685] finished
0709 12:25:23.724425 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.724570 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7354,"raft.queue_time_us":16695,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":8547,"thread_start_us":124824,"threads_started":1,"wal-append.queue_time_us":255487}
I20260709 12:25:23.727795 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.789006 26122 rpcz_store.cc:276] 0709 12:25:20.339440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.437858 (+ 98418us) service_pool.cc:224] Handling call
0709 12:25:20.438189 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.071661 (+1633472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.725260 (+1653599us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54001f90d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:23.725491 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3436668800}
I20260709 12:25:23.786286 26045 rpcz_store.cc:276] 0709 12:25:21.736014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.752557 (+ 16543us) service_pool.cc:224] Handling call
0709 12:25:23.726659 (+1974102us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400172354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:23.726769 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707197312}
I20260709 12:25:23.790645 26196 rpcz_store.cc:276] 0709 12:25:21.602102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.602296 (+   194us) service_pool.cc:224] Handling call
0709 12:25:21.602651 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.340720 (+738069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.340753 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.340808 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:22.340858 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.725958 (+1385100us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400399c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:23.726017 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.726157 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696862976}
I20260709 12:25:23.737993 30355 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:23.724069 30296 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.000s
I20260709 12:25:23.803122 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.831334 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:23.832159 26196 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:23.832775 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.690491 30276 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:23.808048 26195 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:23.793897 26051 rpcz_store.cc:276] 0709 12:25:22.623148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.643306 (+ 20158us) service_pool.cc:224] Handling call
0709 12:25:23.727908 (+1084602us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400228654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:23.728011 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371747712}
I20260709 12:25:23.690629 30315 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:23.739560 30340 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.834342 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.834887 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.845432 26203 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:25:23.835635 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 205055us
I20260709 12:25:23.847070 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.796095 26786 rpcz_store.cc:276] 0709 12:25:20.152225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.210243 (+ 58018us) service_pool.cc:224] Handling call
0709 12:25:20.227997 (+ 17754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.362260 (+2134263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.733606 (+1371346us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400650254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:23.733795 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4324382464}
W20260709 12:25:23.836874 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 110078us
I20260709 12:25:23.842722 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:23.802577 30275 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:23.854076 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94752.2us
W20260709 12:25:23.857276 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.686517 30389 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10.
I20260709 12:25:23.848711 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:23.849596 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:23.850090 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.865579 26791 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:23.733922 30264 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:23.769676 30386 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.807806 30309 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.858721 26045 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:23.867434 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.858632 30255 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.868050 26045 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
W20260709 12:25:23.840916 30391 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:23.863654 26196 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:23.694720 30163 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.823325 30355 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:23.881433 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.881529 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:23.844600 30392 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.006s
I20260709 12:25:23.883316 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:23.889977 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.891618 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.892614 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:23.893522 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:23.868048 30255 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.903555 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.909739 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:23.910231 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.911274 26203 raft_consensus.cc:2376] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12. Re-sending same reply.
I20260709 12:25:23.912451 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.913197 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:23.879848 30269 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:23.915493 26051 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:23.915987 29511 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.916652 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.877921 30381 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.921249 26128 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:23.722345 30373 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.896265 30275 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:23.919287 26197 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:23.901783 30340 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.903371 30255 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.929098 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:23.932636 26051 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:23.938117 30268 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.003s
I20260709 12:25:23.938764 30268 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:23.941185 30268 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:23.941491 30315 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.944387 30276 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:23.948792 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 93681.1us
I20260709 12:25:23.949681 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:23.947943 30252 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.921965 30381 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.953004 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:23.955907 26197 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:23.958572 30252 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:23.961167 30390 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 12:25:23.964370 30350 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 1.458s	user 0.011s	sys 0.008s
I20260709 12:25:23.966609 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2791) took 1020 ms. Trace:
I20260709 12:25:23.971067 30256 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.971853 30267 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.972506 30267 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:23.974831 30267 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:23.976423 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:23.977126 26195 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.967154 26126 rpcz_store.cc:276] 0709 12:25:22.946543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.470612 (+524069us) service_pool.cc:224] Handling call
0709 12:25:23.470930 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:23.471282 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.471310 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:23.471871 (+   561us) log.cc:844] Serialized 52 byte log entry
0709 12:25:23.620492 (+148621us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:23.620533 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.620797 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.966264 (+345467us) raft_consensus.cc:1685] finished
0709 12:25:23.966294 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.966514 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":12480,"thread_start_us":40358,"threads_started":1,"wal-append.queue_time_us":173490}
I20260709 12:25:23.980069 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:23.987574 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.967969 30329 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.864s	user 0.010s	sys 0.001s
I20260709 12:25:23.997318 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.001528 26786 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:24.002416 26791 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:23.929718 30340 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:23.931988 30106 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:24.006628 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.008464 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:24.009655 26051 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:24.019861 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:24.020659 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2787) took 1075 ms. Trace:
I20260709 12:25:24.021293 26123 rpcz_store.cc:276] 0709 12:25:22.945459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.370668 (+425209us) service_pool.cc:224] Handling call
0709 12:25:23.371016 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.523336 (+152320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.523365 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.592736 (+ 69371us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.693766 (+101030us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:23.693801 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.693866 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:24.020318 (+326452us) raft_consensus.cc:1685] finished
0709 12:25:24.020350 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.020559 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54395,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1702,"thread_start_us":237323,"threads_started":2,"wal-append.queue_time_us":142917}
I20260709 12:25:24.020602 26197 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:23.928893 30336 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:25:23.994464 30373 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.045390 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:24.045796 29961 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:24.001173 30256 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.978570 30381 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:24.015914 30289 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.742s	user 0.016s	sys 0.008s
I20260709 12:25:24.047912 26051 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:24.010686 30329 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:24.054040 26195 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:24.010020 30340 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.063658 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.064445 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:24.065244 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.065840 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.065243 26195 raft_consensus.cc:2376] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11. Re-sending same reply.
I20260709 12:25:24.068022 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.068763 26195 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.071660 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:24.071626 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:24.073572 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 224907us
I20260709 12:25:24.081972 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:24.083511 26051 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:24.081736 26123 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:25:24.074657 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.053094 30256 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.053351 29961 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.089218 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:24.090030 30256 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.091187 30268 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:24.091604 30268 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.091962 30268 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.095180 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.087735 26047 raft_consensus.cc:2410] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:24.107184 26123 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:24.123935 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.126777 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.134765 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.075214 30289 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.134389 30329 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.139670 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.105070 30040 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.143347 30274 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:24.145386 30274 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.145777 30274 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:24.163297 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1963) took 1471 ms. Trace:
I20260709 12:25:24.137456 30252 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:24.163803 26203 rpcz_store.cc:276] 0709 12:25:22.692014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.162299 (+1470285us) service_pool.cc:224] Handling call
0709 12:25:24.162670 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.162903 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.162931 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.162975 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:24.163022 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.163085 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.163208 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:24.177760 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.178339 30329 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:24.184701 30355 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.210973 26788 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:24.208374 30401 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:25:24.197407 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.216091 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1966) took 1410 ms. Trace:
I20260709 12:25:24.210393 30329 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:24.220093 26203 rpcz_store.cc:276] 0709 12:25:22.805664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.214909 (+1409245us) service_pool.cc:224] Handling call
0709 12:25:24.215347 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.215610 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.215652 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.215710 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:24.215756 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.215830 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.216012 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.222440 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1967) took 1416 ms. Trace:
I20260709 12:25:24.223028 26203 rpcz_store.cc:276] 0709 12:25:22.806380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.221568 (+1415188us) service_pool.cc:224] Handling call
0709 12:25:24.221895 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.222116 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.222143 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.222184 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:24.222216 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.222271 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.222387 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.187952 30252 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.210855 30327 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.212s
I20260709 12:25:24.233101 26203 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:25:24.217274 30081 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:24.219960 30323 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.279903 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1556) took 1240 ms. Trace:
I20260709 12:25:24.280743 26051 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:24.282968 26203 rpcz_store.cc:276] 0709 12:25:23.039084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.232370 (+1193286us) service_pool.cc:224] Handling call
0709 12:25:24.232721 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.278860 (+ 46139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.279840 (+   980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.283720 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:24.254369 30252 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:24.288731 30376 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.834s	user 0.011s	sys 0.013s
I20260709 12:25:24.314630 26195 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:24.315343 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:24.322491 26051 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:24.315721 26203 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:24.323320 26197 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:25:24.323750 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1561) took 1283 ms. Trace:
I20260709 12:25:24.325546 26203 rpcz_store.cc:276] 0709 12:25:23.040540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.294353 (+1253813us) service_pool.cc:224] Handling call
0709 12:25:24.315392 (+ 21039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.323492 (+  8100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.323681 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.318087 30255 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.326239 26195 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:24.327157 26203 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:24.327767 26195 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:24.329800 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:24.330642 26051 raft_consensus.cc:2376] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10. Re-sending same reply.
I20260709 12:25:24.331771 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:24.328976 30255 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.294067 30396 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:24.333253 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:24.287098 30081 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.278815 30329 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:24.347160 30356 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.943s	user 0.010s	sys 0.000s
I20260709 12:25:24.348937 30184 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.349611 30184 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.352579 30225 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:24.353178 30225 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:24.354813 30278 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.789s	user 0.014s	sys 0.006s
W20260709 12:25:24.355181 30262 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.894s	user 0.020s	sys 0.000s
I20260709 12:25:24.355358 30278 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:24.355612 30262 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.287050 30323 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.325888 30376 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:24.354216 30225 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:24.368697 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 132642us
I20260709 12:25:24.347733 30356 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.329365 30040 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:24.385267 30163 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.387243 30268 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.360574 30184 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:24.375782 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.361528 30331 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.023s	user 0.017s	sys 0.008s
I20260709 12:25:24.402045 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:25:24.400122 30384 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.561s	user 0.021s	sys 0.000s
W20260709 12:25:24.405102 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.365660 30036 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.014s
I20260709 12:25:24.403981 30331 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:24.411849 30386 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
I20260709 12:25:24.412536 30331 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.412421 30386 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.413244 30036 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.414804 30386 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.413432 30381 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.416760 26790 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:24.406910 30309 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.012s
I20260709 12:25:24.356966 30396 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.422622 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:24.423583 26049 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:24.400760 30225 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.402716 30289 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.397753 30040 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.445367 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:24.449929 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 495) took 1079 ms. Trace:
I20260709 12:25:24.450608 26049 rpcz_store.cc:276] 0709 12:25:23.370575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.429712 (+1059137us) service_pool.cc:224] Handling call
0709 12:25:24.449833 (+ 20121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.432444 30036 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.415875 30384 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.457883 30396 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.460150 30262 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:24.461736 30256 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.463802 30176 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:24.465186 30381 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.466102 26121 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:24.467312 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.468693 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.469095 30256 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.472792 30356 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.473469 30040 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.474224 30278 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:24.474828 30340 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:24.476177 30385 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.839s	user 0.000s	sys 0.011s
I20260709 12:25:24.476610 30385 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.478602 30385 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:24.481621 26051 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:24.486716 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.487464 26126 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:24.489692 30323 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.492385 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.493247 26123 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:24.450590 30260 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:24.486238 30366 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.056s	user 0.021s	sys 0.007s
I20260709 12:25:24.426203 30309 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.507455 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.512388 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:24.525830 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.514168 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.527251 26121 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:24.526907 26126 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:24.530431 26053 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:25:24.534497 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.541581 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.530900 30325 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.530202 30385 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:24.522850 30366 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.550405 26121 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:25:24.554008 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.557998 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.559427 26121 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:24.562350 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 629) took 1611 ms. Trace:
I20260709 12:25:24.562876 26127 rpcz_store.cc:276] 0709 12:25:22.950488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.077512 (+127024us) service_pool.cc:224] Handling call
0709 12:25:24.562080 (+1484568us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54001c3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:24.562273 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197913856}
I20260709 12:25:24.563550 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2807) took 1197 ms. Trace:
I20260709 12:25:24.549909 30325 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.540153 29961 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.564225 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.576063 26126 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:24.563987 26129 rpcz_store.cc:276] 0709 12:25:23.366186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.838132 (+471946us) service_pool.cc:224] Handling call
0709 12:25:23.838456 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.325951 (+487495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.325989 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.328585 (+  2596us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.381333 (+ 52748us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:24.381421 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.381643 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:24.563346 (+181703us) raft_consensus.cc:1685] finished
0709 12:25:24.563364 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.563485 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23202,"raft.queue_time_us":75439,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2889,"thread_start_us":529851,"threads_started":2,"wal-append.queue_time_us":180741}
I20260709 12:25:24.582145 26195 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:24.577891 30309 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.553835 30268 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:24.589143 30421 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:25:24.590729 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.557353 30274 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.560601 30340 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.592950 30384 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.596129 26127 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:24.608701 26200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1949) took 2643 ms (client timeout 2487 ms). Trace:
W20260709 12:25:24.609138 26200 rpcz_store.cc:269] 0709 12:25:21.944977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.496982 (+552005us) service_pool.cc:224] Handling call
0709 12:25:24.581632 (+2084650us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:24.588011 (+  6379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2529198464}
I20260709 12:25:24.606072 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.611294 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:24.614114 30335 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.617692 30366 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:24.618769 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.552295 30385 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.595036 30274 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.595155 29961 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.625161 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.627126 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.627846 26124 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.629861 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:24.641489 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56632us
I20260709 12:25:24.643357 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.645952 30366 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:24.595893 30252 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:24.646550 30366 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:24.644692 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.648792 26123 raft_consensus.cc:2368] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:24.647552 30366 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.650079 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.651438 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.652401 26127 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:24.664556 26124 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:24.665099 30404 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:24.662328 30254 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:24.665920 30252 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.674157 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 638) took 1348 ms. Trace:
I20260709 12:25:24.661101 30278 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:25:24.662601 30163 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:25:24.672894 30081 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.683956 26122 rpcz_store.cc:276] 0709 12:25:23.325442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.797170 (+471728us) service_pool.cc:224] Handling call
0709 12:25:23.797609 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.884474 (+ 86865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.884512 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.899744 (+ 15232us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.617855 (+718111us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b18003fdf00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:24.661726 (+ 43871us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:24.661791 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.661913 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:24.671598 (+  9685us) raft_consensus.cc:1685] finished
0709 12:25:24.671620 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.674045 (+  2425us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11506,"raft.queue_time_us":81796,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":5802,"spinlock_wait_cycles":2824960,"thread_start_us":91621,"threads_started":2,"wal-append.queue_time_us":762930}
I20260709 12:25:24.684798 30254 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:24.687160 30075 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.700776 30254 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:24.704788 30385 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:24.697218 30075 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:24.725193 30075 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:24.725648 30075 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:24.726749 30163 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:25:24.735410 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2783) took 2107 ms. Trace:
I20260709 12:25:24.735925 26052 rpcz_store.cc:276] 0709 12:25:22.628037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.840879 (+212842us) service_pool.cc:224] Handling call
0709 12:25:22.841209 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.435680 (+594471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.735133 (+1299453us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b5400229a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:24.735335 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309600512}
I20260709 12:25:24.726567 30075 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.750670 30075 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:24.756697 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1959) took 2066 ms (client timeout 2405 ms). Trace:
I20260709 12:25:24.757369 26052 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:25:24.758404 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2789) took 1961 ms. Trace:
I20260709 12:25:24.758874 26052 rpcz_store.cc:276] 0709 12:25:22.797610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.748424 (+1950814us) service_pool.cc:224] Handling call
0709 12:25:24.757001 (+  8577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:24.758007 (+  1006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.758323 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.761433 30325 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:24.757177 26201 rpcz_store.cc:269] 0709 12:25:22.690700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.863482 (+172782us) service_pool.cc:224] Handling call
0709 12:25:24.756483 (+1893001us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400335c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:24.756630 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177716480}
I20260709 12:25:24.767481 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:24.770629 30433 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:24.771461 30433 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:24.772831 30433 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:24.779453 26052 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 12:25:24.781689 26050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 488) took 2281 ms (client timeout 2245 ms). Trace:
I20260709 12:25:24.785079 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2790) took 1969 ms. Trace:
I20260709 12:25:24.785550 26052 rpcz_store.cc:276] 0709 12:25:22.816088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.759932 (+1943844us) service_pool.cc:224] Handling call
0709 12:25:24.779189 (+ 19257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.784737 (+  5548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.785015 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:24.785106 26050 rpcz_store.cc:269] 0709 12:25:22.499824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.521843 (+ 22019us) service_pool.cc:224] Handling call
0709 12:25:24.781437 (+2259594us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:24.781576 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2526449024}
I20260709 12:25:24.790089 26050 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:25:24.790849 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2794) took 1948 ms. Trace:
I20260709 12:25:24.791213 26050 rpcz_store.cc:276] 0709 12:25:22.842643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.787251 (+1944608us) service_pool.cc:224] Handling call
0709 12:25:24.789780 (+  2529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.790624 (+   844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.790799 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15104}
I20260709 12:25:24.790143 26052 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:24.792925 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2793) took 1950 ms. Trace:
I20260709 12:25:24.793272 26052 rpcz_store.cc:276] 0709 12:25:22.842320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.786620 (+1944300us) service_pool.cc:224] Handling call
0709 12:25:24.789924 (+  3304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.792689 (+  2765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.792873 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.793392 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.795676 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.802377 26129 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:25:24.806584 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.807060 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.807358 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.810940 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1541) took 2793 ms (client timeout 1515 ms). Trace:
W20260709 12:25:24.811365 26053 rpcz_store.cc:269] 0709 12:25:22.018873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.025397 (+  6524us) service_pool.cc:224] Handling call
0709 12:25:24.810724 (+2785327us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b5400029254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:24.810872 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4971009280}
I20260709 12:25:24.812517 26050 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:24.813418 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2795) took 1970 ms. Trace:
I20260709 12:25:24.813864 26050 rpcz_store.cc:276] 0709 12:25:22.842933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.792085 (+1949152us) service_pool.cc:224] Handling call
0709 12:25:24.812266 (+ 20181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.813096 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.813346 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.817340 26052 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:24.818279 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1557) took 1904 ms. Trace:
I20260709 12:25:24.818722 26052 rpcz_store.cc:276] 0709 12:25:22.914144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.794227 (+1880083us) service_pool.cc:224] Handling call
0709 12:25:24.817062 (+ 22835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.817956 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.818208 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.821659 26053 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:24.822414 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1558) took 1907 ms. Trace:
I20260709 12:25:24.822810 26053 rpcz_store.cc:276] 0709 12:25:22.914525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.812214 (+1897689us) service_pool.cc:224] Handling call
0709 12:25:24.821445 (+  9231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.822150 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.822351 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.829262 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.829656 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:24.836513 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.840960 26129 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:24.841482 26050 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:24.842159 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1560) took 1926 ms. Trace:
I20260709 12:25:24.845709 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.844712 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:24.846617 26122 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.846048 26050 rpcz_store.cc:276] 0709 12:25:22.915284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.815058 (+1899774us) service_pool.cc:224] Handling call
0709 12:25:24.841126 (+ 26068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.841910 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.842103 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.848877 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:24.849728 26788 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:24.850747 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:24.852758 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:24.853037 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:24.856412 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:24.862948 26784 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:24.865425 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:24.866590 26050 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:24.867555 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1565) took 1819 ms. Trace:
I20260709 12:25:24.868019 26050 rpcz_store.cc:276] 0709 12:25:23.048541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.865877 (+1817336us) service_pool.cc:224] Handling call
0709 12:25:24.866319 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.867199 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.867461 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.868741 26053 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:25:24.874282 26053 pending_rounds.cc:85] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 10
I20260709 12:25:24.878973 26053 pending_rounds.cc:107] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:25:24.879351 26053 raft_consensus.cc:2889] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.882480 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.883145 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.884057 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:24.884371 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:24.884914 26786 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:24.885113 26789 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:24.885279 26050 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:24.886081 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1567) took 1790 ms. Trace:
I20260709 12:25:24.828665 30275 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:24.833673 30288 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.893962 26050 rpcz_store.cc:276] 0709 12:25:23.095698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.870597 (+1774899us) service_pool.cc:224] Handling call
0709 12:25:24.870920 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.885801 (+ 14881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.886018 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.887915 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.896778 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1564) took 1980 ms. Trace:
I20260709 12:25:24.833554 30075 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:25:24.897722 26053 rpcz_store.cc:276] 0709 12:25:22.916755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.823626 (+1906871us) service_pool.cc:224] Handling call
0709 12:25:24.844043 (+ 20417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.887769 (+ 43726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.896694 (+  8925us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:24.899890 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56437us
I20260709 12:25:24.902355 26053 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:24.903390 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 497) took 1532 ms. Trace:
I20260709 12:25:24.868731 30274 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:24.906703 30274 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:24.903924 26053 rpcz_store.cc:276] 0709 12:25:23.371166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.901488 (+1530322us) service_pool.cc:224] Handling call
0709 12:25:24.901899 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.903034 (+  1135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.903312 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.878342 30252 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:24.908664 30243 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:24.909224 30243 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:24.909258 30252 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:24.910269 30243 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.911685 30243 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.910868 30252 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.869531 30254 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.869056 30327 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.912652 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.905783 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:24.908699 26053 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:25:24.917517 26053 pending_rounds.cc:85] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 4
I20260709 12:25:24.888577 30289 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:24.897212 30366 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.917026 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51723.8us
I20260709 12:25:24.917069 26203 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.917891 26053 pending_rounds.cc:107] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:25:24.901100 26050 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:24.916568 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:24.941555 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:24.941375 26053 raft_consensus.cc:2889] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.942003 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 496) took 1571 ms. Trace:
I20260709 12:25:24.946604 30331 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:24.947209 30331 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:24.948170 30331 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:24.921036 29907 raft_consensus.cc:670] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:24.920675 30075 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:24.949716 30075 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.888290 30325 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:25:24.952452 30309 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:24.945431 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:24.954603 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.945827 26050 rpcz_store.cc:276] 0709 12:25:23.370888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.900184 (+1529296us) service_pool.cc:224] Handling call
0709 12:25:24.900849 (+   665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.941729 (+ 40880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.941951 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.896523 30385 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:25:24.899395 30432 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:24.956492 29511 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:24.940904 30289 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.960552 29511 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.907840 30274 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.901425 30288 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.921155 30269 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.952927 30309 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:24.959558 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.961079 29511 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.956913 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2801) took 1346 ms. Trace:
I20260709 12:25:24.943200 30268 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.962191 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.964336 26786 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:24.964643 26018 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.592s (SENT)
I20260709 12:25:24.965376 26053 rpcz_store.cc:276] 0709 12:25:23.610541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.907691 (+1297150us) service_pool.cc:224] Handling call
0709 12:25:24.908150 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.956440 (+ 48290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.956799 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.969533 26053 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:24.970089 26053 pending_rounds.cc:85] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 4
I20260709 12:25:24.970402 26053 pending_rounds.cc:107] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:24.970737 26053 raft_consensus.cc:2889] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.961437 26050 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:24.971999 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1570) took 1243 ms. Trace:
I20260709 12:25:24.972536 26053 rpcz_store.cc:276] 0709 12:25:23.728189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.968051 (+1239862us) service_pool.cc:224] Handling call
0709 12:25:24.968485 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.971651 (+  3166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.971921 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.977300 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:24.979555 26788 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:24.939492 30327 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.964870 30385 consensus_queue.cc:1243] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:24.979681 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2805) took 1361 ms. Trace:
I20260709 12:25:24.984709 30288 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:24.968396 30075 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:24.964144 30309 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.964718 30433 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:24.987479 26122 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:24.960803 30275 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.989357 26203 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:24.989780 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:24.921301 30254 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:25.008761 26019 leader_election.cc:336] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.013s (SENT)
I20260709 12:25:25.009569 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.010222 26201 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:25.010576 30254 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:24.996762 26050 rpcz_store.cc:276] 0709 12:25:23.617827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.959300 (+1341473us) service_pool.cc:224] Handling call
0709 12:25:24.961252 (+  1952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.979392 (+ 18140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.979621 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:24.996981 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:25.011992 26789 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.005424 26124 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:25.021857 26788 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:25:25.022728 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:25.024277 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:25.022876 26789 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:25.026780 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.035622 29511 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.052742 26789 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:25.003279 30433 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:24.978941 30268 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:24.945338 30366 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.068655 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:25.086115 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.083797 26093 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:25.071863 30289 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:25.097965 26019 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.405s (SENT)
I20260709 12:25:25.033097 30309 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:25.095598 30410 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:25.102415 26122 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:25.108649 26789 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:25:25.124833 26756 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.654s (SENT)
I20260709 12:25:25.100535 30410 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.102296 30327 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.138044 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1563) took 2221 ms. Trace:
I20260709 12:25:25.138613 26052 rpcz_store.cc:276] 0709 12:25:22.916340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.819735 (+1903395us) service_pool.cc:224] Handling call
0709 12:25:24.833755 (+ 14020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.834866 (+  1111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.834901 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.848757 (+ 13856us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.966932 (+118175us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:25.089086 (+122154us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.101100 (+ 12014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.101170 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.137805 (+ 36635us) raft_consensus.cc:1685] finished
0709 12:25:25.137825 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.137977 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":169053,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":12703,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":7867,"thread_start_us":159019,"threads_started":2,"wal-append.queue_time_us":164512}
I20260709 12:25:25.143334 30366 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:25.129063 30410 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:25.147004 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.098024 30433 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.150308 30433 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.150624 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1015) took 2112 ms. Trace:
I20260709 12:25:25.151095 26785 rpcz_store.cc:276] 0709 12:25:23.038038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.038177 (+   139us) service_pool.cc:224] Handling call
0709 12:25:23.038523 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.343137 (+304614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.150355 (+1807218us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54001162d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:25.150563 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698009856}
I20260709 12:25:25.156905 30254 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:25.158231 30254 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:25.158697 30254 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:25.161397 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.162281 26200 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:25.147697 26201 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:25.149082 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.172652 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 7.
I20260709 12:25:25.160137 30254 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.177636 30254 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:25.179153 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.180868 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.184800 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:25.185629 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.185967 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:25.186457 26785 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:25.185581 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:25.187242 26784 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:25.186390 26789 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:25.193239 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.194432 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:25.225354 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.226564 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:25.228803 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:25.229444 30453 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.229827 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:25.230290 30453 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.234496 30453 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.244057 30453 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.244834 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.245007 30453 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.246768 30035 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:25.252882 30035 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:25.253360 30035 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.253611 30384 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:25.255180 30384 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:25.255977 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:25.256713 30384 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:25.257678 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:25.258163 30384 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:25.251818 30456 raft_consensus.cc:2764] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:25.250550 30398 raft_consensus.cc:2764] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:25:25.257730 26199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1940) took 3454 ms (client timeout 2882 ms). Trace:
I20260709 12:25:25.259186 30456 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:25.259107 26199 rpcz_store.cc:269] 0709 12:25:21.827836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.836337 (+  8501us) service_pool.cc:224] Handling call
0709 12:25:25.257506 (+3421169us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.257653 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2631664896}
I20260709 12:25:25.259757 30456 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:25.260272 30456 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.262529 30035 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.264854 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:25.265014 30035 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.265734 30398 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:25.266309 30398 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:25.267432 30453 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.267207 30398 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.272861 30398 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:25.275806 30453 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:25.258697 30384 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:25.279628 30384 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:25.286149 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.288424 30431 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:25.289188 30431 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.289687 30456 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.292487 30066 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.293092 30066 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.295644 30066 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:25.297767 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.292003 30431 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.301249 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.302379 26049 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:25.303193 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.292523 30456 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:25.308032 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.307888 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.309379 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.310544 30270 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.310750 26199 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:25.311905 30270 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.313195 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.314023 26199 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:25.314271 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 1962) took 2622 ms. Trace:
I20260709 12:25:25.314905 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.315521 30270 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.316285 30270 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.317433 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1562) took 2401 ms (client timeout 2922 ms). Trace:
I20260709 12:25:25.317255 30270 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:25.317906 26047 rpcz_store.cc:269] 0709 12:25:22.915950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.181493 (+265543us) service_pool.cc:224] Handling call
0709 12:25:25.317206 (+2135713us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b5400021a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.317361 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990415232}
I20260709 12:25:25.314718 26197 rpcz_store.cc:276] 0709 12:25:22.691666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.141552 (+1449886us) service_pool.cc:224] Handling call
0709 12:25:24.323124 (+181572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.323733 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.314060 (+990327us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400216bd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.314213 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416695040}
I20260709 12:25:25.320197 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.321267 30270 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.321429 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.322244 26197 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:25:25.323073 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.323653 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.326465 26199 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:25.328123 26199 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:25.329303 26197 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:25.330850 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:25.331513 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:25.332489 30270 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:25.332796 30453 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.333002 30270 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.333405 30453 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.333604 30270 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.334110 30453 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.334436 30384 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.336956 30384 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.331722 26197 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:25:25.326527 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 730) took 1974 ms (client timeout 1592 ms). Trace:
I20260709 12:25:25.337565 30384 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 12:25:25.341557 26783 rpcz_store.cc:269] 0709 12:25:23.352802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.371805 (+ 19003us) service_pool.cc:224] Handling call
0709 12:25:25.326322 (+1954517us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54005708d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.326459 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361353088}
I20260709 12:25:25.345357 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.346664 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:25.343474 30453 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.349294 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.350373 30453 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.347806 30456 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:25.352452 26783 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:25.354019 26783 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEARNER]: Advancing to term 12
I20260709 12:25:25.355266 30431 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:25.356459 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.356788 30431 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:25.357203 30431 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:25.358132 30431 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.359483 30431 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.361364 26782 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:25.369372 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.375648 30270 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.378163 26782 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:25.386518 29511 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.392272 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:25.395128 26782 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.396549 30433 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:25.397428 30433 consensus_queue.cc:1243] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:25.400467 30270 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.403927 26782 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:25.405126 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:25.406319 30270 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:25.407176 30270 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:25.413949 30270 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:25.414963 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.418769 26783 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:25.421311 30431 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:25.432621 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47072.1us
W20260709 12:25:25.433341 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.440781 30431 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.441783 30431 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.445639 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:25.446365 26199 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:25.457527 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.457557 30456 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:25.459332 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:25.459921 26199 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.446298 30431 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.455799 30317 raft_consensus.cc:1064] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:25:25.462538 30317 raft_consensus.cc:1068] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:25.469597 26199 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:25.472539 30325 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:25.473258 30325 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.476459 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:25.477487 30384 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.477986 30384 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:25.478894 30384 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.480655 30325 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.487845 30325 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.488564 30325 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.491743 30325 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.517031 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:25.518110 30317 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.519524 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:25.523993 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:25.526714 26125 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:25.526523 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:25.528764 26124 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:25.529531 26120 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:25.530848 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:25.531348 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:25.533620 26126 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:25.531698 26120 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:25.535396 29511 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:25.537575 30289 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:25.540151 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:25.539898 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:25.541018 30431 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:25.536371 30354 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.541944 30317 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:25.542287 30354 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.542400 30317 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:25.542887 30354 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.544788 30317 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.540341 30066 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:25:25.546002 30317 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:25.532207 30243 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:25.547046 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2788) took 2833 ms (client timeout 1896 ms). Trace:
I20260709 12:25:25.547698 30243 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:25.550005 30243 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:25.553380 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1021) took 2411 ms (client timeout 2067 ms). Trace:
W20260709 12:25:25.553833 26790 rpcz_store.cc:269] 0709 12:25:23.142234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.143820 (+  1586us) service_pool.cc:224] Handling call
0709 12:25:25.553095 (+2409275us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.553259 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2841316736}
W20260709 12:25:25.547663 26054 rpcz_store.cc:269] 0709 12:25:22.713691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.732402 (+ 18711us) service_pool.cc:224] Handling call
0709 12:25:25.546832 (+2814430us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54003598d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.546977 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906183424}
I20260709 12:25:25.542994 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:25.558125 30066 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.543448 30431 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.558585 30066 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.565814 30035 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:25.565060 30066 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.566397 30035 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.569141 30035 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:25.560811 30254 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:25:25.594890 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.605353 30354 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.610215 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:25.610828 26124 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.612605 30354 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:25.613974 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2804) took 1996 ms. Trace:
I20260709 12:25:25.614382 26049 rpcz_store.cc:276] 0709 12:25:23.617452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.492859 (+875407us) service_pool.cc:224] Handling call
0709 12:25:25.613741 (+1120882us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b5400172854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:25.613911 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788412672}
I20260709 12:25:25.611240 30066 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.618599 30066 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:25.622035 26124 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:25.622439 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:25.623230 26120 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.627216 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:25.629241 30243 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.629814 30243 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:25.629952 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:25.630633 26200 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:25.630817 30243 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:25.633232 26120 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:25.633368 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:25.637287 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:25.680382 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.710889 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.641149 30370 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:25.638913 30317 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.717442 30254 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:25.717497 30317 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:25.759083 30317 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:25.718091 30254 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:25.799805 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.812531 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.836561 26094 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.922s (SENT) [suppressed 38 similar messages]
I20260709 12:25:25.714250 30370 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:25.848665 30370 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:25.913224 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.877401 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.869510 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.013898 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.887449 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50933.7us
I20260709 12:25:25.843186 30066 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:26.031842 30066 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:25.890360 30433 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:26.018656 30254 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.034539 30433 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.039881 30066 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.040916 30433 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:26.047358 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.048616 26019 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.882s (SENT)
I20260709 12:25:26.049052 30404 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:26.049089 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.050025 30404 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:26.050441 30402 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:26.053548 30370 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:26.055001 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 493) took 2711 ms (client timeout 2885 ms). Trace:
W20260709 12:25:26.055502 26051 rpcz_store.cc:269] 0709 12:25:23.343212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.333010 (+989798us) service_pool.cc:224] Handling call
0709 12:25:26.054762 (+1721752us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400171e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:26.054931 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330085632}
I20260709 12:25:26.101891 26127 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:25:26.137913 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.163332 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.225108 26019 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.818s (SENT)
I20260709 12:25:26.236305 29511 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.236601 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 645) took 2554 ms. Trace:
I20260709 12:25:26.236925 29511 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.237071 26195 rpcz_store.cc:276] 0709 12:25:23.682603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.328515 (+645912us) service_pool.cc:224] Handling call
0709 12:25:24.581791 (+253276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.582609 (+   818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.236355 (+1653746us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002b4b50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:26.236521 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581828352}
I20260709 12:25:26.055658 30370 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:26.247963 30355 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.238770 30269 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.251865 29511 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.251505 30269 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.238842 30243 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:25:26.254828 30243 consensus_queue.cc:1243] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:26.254949 30269 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:26.256524 30399 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:25:26.257797 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:26.258271 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.259415 26123 pending_rounds.cc:85] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 6
I20260709 12:25:26.259712 26123 pending_rounds.cc:107] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:25:26.259975 26123 raft_consensus.cc:2889] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:26.263670 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.264374 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.264681 26049 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:26.264962 26047 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.263751 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.265364 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:26.266003 26054 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:26.263550 30369 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:26.268870 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.269945 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.270234 26049 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:26.268112 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.275308 26054 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.276611 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.277248 26049 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:26.278594 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:26.279114 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.279774 26053 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:26.266019 26127 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:26.269968 30369 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:26.272045 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.286877 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.287670 26050 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:26.281486 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.289388 26053 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.281802 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:26.283125 26054 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:26.285010 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.291699 26046 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:26.295368 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.296262 26045 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:26.309249 26049 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:26.310706 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.295279 30453 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:26.313189 30453 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.315346 30453 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:26.319644 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49298.5us
I20260709 12:25:26.321734 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:26.321048 24867 debug-util.cc:398] Leaking SignalData structure 0x7b080037b8c0 after lost signal to thread 26020
I20260709 12:25:26.311728 30243 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:26.327041 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 499) took 2924 ms (client timeout 1654 ms). Trace:
I20260709 12:25:26.325841 30243 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.329052 30243 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.331125 30356 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:25:26.326201 24867 kernel_stack_watchdog.cc:198] Thread 26020 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 764ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:25:26.333485 30485 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.334012 30485 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.336401 30369 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.338861 30369 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:26.341502 26052 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
W20260709 12:25:26.327647 26048 rpcz_store.cc:269] 0709 12:25:23.402660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.454605 (+ 51945us) service_pool.cc:224] Handling call
0709 12:25:26.326763 (+2872158us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b540022b354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:26.326914 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2063595392}
I20260709 12:25:26.344683 26053 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:26.346190 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.346946 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.352589 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72131.2us
I20260709 12:25:26.336162 30485 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.353732 26786 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.354041 30433 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:26.354581 30433 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:26.352958 30404 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.357110 30433 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.358018 30433 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.358595 30433 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:26.359684 30468 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.517s	user 0.017s	sys 0.009s
I20260709 12:25:26.360899 30433 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:26.361510 26168 connection.cc:256] RPC call timeout handler was delayed by -1.2709276390000923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 12:25:26.361634 30404 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:26.362015 26168 connection.cc:256] RPC call timeout handler was delayed by -1.1854276389999541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 12:25:26.362567 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0738276390002284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 12:25:26.365489 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.366261 26200 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.367620 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:26.369144 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.375337 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.375990 26195 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:26.378886 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.379271 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.379835 26195 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:26.380169 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.380885 26786 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:26.381302 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.382012 26199 raft_consensus.cc:2368] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:26.382565 26203 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.382750 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.383824 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.384392 26199 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:26.385289 30243 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.387212 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1020) took 3249 ms. Trace:
I20260709 12:25:26.388733 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:26.389364 26123 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.391177 26200 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:26.388713 30485 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 12:25:26.395370 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.308s	user 0.015s	sys 0.018s
W20260709 12:25:26.402271 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.315s	user 0.016s	sys 0.020s
W20260709 12:25:26.402855 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 1.31903e+06us
W20260709 12:25:26.398679 26785 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.212s	user 0.016s	sys 0.003s
I20260709 12:25:26.404768 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:26.387610 26787 rpcz_store.cc:276] 0709 12:25:23.137789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.138243 (+   454us) service_pool.cc:224] Handling call
0709 12:25:23.138587 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.324357 (+185770us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:23.332022 (+  7665us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.357232 (+ 25210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:23.357300 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:23.357358 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.386979 (+3029621us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:26.387036 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.387153 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10918,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":9649,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":426620288,"thread_start_us":24867,"threads_started":1,"wal-append.queue_time_us":25152,"wal-append.run_cpu_time_us":1390,"wal-append.run_wall_time_us":1007871}
I20260709 12:25:26.405944 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 739) took 1223 ms. Trace:
I20260709 12:25:26.407711 26785 rpcz_store.cc:276] 0709 12:25:25.182475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.185724 (+  3249us) service_pool.cc:224] Handling call
0709 12:25:26.405633 (+1219909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.404011 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:26.398583 30485 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:26.410079 30485 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:26.411123 30485 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.398145 30356 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:25:26.409013 26123 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:26.415313 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:26.416436 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:26.417223 29511 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:26.417294 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.418169 29511 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:26.400636 30269 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:26.423732 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.423568 30269 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.426819 30269 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.429155 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:26.419339 29511 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.393038 30243 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.421587 30327 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:25:26.421113 30453 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:26.435945 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.439602 30453 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:26.441708 30410 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.294s	user 0.018s	sys 0.013s
I20260709 12:25:26.412904 30404 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:25:26.443071 30434 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:26.443269 26203 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:26.449874 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.450481 26203 raft_consensus.cc:2376] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11. Re-sending same reply.
I20260709 12:25:26.449896 30434 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:26.451342 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.451920 26203 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:26.443852 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.452397 30410 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.451613 30434 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:26.461349 30434 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.461498 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:26.464272 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:26.464329 30370 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:26.465116 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2795) took 3517 ms (client timeout 2992 ms). Trace:
I20260709 12:25:26.465564 30474 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:26.466598 30410 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.455339 30404 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:26.456589 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.457650 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.453802 30453 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.436924 30485 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:26.465498 26128 rpcz_store.cc:269] 0709 12:25:22.947625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.087966 (+140341us) service_pool.cc:224] Handling call
0709 12:25:26.464909 (+3376943us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54001f81d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:26.465044 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1766534528}
I20260709 12:25:26.457954 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.482745 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:26.483635 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.468845 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.468922 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:26.471395 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:26.465466 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.470150 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:26.485605 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.488226 26791 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:25:26.489223 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:26.455921 30268 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.476893 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:26.486364 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:26.491991 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:26.477039 26790 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.492458 26783 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:26.484460 26203 raft_consensus.cc:2376] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8. Re-sending same reply.
I20260709 12:25:26.493338 26786 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [6.8-9.9]
I20260709 12:25:26.493790 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.494406 26203 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.485589 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.495599 26200 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:26.497750 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.499233 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.500885 30433 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.501366 30433 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:26.501762 30433 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.504408 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.505055 26200 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.505221 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:26.505939 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.507203 26199 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:26.507344 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.508132 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.512574 26199 raft_consensus.cc:2368] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:26.514719 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.515791 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:26.517050 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.518473 26199 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:26.477146 30404 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:26.519663 26200 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:26.477375 30366 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:26.522049 29511 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:26.537680 29511 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:26.538316 29511 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:26.520834 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.521009 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.520040 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.522010 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:26.532845 30355 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.476711 30327 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:25:26.481714 30262 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:26.541189 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.484182 30474 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:25:26.542943 29511 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:26.495532 30289 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:26.543615 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.545182 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.545193 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.546599 26200 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:26.546299 30355 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.548480 30355 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.534032 30431 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:26.549253 30066 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:26.549598 30431 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:26.549939 30433 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:26.545567 29511 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:26.550505 30431 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:26.551028 30431 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:26.551415 29511 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.550482 30366 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:25:26.553424 30262 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.555764 30262 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:26.550333 26203 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:26.556591 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:26.557613 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:26.555809 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1586) took 1229 ms. Trace:
I20260709 12:25:26.558336 26197 rpcz_store.cc:276] 0709 12:25:25.325974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.338676 (+ 12702us) service_pool.cc:224] Handling call
0709 12:25:25.339268 (+   592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.339920 (+   652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.339949 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.340471 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.555274 (+1214803us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b540031cca0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b7431c6b8 00007f5b743a592b 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c
0709 12:25:26.555346 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:26.555391 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.555471 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.555512 (+    41us) raft_consensus.cc:1685] finished
0709 12:25:26.555526 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.555752 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":56945024,"thread_start_us":74705,"threads_started":1,"wal-append.queue_time_us":73049,"wal-append.run_cpu_time_us":1906,"wal-append.run_wall_time_us":1020258}
I20260709 12:25:26.561235 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:26.571623 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.562422 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:26.495858 30243 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:25:26.536665 30269 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:26.566229 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.578217 30370 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.579069 30370 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.581123 30370 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.564630 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.582578 26199 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11. Re-sending same reply.
I20260709 12:25:26.583683 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:26.584582 26199 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:26.561908 26200 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:26.573096 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.574152 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:26.566295 30360 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.551843 30327 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:26.608407 30327 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:26.611271 30327 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:26.546314 30289 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:26.587898 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.552043 30433 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:26.589548 26200 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:26.590783 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:26.691083 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:26.582638 30066 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:26.576608 30305 raft_consensus.cc:2764] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:26.594056 26203 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:26.598594 26790 raft_consensus.cc:1217] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 8.9->[9.10-12.11]   Dedup: 9.10->[12.11-12.11]
I20260709 12:25:26.721426 26203 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:26.603842 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.566475 30366 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:26.606375 30269 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:26.567262 30262 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:26.721478 26790 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [9.10-12.11]
I20260709 12:25:26.723052 26203 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:26.729694 26203 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:25:26.692478 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.731964 26203 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:26.734027 26203 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:25:26.724255 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 120464us
I20260709 12:25:26.727425 26200 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:25:26.737107 26200 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:26.543072 30404 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.551852 30431 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:26.738802 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:26.728348 26199 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:26.743835 29511 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:26.744284 29511 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:26.744658 29511 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:26.723839 30366 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.725574 30269 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:26.746455 26197 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:26.733863 30066 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:26.745138 30305 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:26.757320 30305 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.757855 30305 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.761507 30066 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:26.748400 30269 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.747517 30262 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:25:26.775074 30262 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:26.782764 30262 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:26.783218 30262 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.783668 30262 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:26.791036 30305 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.797214 30163 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:26.748929 30366 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:26.807615 30269 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.825476 30289 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.829618 29511 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.831172 30404 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:26.832026 30431 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:26.830911 30305 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.845044 29511 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:26.845916 29511 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:26.847573 29511 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:26.813464 30262 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.874703 30498 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:26.877152 30498 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:26.926831 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1955) took 4238 ms (client timeout 2083 ms). Trace:
W20260709 12:25:26.927263 26198 rpcz_store.cc:269] 0709 12:25:22.688959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.797963 (+109004us) service_pool.cc:224] Handling call
0709 12:25:26.926586 (+4128623us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400302854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:26.926740 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469206784}
W20260709 12:25:26.931574 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.932188 26019 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.772s (SENT)
W20260709 12:25:26.933714 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.948415 26020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 2.396s	user 0.013s	sys 0.015s
W20260709 12:25:26.949102 26020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.396s	user 0.013s	sys 0.015s
W20260709 12:25:26.949460 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 2.39654e+06us
W20260709 12:25:26.959391 26020 connection.cc:256] RPC call timeout handler was delayed by -1.3468141469998045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:25:26.959981 26020 connection.cc:256] RPC call timeout handler was delayed by -1.1830141469999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:25:26.960454 26020 connection.cc:256] RPC call timeout handler was delayed by -1.150614146999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 12:25:26.960805 26020 connection.cc:256] RPC call timeout handler was delayed by -1.0993141469998591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 12:25:26.961501 26020 connection.cc:256] RPC call timeout handler was delayed by -1.0759141469998212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 12:25:26.961864 26020 connection.cc:256] RPC call timeout handler was delayed by -1.0588141469997936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 12:25:26.962214 26020 connection.cc:256] RPC call timeout handler was delayed by -1.0579141469997921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:25:26.962543 26020 connection.cc:256] RPC call timeout handler was delayed by -1.0507141469997805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 12:25:26.963115 30268 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.964316 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.963718 30268 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.965548 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.967597 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.967813 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.968755 26053 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:26.970521 30268 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.971024 30268 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.971390 30268 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:26.973528 30268 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:26.974001 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:26.974725 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.975557 26053 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:26.975528 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:26.978714 26784 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.791s	user 0.010s	sys 0.002s
I20260709 12:25:26.979843 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
W20260709 12:25:26.980520 26784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 742) took 1797 ms (client timeout 1585 ms). Trace:
I20260709 12:25:26.980801 26053 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:26.980890 26784 rpcz_store.cc:269] 0709 12:25:25.183461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.185290 (+  1829us) service_pool.cc:224] Handling call
0709 12:25:26.980452 (+1795162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304}
I20260709 12:25:26.982259 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.983034 26053 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:26.983855 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.984462 26053 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:26.988328 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:26.984673 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.989877 26194 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:26.989697 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.990756 26052 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:26.994325 30485 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.001s
W20260709 12:25:26.984920 26788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.803s	user 0.021s	sys 0.041s
W20260709 12:25:26.995352 26788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.814s	user 0.022s	sys 0.041s
I20260709 12:25:26.995502 26053 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:26.986529 26020 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:26.832080 30269 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:26.985500 30268 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:26.996471 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:26.990736 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1569) took 1693 ms. Trace:
I20260709 12:25:26.998277 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 737) took 1818 ms. Trace:
I20260709 12:25:26.998988 26046 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:26.998782 26194 rpcz_store.cc:276] 0709 12:25:25.297642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.989271 (+1691629us) service_pool.cc:224] Handling call
0709 12:25:26.989624 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.990463 (+   839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.990676 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.997715 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:27.000789 30268 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:26.998832 26053 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.998988 26788 rpcz_store.cc:276] 0709 12:25:25.180031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.180172 (+   141us) service_pool.cc:224] Handling call
0709 12:25:25.180545 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.180987 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.181013 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.181491 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.995769 (+1814278us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:26.996114 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.997938 (+  1824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.998046 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.998101 (+    55us) raft_consensus.cc:1685] finished
0709 12:25:26.998112 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.998222 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":426,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":33,"raft.queue_time_us":1322,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"spinlock_wait_cycles":845568,"thread_start_us":1815102,"threads_started":2,"wal-append.queue_time_us":60350,"wal-append.run_cpu_time_us":2250,"wal-append.run_wall_time_us":1006262}
I20260709 12:25:27.001739 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:26.998745 26052 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:27.002589 30485 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.003194 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.003818 26052 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10. Re-sending same reply.
I20260709 12:25:27.000595 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.005373 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.006147 26046 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:27.005702 30485 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:27.006476 30268 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.008036 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.008464 30268 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.009711 26053 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:27.010386 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.010690 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.011885 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.012302 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.012535 26053 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:27.013494 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.014187 26053 raft_consensus.cc:2368] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12.
I20260709 12:25:27.007354 30485 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:27.014892 30485 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:27.015345 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.007014 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.016021 26053 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:27.007794 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.015826 30485 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.017103 26052 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.017518 30485 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:27.014119 26020 leader_election.cc:400] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:27.017191 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.016150 30268 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.018692 26053 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:27.019618 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.020768 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.020853 30366 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.021845 26053 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:27.022922 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.023236 30366 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.023780 26053 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 8: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:27.020839 30268 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.025512 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.024312 26020 leader_election.cc:403] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.029382 26053 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:27.029517 30268 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:27.030088 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:27.033464 26053 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:27.031522 30268 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.039378 30268 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.040290 30268 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:27.040653 26052 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:27.040966 30268 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:27.041749 30268 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:27.042738 30268 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.051983 26053 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:27.052958 26053 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:27.053755 26053 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.054960 26053 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.016542 26046 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:27.057449 26046 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:27.058756 26046 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:27.059914 26046 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:27.062505 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.064807 26053 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:25:27.065186 26053 pending_rounds.cc:85] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 5
I20260709 12:25:27.065384 26053 pending_rounds.cc:107] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:25:27.065614 26053 raft_consensus.cc:2889] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.068790 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.069600 26121 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:27.071308 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.145354 26052 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:25:27.214880 30434 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.753s	user 0.012s	sys 0.003s
I20260709 12:25:27.238492 26054 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:25:27.251935 26019 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.083s (SENT)
W20260709 12:25:27.253289 26019 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.789s (SENT)
I20260709 12:25:27.253739 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:27.358557 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.358536 30356 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:27.359323 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.427785 26124 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:27.427873 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.428282 26124 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:27.429055 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.429003 26124 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.429621 26125 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.430124 26124 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.430651 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.431769 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.432729 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.433612 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.434202 26123 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:27.434753 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:27.435192 26123 raft_consensus.cc:606] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:27.438055 26121 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:27.439980 26121 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:27.441823 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.437271 26125 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:27.442029 26124 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:25:27.443042 26124 pending_rounds.cc:85] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 10
I20260709 12:25:27.443066 30370 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:27.443495 26124 pending_rounds.cc:107] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:25:27.443820 26124 raft_consensus.cc:2889] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.443784 30370 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.445837 30370 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:27.446010 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.446302 30370 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.446856 26123 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:25:27.446748 30370 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:27.447908 29837 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:27.448801 30370 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:27.450222 25940 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 2.182s	user 0.004s	sys 0.002s
I20260709 12:25:27.450606 29837 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.455758 29837 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:27.456411 26124 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:27.462424 29837 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:27.463562 30355 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:27.464113 30355 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:27.467244 30355 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.468556 30355 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.468851 25940 maintenance_manager.cc:643] P 5eede3841e1242afbeab30840148b558: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 13.129s	user 0.227s	sys 0.098s Metrics: {"bytes_written":29473,"cfile_init":1,"dirs.queue_time_us":421064,"dirs.run_cpu_time_us":498,"dirs.run_wall_time_us":15427,"drs_written":1,"lbm_read_time_us":472,"lbm_reads_lt_1ms":4,"lbm_write_time_us":607064,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":11,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":17,"peak_mem_usage":0,"rows_written":63,"thread_start_us":312271,"threads_started":1}
I20260709 12:25:27.470309 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.477502 30262 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.479241 30370 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.479828 30370 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.482120 30370 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.481954 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.483006 26122 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:27.484342 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:27.485133 30402 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.485549 30402 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.485903 30402 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:27.488988 24867 debug-util.cc:398] Leaking SignalData structure 0x7b0800380680 after lost signal to thread 26020
W20260709 12:25:27.490969 26075 debug-util.cc:398] Leaking SignalData structure 0x7b08000bcbe0 after lost signal to thread 24865
W20260709 12:25:27.492074 26075 debug-util.cc:398] Leaking SignalData structure 0x7b08000bca80 after lost signal to thread 25986
I20260709 12:25:27.492143 30402 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:27.493466 26075 debug-util.cc:398] Leaking SignalData structure 0x7b08000de440 after lost signal to thread 26078
I20260709 12:25:27.494961 30402 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.495514 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.496306 26122 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.501088 26122 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:27.502135 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:27.503083 30402 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:27.503692 30402 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:27.504716 30402 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.506453 26195 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:27.508993 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 1999) took 1050 ms. Trace:
I20260709 12:25:27.509394 26195 rpcz_store.cc:276] 0709 12:25:26.458566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.483398 (+ 24832us) service_pool.cc:224] Handling call
0709 12:25:27.508727 (+1025329us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400302354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:27.508906 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350170496}
W20260709 12:25:27.510926 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.538029 30262 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.538695 30262 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.541131 30262 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:27.546636 30431 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.005s
W20260709 12:25:27.548916 30066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.006s	sys 0.001s
W20260709 12:25:27.558470 30066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.007s	sys 0.001s
W20260709 12:25:27.553550 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.567752 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.570247 30370 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:27.570859 30370 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.573119 30370 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.574301 30398 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:27.574892 30398 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.577665 30398 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.578329 30398 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.578900 30398 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.579412 26201 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:27.600443 30398 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.604086 30262 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:27.604717 30262 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.606195 26048 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:27.606791 30262 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:27.607190 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.612638 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.613219 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.613473 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.613876 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.614104 26125 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.614449 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.613416 26123 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:27.615028 26128 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:27.618712 30275 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:27.619626 30275 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:27.624118 26125 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:27.619969 30275 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:27.621062 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:27.626392 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:27.627355 30269 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:27.626089 30275 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.627985 30360 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:27.628639 30360 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.628901 30275 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:27.627453 30262 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.630600 30262 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:27.633054 30262 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.635480 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.636263 26047 raft_consensus.cc:2368] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11.
W20260709 12:25:27.639506 26019 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.590s (SENT)
I20260709 12:25:27.640048 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:27.642026 30147 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:27.628041 30269 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:27.647174 30269 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.659590 30262 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.666484 30066 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:27.667095 30066 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.665469 30360 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.668893 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.669759 26120 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:27.669633 30066 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:27.670965 30262 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.671435 30369 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.672533 30075 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.672652 30369 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.673121 30075 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:27.674393 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.038s	sys 0.071s
W20260709 12:25:27.674726 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.038s	sys 0.071s
I20260709 12:25:27.675278 30075 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.671993 30360 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:27.676378 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 924551us
I20260709 12:25:27.677136 26124 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:27.677718 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:27.678439 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.679092 26120 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.681666 29837 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:27.684414 30289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.007s	sys 0.000s
W20260709 12:25:27.684800 30289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.007s	sys 0.000s
I20260709 12:25:27.685091 30269 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:27.675009 30369 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.675808 30075 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.696643 30075 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.697636 30264 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.698763 30075 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.705159 25941 maintenance_manager.cc:419] P 5eede3841e1242afbeab30840148b558: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9435 bytes on disk
I20260709 12:25:27.706224 26120 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:27.707604 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:27.708561 30269 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:27.709089 30269 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:27.710000 30269 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:27.715474 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.719121 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.727203 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:27.727849 26120 raft_consensus.cc:606] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:27.728294 26120 raft_consensus.cc:614] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:27.727974 25940 maintenance_manager.cc:643] P 5eede3841e1242afbeab30840148b558: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":346,"lbm_reads_lt_1ms":4}
I20260709 12:25:27.747623 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:27.748522 26120 raft_consensus.cc:606] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:27.748978 26120 raft_consensus.cc:614] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:27.757529 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:27.758180 26120 raft_consensus.cc:606] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:27.759135 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1558482039999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 12:25:27.759675 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:25:27.759701 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1126482039999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 12:25:27.761369 26756 connection.cc:256] RPC call timeout handler was delayed by -1.0118482039997616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
W20260709 12:25:27.767962 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.769133 26120 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:27.770542 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:27.773543 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.775054 30262 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:27.778367 26075 debug-util.cc:398] Leaking SignalData structure 0x7b0800129e20 after lost signal to thread 26153
W20260709 12:25:27.778741 26075 debug-util.cc:398] Leaking SignalData structure 0x7b080024d840 after lost signal to thread 26154
W20260709 12:25:27.779788 26075 debug-util.cc:398] Leaking SignalData structure 0x7b080037be60 after lost signal to thread 26227
W20260709 12:25:27.780946 26075 debug-util.cc:398] Leaking SignalData structure 0x7b0800356080 after lost signal to thread 26815
I20260709 12:25:27.781404 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:27.781687 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:27.782487 26784 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:27.782902 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.783131 26791 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:27.783725 30269 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.784696 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1570) took 2486 ms. Trace:
I20260709 12:25:27.785080 26201 rpcz_store.cc:276] 0709 12:25:25.297998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.322040 (+ 24042us) service_pool.cc:224] Handling call
0709 12:25:27.578951 (+2256911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.579945 (+   994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.784520 (+204575us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b5400302850. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:27.784641 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5189424000}
I20260709 12:25:27.785486 30269 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.787762 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.793344 30262 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.793824 30262 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:27.794234 30262 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.800896 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:27.811192 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.343s	user 0.021s	sys 0.038s
W20260709 12:25:27.811669 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.343s	user 0.022s	sys 0.038s
I20260709 12:25:27.812043 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.814023 30498 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:27.814430 30498 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.814775 30498 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.815837 30275 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:27.817854 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1565) took 3790 ms. Trace:
W20260709 12:25:27.824185 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.337s	user 0.021s	sys 0.017s
W20260709 12:25:27.827314 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.340s	user 0.021s	sys 0.019s
I20260709 12:25:27.827466 30288 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:27.828076 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.804159 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.804796 26122 raft_consensus.cc:1217] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-10.7]   Dedup: 10.7->[]
I20260709 12:25:27.831285 26122 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:27.832028 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:27.805217 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.832857 26783 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:27.809262 30075 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.807202 30243 raft_consensus.cc:2764] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:27.821214 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:27.825778 30330 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
W20260709 12:25:27.836688 26789 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 744) took 2504 ms (client timeout 1547 ms). Trace:
I20260709 12:25:27.803017 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.838198 26125 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:27.826478 30275 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:25:27.839766 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 932) took 1361 ms. Trace:
I20260709 12:25:27.840157 26790 rpcz_store.cc:276] 0709 12:25:26.481705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.505973 (+ 24268us) service_pool.cc:224] Handling call
0709 12:25:26.598374 (+ 92401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:26.728519 (+130145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.839568 (+1111049us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b5400433fd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:27.839709 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142872832}
I20260709 12:25:27.841928 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:27.803845 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.846812 26785 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:27.847019 30498 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.823276 30369 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:27.824115 30360 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:27.833117 26121 raft_consensus.cc:2368] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:27.848315 26788 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 12:25:27.837224 26789 rpcz_store.cc:269] 0709 12:25:25.331978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.836424 (+2504446us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:27.836585 (+   161us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:25:27.848881 30369 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.825224 26791 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.848474 30360 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.851366 30369 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.853097 30360 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:27.825351 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:27.833840 30075 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.826773 26202 rpcz_store.cc:276] 0709 12:25:24.027564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.540342 (+512778us) service_pool.cc:224] Handling call
0709 12:25:24.549084 (+  8742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.625295 (+ 76211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.625340 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.642095 (+ 16755us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.718315 (+ 76220us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:24.718355 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.817474 (+3099119us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b54002189d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:27.817525 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.817565 (+    40us) raft_consensus.cc:1685] finished
0709 12:25:27.817576 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.817799 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174658432,"thread_start_us":57010,"threads_started":1,"wal-append.queue_time_us":125002,"wal-append.run_cpu_time_us":2018,"wal-append.run_wall_time_us":1002032}
I20260709 12:25:27.856640 30075 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:27.804785 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:27.857436 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:27.858153 26783 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:27.811978 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.34373e+06us
I20260709 12:25:27.865146 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.865974 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.866631 26050 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:27.867017 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.867867 26053 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:25:27.868129 26047 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:25:27.868897 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.872036 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.872620 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.873217 26195 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:27.872298 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016d6c0 after lost signal to thread 24865
W20260709 12:25:27.874905 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016a1c0 after lost signal to thread 25949
I20260709 12:25:27.875226 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.876305 26198 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:27.879070 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:27.883486 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.193s	user 0.040s	sys 0.075s
I20260709 12:25:27.868260 26053 pending_rounds.cc:85] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 9
I20260709 12:25:27.883984 26053 pending_rounds.cc:107] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:25:27.884322 26053 raft_consensus.cc:2889] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:27.875635 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016dae0 after lost signal to thread 25986
W20260709 12:25:27.885849 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016d540 after lost signal to thread 26006
I20260709 12:25:27.886132 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.887121 26201 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:27.888165 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.886163 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016d500 after lost signal to thread 26020
I20260709 12:25:27.893724 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:27.895098 26198 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:27.890889 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.818154 30262 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:27.901654 26123 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:27.902127 26121 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:25:27.903589 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2210966700004064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 12:25:27.902629 26169 leader_election.cc:400] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
W20260709 12:25:27.904023 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1617364650001036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
W20260709 12:25:27.904489 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0744364650004172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 12:25:27.904368 26169 leader_election.cc:403] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:27.904949 26094 connection.cc:256] RPC call timeout handler was delayed by -1.038436465000359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
I20260709 12:25:27.900116 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:27.907124 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.910094 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:27.910392 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.040s	sys 0.075s
W20260709 12:25:27.911392 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 1.32347e+06us
I20260709 12:25:27.912917 30276 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:27.917402 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:27.918205 30495 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:27.919198 30360 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.919900 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.929456 26048 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:27.929293 30360 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.920926 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.922420 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:27.919323 30276 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:27.944016 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.944526 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:27.944963 26783 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.924811 30355 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.928416 30264 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:27.931499 30360 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:27.931883 30453 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.944526 30276 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:27.955201 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:27.955302 30276 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:27.956085 26047 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:27.956679 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.957815 26048 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:27.923563 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.934649 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.961877 26123 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:27.962764 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.921671 30075 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.938783 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.973829 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:27.946943 26050 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:27.947341 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.947712 26198 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:27.944034 26201 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [11.10-14.12]
I20260709 12:25:27.949709 26120 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:27.950816 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.979439 26051 raft_consensus.cc:2376] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10. Re-sending same reply.
I20260709 12:25:27.952665 30498 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:27.953766 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:27.937394 26053 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:27.964538 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:27.965378 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:27.966123 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.966998 26195 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
W20260709 12:25:27.985882 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.968390 26045 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.968276 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.986868 26195 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:27.933415 30369 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.989338 26201 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:27.944320 30384 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:27.975728 26121 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
W20260709 12:25:27.986202 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.996521 26019 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.673s (SENT)
I20260709 12:25:27.996927 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:27.997406 26019 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.585s (SENT)
I20260709 12:25:27.977366 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:27.989025 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:27.948782 30434 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:27.933794 30262 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:27.981479 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:27.983476 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.993232 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:27.993961 30527 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:27.961112 30495 consensus_queue.cc:1243] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:27.996392 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.002262 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.001724 26194 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:27.973356 30276 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.017554 30294 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:27.962936 30370 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:27.958474 30355 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.014803 26045 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:27.989820 30360 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:27.966619 30404 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:25:28.001741 30434 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:28.025656 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:28.032109 26194 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:27.997992 30431 raft_consensus.cc:2764] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:28.035312 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:27.969201 30453 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:25:28.010924 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:28.038900 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80000.8us
I20260709 12:25:28.042311 26787 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:28.046159 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.036113 30402 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.040661 30294 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:28.062822 30453 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.063797 30453 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:28.064181 30453 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.065024 30453 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.055394 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.041116 30434 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.001693 30369 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.052942 30456 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:28.045842 30370 consensus_queue.cc:1243] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:25:28.069691 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80750.2us
I20260709 12:25:28.088804 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.068931 30264 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:25:28.070398 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.090139 26787 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:28.071846 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.072490 30529 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:28.073875 30369 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.103896 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.104962 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.078917 26126 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.082185 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:28.090847 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.105001 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016dea0 after lost signal to thread 26078
W20260709 12:25:28.107523 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080016de80 after lost signal to thread 26079
I20260709 12:25:28.061321 30294 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.108768 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2086) took 1053 ms. Trace:
I20260709 12:25:28.094729 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.098659 29511 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.112066 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.115890 29511 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.116508 26790 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:28.111943 26129 rpcz_store.cc:276] 0709 12:25:27.055608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.081829 (+1026221us) service_pool.cc:224] Handling call
0709 12:25:28.108712 (+ 26883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:28.092587 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.109448 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:28.110316 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:28.118185 26123 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:28.118460 26049 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.118548 26051 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.102324 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.119902 26782 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:28.106410 29961 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:25:28.123128 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.123536 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.124353 26129 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:28.126772 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.084705 30456 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.133941 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.135746 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.157523 26126 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:28.159096 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:28.087733 30269 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:25:28.169752 30521 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:25:28.170442 29961 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:28.170940 29961 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:28.171904 29961 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.173305 29961 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.159355 30456 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.106245 30268 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:28.160787 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50972.6us
I20260709 12:25:28.177265 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.108340 30370 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.106660 30453 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.163406 26787 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:28.108536 30495 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:28.161841 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080011c080 after lost signal to thread 26153
I20260709 12:25:28.103148 30397 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:25:28.185896 30275 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.186581 30275 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.187086 30275 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.187525 30294 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.189047 30294 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.159322 30243 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:28.198541 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:28.180208 26051 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
W20260709 12:25:28.183657 26150 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbb20 after lost signal to thread 26154
I20260709 12:25:28.116976 30404 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:28.204425 30404 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.205372 30404 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.205580 30456 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.206740 30404 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.207888 30456 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.159510 30262 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:25:28.184633 30075 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
W20260709 12:25:28.208978 26150 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2180 after lost signal to thread 26227
I20260709 12:25:28.208978 30264 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:25:28.222465 26150 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6a80 after lost signal to thread 26815
W20260709 12:25:28.223419 26150 debug-util.cc:398] Leaking SignalData structure 0x7b080000f6c0 after lost signal to thread 30437
W20260709 12:25:28.223805 26150 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5ba0 after lost signal to thread 30454
I20260709 12:25:28.224754 30243 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:28.225835 30243 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:28.227340 30523 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 12:25:28.183930 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50061.2us
I20260709 12:25:28.228957 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.230237 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.209472 30381 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:28.211359 30355 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:28.224109 26150 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1f40 after lost signal to thread 30470
W20260709 12:25:28.213223 30525 consensus_queue.cc:1175] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 10 status { last_received { term: 6 index: 7 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:28.214821 29511 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:28.234279 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f640 after lost signal to thread 24865
W20260709 12:25:28.238547 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f460 after lost signal to thread 25949
W20260709 12:25:28.239427 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f6c0 after lost signal to thread 25986
W20260709 12:25:28.240007 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080014d840 after lost signal to thread 26006
I20260709 12:25:28.244843 29511 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:28.251123 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.267181 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 256293us
I20260709 12:25:28.269721 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:28.271092 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.270956 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:28.272181 26783 raft_consensus.cc:2376] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10. Re-sending same reply.
I20260709 12:25:28.236577 30268 raft_consensus.cc:2764] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:28.236362 29907 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.244488 30330 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.244639 30355 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.274407 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:28.277032 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.283142 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:28.283974 26790 raft_consensus.cc:2376] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8. Re-sending same reply.
W20260709 12:25:28.293167 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 205863us
W20260709 12:25:28.294502 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.294929 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173321us
I20260709 12:25:28.262120 30288 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:25:28.317720 26124 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:28.266945 30262 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:25:28.327749 26049 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:28.322424 29511 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:28.342437 30526 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 12:25:28.345422 30538 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:25:28.345765 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.346853 26127 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:28.347371 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2085) took 1292 ms (client timeout 1585 ms). Trace:
I20260709 12:25:28.350032 30398 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:28.350649 30398 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.351440 30398 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.356868 30533 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:28.357926 30539 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:28.358512 30539 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:28.359373 30539 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.363641 30533 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.364174 30533 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:28.347827 26127 rpcz_store.cc:269] 0709 12:25:27.055330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.345349 (+1290019us) service_pool.cc:224] Handling call
0709 12:25:28.347298 (+  1949us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:28.375559 26019 leader_election.cc:336] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.742s (SENT)
I20260709 12:25:28.298516 30434 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:28.375392 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 198203us
I20260709 12:25:28.287787 30355 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.379912 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.383785 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:28.385679 26094 leader_election.cc:400] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12.
I20260709 12:25:28.385979 26094 leader_election.cc:403] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.388592 30533 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.397653 26200 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:28.396958 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 99644.3us
I20260709 12:25:28.403581 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:28.383356 30398 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.405839 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:28.396209 30456 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:28.412212 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.408111 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.400947 30541 raft_consensus.cc:2764] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:25:28.419580 30276 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:28.407002 30540 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12.
I20260709 12:25:28.406518 30404 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:28.447171 30434 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.448349 30330 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:28.449977 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08001dd4e0 after lost signal to thread 26042
W20260709 12:25:28.450874 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f7c0 after lost signal to thread 26073
W20260709 12:25:28.451287 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f940 after lost signal to thread 26078
W20260709 12:25:28.451591 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f920 after lost signal to thread 26079
I20260709 12:25:28.452881 26123 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:28.429230 30495 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:28.451884 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080016f960 after lost signal to thread 26088
W20260709 12:25:28.462021 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08002fbd40 after lost signal to thread 26131
W20260709 12:25:28.462708 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08003edb00 after lost signal to thread 26153
W20260709 12:25:28.463052 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8540 after lost signal to thread 26154
I20260709 12:25:28.463384 26094 leader_election.cc:400] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11.
W20260709 12:25:28.463639 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3b20 after lost signal to thread 26188
W20260709 12:25:28.464143 26224 debug-util.cc:398] Leaking SignalData structure 0x7b0800291d60 after lost signal to thread 26202
W20260709 12:25:28.464763 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080035b9c0 after lost signal to thread 26208
I20260709 12:25:28.464674 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:28.465373 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08003371c0 after lost signal to thread 26227
W20260709 12:25:28.465807 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08002935c0 after lost signal to thread 26228
I20260709 12:25:28.423030 30529 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 12:25:28.466827 26224 debug-util.cc:398] Leaking SignalData structure 0x7b0800313860 after lost signal to thread 26779
W20260709 12:25:28.469169 26224 debug-util.cc:398] Leaking SignalData structure 0x7b0800372620 after lost signal to thread 26815
W20260709 12:25:28.469640 26224 debug-util.cc:398] Leaking SignalData structure 0x7b08003f87a0 after lost signal to thread 29907
W20260709 12:25:28.470644 26224 debug-util.cc:398] Leaking SignalData structure 0x7b080031aba0 after lost signal to thread 30417
W20260709 12:25:28.471002 26224 debug-util.cc:398] Leaking SignalData structure 0x7b0800318140 after lost signal to thread 30418
I20260709 12:25:28.451855 30264 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:25:28.402112 30533 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:25:28.462349 30527 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:28.467722 30410 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:25:28.453168 30397 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:25:28.490978 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.496345 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 495578us
I20260709 12:25:28.501616 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:28.504979 30360 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:28.505929 30360 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.500603 30495 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.510283 30529 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:28.511086 30529 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:28.511511 30529 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.511549 30495 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.512934 30264 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:28.513350 30264 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.514235 30264 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.468171 30275 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.515472 30264 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:28.516963 30275 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.517700 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:28.518199 30545 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:28.530423 30530 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:28.531050 30075 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:28.531286 30541 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:28.531441 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 235453us
I20260709 12:25:28.531849 30541 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.532225 30541 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.532473 30434 raft_consensus.cc:1064] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
W20260709 12:25:28.533036 30434 raft_consensus.cc:1068] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:28.533176 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:28.533638 30434 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:28.535156 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:28.535578 30434 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:28.536509 30066 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:28.484087 30527 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.538726 30434 consensus_queue.cc:1243] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:28.539297 30369 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:28.538914 30527 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.539709 30369 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:28.540906 30552 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:28.541816 30528 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.547s	user 0.004s	sys 0.000s
I20260709 12:25:28.541100 30369 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.542933 30369 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.538080 30066 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.550310 30066 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.551337 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.486270 30431 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:28.552502 30431 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:28.552449 30075 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.553081 30431 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.554118 30075 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.536825 30404 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:28.537117 30558 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.537421 30453 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:28.562597 30453 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:28.531093 30530 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.538049 30355 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.562979 30531 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:28.563097 30453 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:28.557463 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.559968 30075 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 12:25:28.557798 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.562366 30558 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:28.566776 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.567608 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.522373 30275 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.557049 30527 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.493117 30243 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:28.572113 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.566299 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:28.539577 30538 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.576617 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:28.581310 26788 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.564961 30530 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.577219 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.569490 30531 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.578701 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.588788 30530 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:28.588831 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.589263 30530 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.589684 30530 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:28.591723 26788 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:28.577265 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.594564 26123 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:28.573215 30434 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:28.575254 30398 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:25:28.574699 30558 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.602660 30434 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:28.593004 30243 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.585829 30275 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.608307 30525 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:28.571619 30529 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.618052 30529 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.624267 30559 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.624418 30434 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.624893 30559 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.625702 30434 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.626130 30431 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.627389 30559 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.613240 30525 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.628226 30431 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:28.628074 30558 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.616676 30560 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.630462 30560 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.629099 30525 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.634557 30558 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.635110 30525 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:28.636322 30541 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.637120 30560 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:28.624123 30531 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.638521 30541 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.643141 30530 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.646191 30530 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:28.682492 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:28.686163 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.687429 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.690006 26129 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:28.692812 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 173729us
W20260709 12:25:28.696580 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 607654us
I20260709 12:25:28.697336 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.698825 26126 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.699208 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.710144 30369 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:28.709486 30360 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:28.715482 30275 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:28.720480 26019 leader_election.cc:336] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.678s (SENT)
I20260709 12:25:28.721994 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.722496 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:28.723778 30075 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.723079 30243 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.739964 30563 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:28.741377 30563 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.743582 30563 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.696038 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:28.737700 30075 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:28.756315 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.756860 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:28.758052 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:28.759408 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.760979 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:28.762147 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:28.760977 26784 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:28.764516 30530 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:28.765682 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.767462 26200 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:28.769374 30075 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.780658 26782 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:28.783372 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.783077 30530 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.784482 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:28.785181 30355 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.785779 26199 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:28.786000 30355 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:28.787897 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.789297 30355 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:28.790993 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 102787us
I20260709 12:25:28.792913 29837 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.793139 30536 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:28.793821 29837 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.797114 30536 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.798117 30536 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.777184 30537 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:28.800989 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105293us
I20260709 12:25:28.801724 30537 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.802240 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:28.800922 30530 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:28.805253 30529 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:28.805927 30529 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.807233 30529 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.779304 26197 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.785182 30268 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:28.809311 30536 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.815234 30360 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:28.811815 30268 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.815943 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:28.843190 26125 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:25:28.861861 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.837028 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77981.1us
I20260709 12:25:28.852998 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:28.823076 26195 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:28.874428 26049 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:28.841629 26124 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:28.849455 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:28.877791 26045 raft_consensus.cc:2368] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:28.849989 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:28.824666 30527 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.880162 26199 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:28.791780 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.823972 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:28.848199 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:28.846621 29837 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.851852 30537 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.844331 30530 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.897620 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:28.897974 30397 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:28.898741 30397 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:28.899281 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:28.816172 30360 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.883885 30527 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:28.897164 26126 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:28.896355 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.818990 30453 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.901504 30397 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.912498 26124 raft_consensus.cc:2368] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:28.917673 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.920594 30288 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.921725 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:28.922278 30549 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.924400 30360 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.925649 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:28.907840 30527 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.919108 25951 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:28.940474 26197 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:28.941136 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.924401 30561 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:28.946753 26124 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:28.959862 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:28.960101 30549 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.961081 26127 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:28.960876 30549 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.961946 30568 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.962092 26045 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:25:28.962898 26045 pending_rounds.cc:85] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 7
I20260709 12:25:28.963213 26045 pending_rounds.cc:107] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:25:28.962765 30568 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.963409 30549 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:28.964383 30561 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:28.964673 30147 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:28.965111 30147 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.965510 30147 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.965726 30561 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:28.945358 30530 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:28.963532 26045 raft_consensus.cc:2889] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:28.976595 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 152847us
I20260709 12:25:28.982803 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:28.993997 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.018584 26124 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.012393 30269 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:25:28.992537 30288 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:25:29.027243 30434 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:29.029387 30262 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:25:28.988432 30066 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.018383 30360 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.020803 30530 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:29.021767 30453 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:29.046192 30568 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:29.049404 30573 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:25:29.057287 30567 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:25:29.057394 25952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 190) took 17841 ms (client timeout 15000 ms). Trace:
I20260709 12:25:29.058475 30456 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:29.064908 30330 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:29.069072 29961 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:29.069657 29961 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:29.035722 25951 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 7 to 10, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 10 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.043586 30268 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.095912 30549 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:25:29.129446 30549 consensus_queue.cc:1243] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:29.113238 25951 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.071970 30456 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:25:29.158145 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 58702.3us
W20260709 12:25:29.157248 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 132059us
W20260709 12:25:29.176081 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63029.7us
I20260709 12:25:29.173024 26045 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:29.071221 29961 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:29.192167 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 119330us
I20260709 12:25:29.184742 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.195173 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.198071 30275 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:29.202607 30574 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 12:25:29.202876 25951 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:29.213122 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.212847 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.217836 30066 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:29.221571 30572 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.187s	user 0.002s	sys 0.000s
W20260709 12:25:29.223941 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.188406 30549 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.212678 26169 leader_election.cc:400] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:29.243141 26169 leader_election.cc:403] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:29.239179 26051 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:29.057969 25952 rpcz_store.cc:269] 0709 12:25:11.337254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.345801 (+  8547us) service_pool.cc:224] Handling call
0709 12:25:12.893580 (+1547779us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b743adf3a 00007f5b7a163e9c 00007f5b7a26204e 00007f5b787232a4 00007f5b78723232 00007f5b78723161 00007f5b787230dc 00007f5b78722392 00007f5b726e048c 00007f5b726df8c6
0709 12:25:29.057073 (+16163493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:20.343896 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:20.344572 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:20.344615 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:25:20.344633 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:20.346197 (+  1564us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:20.346278 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:20.346308 (+    30us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:25:20.347793 (+  1485us) tablet.cc:701] Row locks acquired
0709 12:25:20.347836 (+    43us) write_op.cc:260] PREPARE: finished
0709 12:25:20.348030 (+   194us) write_op.cc:270] Start()
0709 12:25:20.348171 (+   141us) write_op.cc:276] Timestamp: P: 1783599920347948 usec, L: 0
0709 12:25:20.348186 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:25:20.542415 (+194229us) log.cc:844] Serialized 7233 byte log entry
0709 12:25:22.084689 (+1542274us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b
0709 12:25:23.823247 (+1738558us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:24.284129 (+460882us) op_driver.cc:464] REPLICATION: finished
0709 12:25:24.612858 (+328729us) write_op.cc:301] APPLY: starting
0709 12:25:24.612966 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:24.614441 (+  1475us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:24.614490 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:24.821178 (+206688us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:24.821599 (+   421us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=36
0709 12:25:24.821651 (+    52us) write_op.cc:312] APPLY: finished
0709 12:25:24.822217 (+   566us) log.cc:844] Serialized 236 byte log entry
0709 12:25:24.822427 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:24.822643 (+   216us) write_op.cc:454] Released schema lock
0709 12:25:24.822718 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:24.823168 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:24.823199 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":392720,"reported_tablets":36,"spinlock_wait_cycles":2959188736,"child_traces":[["op",{"apply.queue_time_us":315505,"apply.run_cpu_time_us":26457,"apply.run_wall_time_us":210292,"cfile_cache_hit":34,"cfile_cache_hit_bytes":35581,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6220,"cfile_init":1,"lbm_read_time_us":17449,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":6,"num_ops":18,"prepare.queue_time_us":154442,"prepare.run_cpu_time_us":29662,"prepare.run_wall_time_us":2111811,"raft.queue_time_us":209956,"raft.run_cpu_time_us":53894,"raft.run_wall_time_us":326669,"replication_time_us":3928165,"spinlock_wait_cycles":29147264,"thread_start_us":2240609,"threads_started":4,"wal-append.queue_time_us":1870454,"wal-append.run_cpu_time_us":197802,"wal-append.run_wall_time_us":5834512}]]}
W20260709 12:25:29.238049 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63151.9us
I20260709 12:25:29.204885 25951 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:25:29.257462 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84675.1us
I20260709 12:25:29.257776 30147 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.259544 26053 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:29.262699 30549 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.265231 30549 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.212945 30264 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:25:29.236678 30268 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:29.280871 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43616.8us
I20260709 12:25:29.281975 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.283131 30264 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.283537 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:29.283751 30264 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:29.284857 30580 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:29.286645 30264 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:29.288931 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.289521 30538 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.290777 30538 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.267758 26120 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:25:29.292979 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55507.8us
I20260709 12:25:29.298244 26124 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:29.246759 29961 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:29.304725 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52597.1us
I20260709 12:25:29.310106 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.311054 26053 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:29.314026 29837 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.312808 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 216673us
I20260709 12:25:29.325801 30587 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:29.327826 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.314694 30453 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:29.294370 30075 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.334960 30398 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:29.336160 30538 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:29.333333 25951 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 7, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 7 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:29.338379 30269 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:29.338975 30269 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:29.340960 30147 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.342375 30531 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:29.322253 29837 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.348943 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.348078 29837 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.354440 30075 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:29.354599 25951 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 6 to 9, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:29.358167 26051 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:29.360571 25951 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:29.364037 30262 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:29.366873 26788 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:25:29.367700 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.379323 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:29.332170 30453 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:29.366025 30583 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:25:29.392812 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.394448 26053 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:29.400429 30453 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:29.401803 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 118400us
I20260709 12:25:29.402786 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.403725 30264 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.386343 25951 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 4 to 7, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 7 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.406895 26051 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:29.388224 30525 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:25:29.388451 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.389436 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.409010 26045 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:29.389739 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.413100 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.414287 26048 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:29.390156 26047 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:29.391183 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.416980 26050 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:29.418743 30262 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:29.418852 30269 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.420557 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.420265 30262 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.421609 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.422458 26051 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 8.
I20260709 12:25:29.423854 30589 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.424742 30589 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:29.428135 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54835.4us
I20260709 12:25:29.428789 30288 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:29.429505 30288 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:29.429751 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:29.430687 30288 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.431210 30275 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:29.432034 30275 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:29.433053 30275 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.407217 25951 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.450157 25951 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 4 to 7, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 7 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:29.452529 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49895.4us
I20260709 12:25:29.452713 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.453640 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.454713 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.455636 26049 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:29.456287 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:29.461501 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.462659 26053 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:29.464021 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.464895 26050 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:29.466349 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 164558us
I20260709 12:25:29.476259 26049 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
W20260709 12:25:29.479732 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78239us
W20260709 12:25:29.485078 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55731.2us
I20260709 12:25:29.467182 30269 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:29.475340 30264 raft_consensus.cc:2764] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:29.478739 30525 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.497234 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:29.498395 30568 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:29.499023 30568 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.500444 30369 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:29.501499 30568 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.501269 30369 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.501818 26051 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:29.482028 30075 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.502204 30568 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:29.503104 30075 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.491923 30589 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:29.492587 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.505519 30075 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:29.506450 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.517855 26195 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:29.511266 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.522531 26200 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:29.512006 30527 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.005s
I20260709 12:25:29.503640 30568 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:29.503891 30369 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.533315 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.533893 26198 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260709 12:25:29.536579 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80471.6us
I20260709 12:25:29.535038 26198 pending_rounds.cc:85] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 6
I20260709 12:25:29.538779 26194 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:29.539132 26198 pending_rounds.cc:107] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:25:29.539398 26198 raft_consensus.cc:2889] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.532522 30527 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.548543 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.551317 26127 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:29.552446 30264 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.552768 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:29.553608 26782 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:29.555318 30527 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.556427 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.556849 30568 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.560673 30147 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.561584 30577 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.516s	user 0.002s	sys 0.004s
I20260709 12:25:29.561408 30147 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.567202 30561 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:29.571457 26050 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:29.585079 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:29.585879 26120 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:29.588340 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.592756 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.602034 26200 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:25:29.579886 30147 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:29.604123 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.605515 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.606678 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.607512 26200 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:29.608354 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.608982 26200 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:29.589212 26789 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:29.617151 26128 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:29.626672 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.609431 26120 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:29.627606 26051 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:29.630471 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:29.632970 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.633368 26124 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:29.633647 30360 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.633888 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51944.9us
I20260709 12:25:29.634498 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.634244 30360 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.635728 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.636641 30360 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:29.636533 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.637368 26784 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEARNER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:29.638756 30530 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:29.639353 30530 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.643738 26049 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:29.644269 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.651304 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.653816 26790 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:29.561394 30453 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:29.661046 30453 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:29.624723 30355 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.663650 26197 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:29.680523 26046 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:29.682168 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.682514 26047 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 12:25:29.683071 26045 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.689255 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.690089 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.690310 30075 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:29.691125 30075 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.690330 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.698382 30147 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:29.699245 26123 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.690146 26197 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:29.701113 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.700901 30355 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.702260 26197 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:29.704398 30397 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.705005 30397 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.707437 30397 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:29.708391 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.703775 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.710060 26197 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:29.710525 30434 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.711288 30537 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:29.715879 30537 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.718407 30537 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.718567 26053 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:25:29.715204 30434 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:29.721113 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.721681 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:29.722587 26129 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 8: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:29.712128 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:29.712534 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.724279 26051 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:29.713660 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.714494 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.714655 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.713335 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.703665 30355 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.735924 26201 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:29.723493 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.738032 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:29.738776 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46629.8us
I20260709 12:25:29.739315 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.740628 26200 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:29.747002 26045 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:29.751319 26053 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.753758 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:29.754455 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.755708 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.721727 30434 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:29.757527 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.758741 26201 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:29.761010 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.761169 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.762449 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.772446 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.775915 26126 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:29.747298 30530 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.761927 26199 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:29.765129 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.784955 26197 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:29.785188 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.785374 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.746083 30268 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:29.754699 30538 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.781029 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.778440 26047 raft_consensus.cc:1217] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Deduplicated request from leader. Original: 4.7->[6.8-9.9]   Dedup: 6.8->[9.9-9.9]
I20260709 12:25:29.793064 26120 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:29.793324 26047 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 9. Current term is 10. Ops: [6.8-9.9]
I20260709 12:25:29.774632 26128 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:29.779913 26783 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:29.757607 30369 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:29.796141 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84144.5us
I20260709 12:25:29.796455 26122 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:29.798374 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:29.803944 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46625.1us
I20260709 12:25:29.809974 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:29.810858 26790 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:29.814447 26168 leader_election.cc:400] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:29.807291 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.803793 26789 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:29.802457 26129 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:29.808593 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:29.797236 30369 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.827755 26200 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:29.830665 26203 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:29.831998 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.832932 26790 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:29.834689 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.835631 26054 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:29.836796 26052 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:29.837134 26128 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:29.836838 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.839402 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.840479 26196 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:29.841730 26196 pending_rounds.cc:85] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 9
I20260709 12:25:29.841964 26196 pending_rounds.cc:107] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:29.842198 26196 raft_consensus.cc:2889] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.843660 26199 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:29.839409 26047 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:29.894258 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48577.3us
I20260709 12:25:29.907435 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:29.829964 30369 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:29.926054 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.926160 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:29.926936 26790 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:29.928910 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:29.816504 30268 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.936666 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.911217 30355 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.931241 30397 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.953596 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 108060us
I20260709 12:25:29.959790 30397 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:29.960676 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:29.963398 30397 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.964152 29837 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:29.964831 29837 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.965348 29837 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:29.967085 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 170146us
I20260709 12:25:29.969669 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:29.970652 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:29.972033 30531 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:29.973187 30601 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:25:29.958642 26053 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:29.974627 30402 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.974916 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:29.976333 30603 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:29.958865 30268 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:29.957945 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:29.979792 26054 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:29.989820 26129 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [11.11-12.12]
I20260709 12:25:29.992041 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:29.999516 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:30.000954 26199 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:30.003510 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:30.005458 26201 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:29.988340 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2867) took 1005 ms. Trace:
I20260709 12:25:30.036470 26127 rpcz_store.cc:276] 0709 12:25:28.979492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.981825 (+  2333us) service_pool.cc:224] Handling call
0709 12:25:29.983229 (+1001404us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:29.983339 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280211712}
I20260709 12:25:30.008913 26123 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:25:30.008859 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:30.008071 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:30.017827 30602 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:25:30.018914 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.019163 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:29.955470 30355 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:30.027546 26789 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:30.027598 30402 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:30.027765 30537 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:30.015375 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:30.009577 26200 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:30.033169 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:30.007004 30531 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:25:30.032769 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:30.055150 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:30.006035 30268 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:30.058477 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.058717 30268 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.062788 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:30.063722 26123 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:30.084434 26202 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:30.087905 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55902.5us
W20260709 12:25:30.096210 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 77421.6us
I20260709 12:25:30.053963 30402 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.056151 30147 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:30.094343 30568 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:30.134435 26019 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.547s (SENT)
I20260709 12:25:30.134997 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:30.136106 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 128133us
I20260709 12:25:30.137526 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:30.157140 30599 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.313s	user 0.004s	sys 0.000s
W20260709 12:25:30.172983 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118634us
I20260709 12:25:30.098934 30530 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:30.123152 30402 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:30.151780 30602 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 12:25:30.228701 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 240595us
W20260709 12:25:30.229947 30606 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 12:25:30.230820 30609 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:25:30.234026 30537 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.235304 30621 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.121735 30360 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:25:30.236809 26201 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 12:25:30.238010 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 117538us
I20260709 12:25:30.250941 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.272881 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:30.280864 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 151980us
I20260709 12:25:30.249959 30397 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.272039 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:30.236792 30624 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:30.272480 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:30.289345 30268 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:30.289669 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:30.289994 30268 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.290813 30268 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:30.292923 30075 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:30.295140 30147 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:30.295610 30147 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:30.299664 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.296066 30147 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.254396 30537 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:30.321835 26194 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:30.325765 29837 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:30.327052 30608 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 12:25:30.283680 30525 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:30.268978 30530 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:30.356418 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:30.357026 30541 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:30.357386 26169 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:30.358568 30541 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:30.359171 30243 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.324671 30537 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:30.363565 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 229155us
W20260709 12:25:30.379267 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 241836us
I20260709 12:25:30.380339 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.387454 30589 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.391101 30264 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:25:30.393045 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 155360us
I20260709 12:25:30.388554 30275 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.390049 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46696.8us
I20260709 12:25:30.390223 30398 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:30.280495 30621 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:30.307192 30397 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.399026 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:30.400704 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.411334 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.412510 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:30.350845 30360 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:30.388835 30147 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.408887 26125 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:30.378499 30531 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.417516 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.388885 30561 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:25:30.388612 30330 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:30.445593 30633 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:30.433205 30531 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.446619 30633 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.447391 30633 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:30.378055 30243 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.353670 30434 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:25:30.445742 30330 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.378336 29837 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:30.455212 30613 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:25:30.455590 30638 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:25:30.459817 26047 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.000s
I20260709 12:25:30.391638 30589 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.506382 26047 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:30.433631 30624 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.446839 30531 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.538090 26788 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:30.450104 30541 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:30.469312 30075 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:30.503603 30626 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:30.411005 30275 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.504592 30568 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:30.505442 30398 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.511152 30638 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:30.451463 30268 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 12:25:30.587098 30642 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:30.589219 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 229746us
I20260709 12:25:30.524611 30355 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:30.598258 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 204001us
I20260709 12:25:30.577363 29837 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:30.586902 30634 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:25:30.567634 30434 consensus_queue.cc:1243] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:25:30.605623 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 211836us
W20260709 12:25:30.601724 26046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.005s	sys 0.031s
W20260709 12:25:30.614136 26046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.005s	sys 0.032s
I20260709 12:25:30.537300 30538 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:25:30.511296 30402 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:30.607470 30615 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.330s	user 0.001s	sys 0.001s
I20260709 12:25:30.621227 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.620059 30330 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:30.598272 30398 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.590023 30626 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:30.606395 30642 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:30.641330 26019 leader_election.cc:336] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.975s (SENT)
I20260709 12:25:30.643234 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:30.627313 30538 consensus_queue.cc:1243] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:30.593261 30075 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.625551 30434 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.665508 30434 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:30.666188 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45056.8us
I20260709 12:25:30.667271 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:30.628916 30587 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.511730 30530 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.589226 30541 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.648339 30643 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.638872 30398 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.676498 30243 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:30.677238 30369 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.739s	user 0.003s	sys 0.009s
I20260709 12:25:30.647179 30642 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.678654 30626 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:30.679251 30587 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:30.677820 30647 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:25:30.639676 30561 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:25:30.669176 30568 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:30.684842 30288 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:30.697906 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:30.684154 30453 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:30.740293 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105585us
I20260709 12:25:30.707518 30643 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:30.705237 30629 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:30.740756 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 285063us
I20260709 12:25:30.690485 30075 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:30.687369 30636 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:30.728296 30568 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:30.730881 30288 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:30.710351 30369 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:30.744119 30527 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.739544 30453 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:30.768551 30627 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:30.825217 26123 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:30.825718 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 759) took 1232 ms. Trace:
I20260709 12:25:30.795569 30525 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.814479 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:30.822281 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:30.768632 30626 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.819679 26120 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:30.804208 30264 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.826176 26782 rpcz_store.cc:276] 0709 12:25:29.586737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.588337 (+  1600us) service_pool.cc:224] Handling call
0709 12:25:29.588708 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:29.589177 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:29.741677 (+152500us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.548251 (+806574us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:30.588210 (+ 39959us) log.cc:844] Serialized 52 byte log entry
0709 12:25:30.588392 (+   182us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:30.588428 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.588500 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.818375 (+229875us) raft_consensus.cc:1685] finished
0709 12:25:30.818403 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.818578 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":458230,"prepare.queue_time_us":26835,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":488505,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":28274,"thread_start_us":409743,"threads_started":3,"wal-append.queue_time_us":289687}
I20260709 12:25:30.809361 30621 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:30.867652 30649 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.867846 30631 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.868446 30640 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.869029 30616 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:25:30.869388 30654 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.869453 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:30.870352 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:30.871037 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:30.870805 30264 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.949527 30398 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:30.950843 30264 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:30.955442 30243 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:30.955565 30649 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.955418 30398 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:30.959525 30649 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:30.964663 30652 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.965548 30652 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.978272 30587 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:30.979892 30625 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.980767 30625 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.984926 26125 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:25:30.985468 26125 pending_rounds.cc:85] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 7
I20260709 12:25:30.985988 26125 pending_rounds.cc:107] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:25:30.986312 26125 raft_consensus.cc:2889] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.988523 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 697) took 1065 ms. Trace:
I20260709 12:25:30.989028 26126 rpcz_store.cc:276] 0709 12:25:29.922818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.931727 (+  8909us) service_pool.cc:224] Handling call
0709 12:25:29.942949 (+ 11222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.964412 (+ 21463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.964464 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.965189 (+   725us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.056599 (+ 91410us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:30.056634 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.056693 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.988218 (+931525us) raft_consensus.cc:1685] finished
0709 12:25:30.988260 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.988423 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":395319,"raft.queue_time_us":4735,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":386688,"thread_start_us":90930,"threads_started":1,"wal-append.queue_time_us":648159}
I20260709 12:25:30.871621 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:30.993263 30640 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:30.872964 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:30.873919 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:30.874392 30147 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:30.874950 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:30.875490 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:30.875713 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:31.021845 26789 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:30.876574 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:30.881215 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 111263us
W20260709 12:25:30.883646 26019 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.170s (SENT)
I20260709 12:25:30.884769 29837 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.893801 30651 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:30.896353 30628 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:30.900473 30641 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:31.037599 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 162191us
I20260709 12:25:30.818970 30075 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:31.046967 26129 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:30.925100 26121 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:30.943423 26788 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
W20260709 12:25:30.941958 30645 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.348s	user 0.002s	sys 0.002s
I20260709 12:25:31.023722 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:30.865021 30589 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:30.929867 30402 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:30.693435 30638 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.068594 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:31.023236 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:31.072407 26789 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:31.038499 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:30.890995 30643 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:30.841629 30527 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.021206 26122 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:31.045795 30649 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:31.073167 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49987.4us
W20260709 12:25:31.090700 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52274.9us
I20260709 12:25:30.843928 30627 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.108352 30639 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:31.102804 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.103612 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:31.107264 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.109663 30369 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:31.110256 30627 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.110076 30630 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.287s
I20260709 12:25:31.112044 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.133751 26047 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:31.112831 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.114015 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.089910 30568 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.135823 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:31.117991 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:30.909581 30525 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.046345 30641 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:31.055022 29837 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:31.124545 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.100318 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:31.125830 26194 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:31.129963 26202 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:25:31.108078 30633 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.005s
I20260709 12:25:31.091467 30147 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.134610 26199 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:25:31.159206 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 461356us
I20260709 12:25:31.187741 30657 raft_consensus.cc:1064] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:25:31.135179 26052 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
W20260709 12:25:31.141475 26054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.014s	sys 0.101s
W20260709 12:25:31.201491 26054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.014s	sys 0.101s
I20260709 12:25:31.124257 30538 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:31.159039 30627 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:31.159127 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:31.159622 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59449.8us
I20260709 12:25:31.159729 26045 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:31.159931 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.160358 30644 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:31.098435 30075 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:31.146798 30641 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.156785 30633 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.176357 30568 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:31.176496 26094 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.623s (SENT) [suppressed 30 similar messages]
I20260709 12:25:31.155758 30652 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.179313 26046 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:31.181334 29837 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.156641 29961 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:25:31.283931 26197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.182s	sys 0.255s
I20260709 12:25:31.071326 30625 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.135411 30638 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.993912 30654 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.156419 30589 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:25:31.207633 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:31.231834 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.138581 30661 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:31.173213 30637 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:31.231701 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 348064us
I20260709 12:25:31.136955 30621 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:25:31.257566 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.258052 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.259554 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.064657 30288 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:31.200663 30360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.007s	sys 0.001s
W20260709 12:25:31.324009 26019 leader_election.cc:336] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.590s (SENT)
I20260709 12:25:31.264189 30636 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:31.264477 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:31.264847 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74335.6us
I20260709 12:25:31.189297 30639 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.258411 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 541) took 1241 ms. Trace:
I20260709 12:25:31.268908 30275 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.272732 26047 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:31.235123 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:31.273267 30537 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.873s	user 0.013s	sys 0.000s
I20260709 12:25:31.340906 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.275061 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:31.341630 26129 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:31.276759 30640 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.214094 30643 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.264477 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.345866 26120 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:31.187410 30243 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:31.026780 30631 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:31.292482 26197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.182s	sys 0.256s
W20260709 12:25:31.330490 30360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.008s	sys 0.002s
I20260709 12:25:31.231779 30644 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.310501 26052 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:31.235870 30629 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:31.319805 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 112284us
W20260709 12:25:31.314929 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 364301us
I20260709 12:25:31.341118 26121 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:31.347486 26127 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:25:31.350718 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.315744 30661 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.270829 30657 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:31.354663 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.402921 26125 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:31.406590 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:31.362241 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 327661us
W20260709 12:25:31.366961 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.407294 26047 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:31.315296 30541 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.378257 26122 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:31.378690 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:31.382712 26123 pending_rounds.cc:85] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
I20260709 12:25:31.330354 26054 rpcz_store.cc:276] 0709 12:25:29.991144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.084158 (+ 93014us) service_pool.cc:224] Handling call
0709 12:25:30.133914 (+ 49756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.150934 (+ 17020us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:30.255637 (+104703us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.232638 (+977001us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:31.232701 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:31.232764 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.232835 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.232959 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13219,"prepare.queue_time_us":87897,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":30843,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":55897,"spinlock_wait_cycles":153472,"thread_start_us":869355,"threads_started":1}
I20260709 12:25:31.378540 30434 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:31.424283 26123 pending_rounds.cc:107] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:25:31.424751 26123 raft_consensus.cc:2889] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.376947 30644 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.342263 30369 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:25:31.370054 30360 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:31.425953 26045 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.388917 30665 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:31.403724 30269 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:31.444414 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:31.440200 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 253384us
I20260709 12:25:31.343256 30636 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.451952 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:31.438622 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.439100 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115134us
W20260709 12:25:31.453598 26019 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.671s (SENT)
I20260709 12:25:31.453981 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:31.384443 30661 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.351533 30537 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.427842 30621 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:31.476343 30243 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.380766 30629 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.464792 26054 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 12:25:31.468152 30525 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.464398 30527 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.469322 30625 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:31.471766 30631 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:31.473134 30618 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.520s	user 0.004s	sys 0.000s
I20260709 12:25:31.474520 30583 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:31.475235 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:31.490748 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:31.401809 30453 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.427475 30541 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:31.478255 26200 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:31.497228 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43659.9us
I20260709 12:25:31.448570 30269 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.440583 30665 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:31.488026 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:31.525465 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:31.527654 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:31.486541 30640 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:31.531584 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.440810 30589 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:25:31.549033 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73878.8us
I20260709 12:25:31.549626 26052 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:31.551405 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 374899us
I20260709 12:25:31.495899 30583 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:31.578065 30583 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.497354 30268 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:31.582314 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.507419 30269 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.601264 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.602145 26049 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:31.590126 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62565.5us
I20260709 12:25:31.532454 30453 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.529480 30627 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:31.549258 30530 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:31.487627 30621 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.494592 30625 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.487358 30243 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.615567 26045 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
W20260709 12:25:31.621708 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.626252 30627 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:31.627364 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.629280 30649 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.635257 26129 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
W20260709 12:25:31.636530 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 306559us
I20260709 12:25:31.678767 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:31.638468 30636 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.638626 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:31.639248 26202 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:31.639757 30537 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.640638 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:31.683835 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:31.684077 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.685017 26128 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:31.683801 26167 connection.cc:256] RPC call timeout handler was delayed by -1.2579162490001181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
I20260709 12:25:31.641531 30629 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:31.647815 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.680729 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:31.616968 30676 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:25:31.657299 30665 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:31.659816 30663 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:25:31.660660 30654 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.505913 30541 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.697366 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:31.703605 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:31.704576 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:31.710588 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:31.711535 26194 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.662009 30639 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:31.606568 30633 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:31.672425 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:31.643853 30621 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.690673 30678 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.680445 26199 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:31.682512 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.587216 30268 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:31.625336 30671 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 12:25:31.598925 30590 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:31.694748 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 313030us
I20260709 12:25:31.699412 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2031) took 2074 ms. Trace:
W20260709 12:25:31.700873 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.702420 29837 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.709862 30658 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:31.719614 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:31.726717 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 228650us
I20260709 12:25:31.716871 26052 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:31.707103 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:31.776149 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 97483.3us
I20260709 12:25:31.713194 30568 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:31.786028 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:31.589735 30360 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:31.790957 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 83926.8us
I20260709 12:25:31.752902 26196 rpcz_store.cc:276] 0709 12:25:29.617883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.841091 (+223208us) service_pool.cc:224] Handling call
0709 12:25:29.841459 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:29.843022 (+  1563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.843052 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:29.843733 (+   681us) log.cc:844] Serialized 52 byte log entry
0709 12:25:30.087626 (+243893us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:30.087696 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.087803 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.446219 (+1358416us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:31.669241 (+223022us) raft_consensus.cc:1685] finished
0709 12:25:31.669286 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.691785 (+ 22499us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":156400384,"thread_start_us":243435,"threads_started":1,"wal-append.queue_time_us":1108530}
I20260709 12:25:31.798635 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.800956 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:31.744742 30665 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.742796 30631 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:31.743569 30268 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.802729 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:31.715624 30525 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:25:31.752705 30590 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.807705 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:31.805598 30531 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.223s	user 0.015s	sys 0.000s
I20260709 12:25:31.814540 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:31.786589 30527 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:31.799397 26202 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:31.798111 30621 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.831053 26019 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.385s (SENT)
I20260709 12:25:31.805277 30658 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:31.835037 26203 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:31.842154 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.819092 30590 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:31.819201 30639 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.825834 30642 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.089s	user 0.012s	sys 0.003s
I20260709 12:25:31.843015 26196 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:31.844300 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:31.846516 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:31.755390 30678 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.850834 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.822814 30264 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:31.842607 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 222371us
I20260709 12:25:31.853718 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.828186 30530 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:31.869290 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:31.873440 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:31.874332 26196 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.876598 26202 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:31.881085 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.887198 30656 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.820s	user 0.005s	sys 0.000s
I20260709 12:25:31.888767 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 959) took 1466 ms. Trace:
I20260709 12:25:31.830479 30531 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:31.857725 30147 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:31.812772 29837 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:31.862949 30657 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:31.895272 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:31.870839 30655 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:31.844522 30631 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:25:31.896978 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.940969 30658 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:31.943141 30695 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:31.863498 30642 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.944487 30243 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.889142 26790 rpcz_store.cc:276] 0709 12:25:30.421914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.422112 (+   198us) service_pool.cc:224] Handling call
0709 12:25:30.452156 (+ 30044us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.586640 (+134484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.586677 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.587255 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.732396 (+145141us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:30.732437 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.732499 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.500545 (+768046us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:31.888444 (+387899us) raft_consensus.cc:1685] finished
0709 12:25:31.888467 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.888705 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102462,"raft.queue_time_us":135193,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":104992640,"thread_start_us":144702,"threads_started":1,"wal-append.queue_time_us":257728}
I20260709 12:25:31.945211 29837 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:31.946234 30642 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:31.947852 30684 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:25:31.868283 30530 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.951686 26203 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:31.929478 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:31.871048 30264 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.833386 30621 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:31.894124 30633 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:31.937090 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55211.6us
I20260709 12:25:31.984166 26194 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:31.950958 30626 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.107s	user 0.013s	sys 0.002s
I20260709 12:25:31.998674 26199 raft_consensus.cc:2368] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13.
I20260709 12:25:31.876526 30639 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:32.000097 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.996070 30655 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:32.002243 30655 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:32.004113 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108929us
I20260709 12:25:31.953487 30530 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:31.885555 30681 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 12:25:31.963973 30264 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.985831 30633 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:32.012617 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 190914us
I20260709 12:25:32.014497 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.017155 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:32.015321 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:32.028702 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 342340us
W20260709 12:25:32.029312 26167 connection.cc:256] RPC call timeout handler was delayed by -1.0882601499997691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
I20260709 12:25:32.036115 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.040367 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.005985 30627 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:32.009104 30657 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:25:32.062199 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.062719 30541 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.023207 30633 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.023855 30666 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:32.056010 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.008842 30631 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.059149 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.078510 26785 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.073253 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.033874 30625 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.076078 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:32.071123 26019 leader_election.cc:336] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.441s (SENT)
I20260709 12:25:32.079421 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:32.071170 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:32.079448 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:32.076205 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.054354 30637 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.062248 30657 consensus_queue.cc:1243] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:32.081082 26054 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:32.083473 26051 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:32.079214 26782 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:32.089661 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.090116 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:32.092533 26047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.013s	sys 0.017s
W20260709 12:25:32.098555 26047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.014s	sys 0.017s
I20260709 12:25:32.101742 26053 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:32.103099 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.103385 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:32.097371 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61401.4us
I20260709 12:25:32.105237 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:32.111723 26198 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:32.113919 26789 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:32.120890 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:32.126940 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75283.1us
I20260709 12:25:32.096859 30625 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:32.101402 29837 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:32.084065 30631 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:32.138195 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 335519us
I20260709 12:25:32.073077 30627 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:32.141557 26019 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.120s (SENT)
I20260709 12:25:32.142388 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:32.113536 30642 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:32.119969 30641 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.842s	user 0.007s	sys 0.007s
I20260709 12:25:32.094863 30637 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.159076 30623 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:32.121134 30360 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.130584 30643 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.000s
I20260709 12:25:32.184571 26785 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:32.186556 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:32.190749 26048 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:25:32.182410 30693 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:25:32.195696 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.175625 30642 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:32.182984 30661 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.000s
I20260709 12:25:32.131538 30678 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:32.196571 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91922.3us
I20260709 12:25:32.200866 29837 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:32.201846 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60334.7us
I20260709 12:25:32.134886 30625 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.214505 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.192355 30636 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:32.217991 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:32.239310 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:32.278192 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60290us
I20260709 12:25:32.278405 26051 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:32.265389 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:32.215085 30531 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.228425 30075 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:25:32.239899 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.285104 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:32.241251 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2147) took 1014 ms. Trace:
I20260709 12:25:32.177168 30641 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.241707 30643 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.245285 30637 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:32.245630 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.247272 30288 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.253965 30631 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:32.257881 30644 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.800s	user 0.009s	sys 0.002s
W20260709 12:25:32.218649 30638 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.900s	user 0.011s	sys 0.002s
I20260709 12:25:32.213882 30642 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:32.359905 30453 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.735s	user 0.005s	sys 0.005s
I20260709 12:25:32.224901 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.307785 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:32.274299 26196 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
W20260709 12:25:32.372572 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64840us
W20260709 12:25:32.233784 30692 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:25:32.233197 30699 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:25:32.238327 30696 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:25:32.283263 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:32.221853 30621 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:32.232518 30707 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:25:32.234467 30703 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:25:32.238889 30698 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:25:32.307370 26123 rpcz_store.cc:276] 0709 12:25:31.228644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.348970 (+120326us) service_pool.cc:224] Handling call
0709 12:25:31.382434 (+ 33464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.447212 (+ 64778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.447252 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.451163 (+  3911us) log.cc:844] Serialized 52 byte log entry
0709 12:25:31.688594 (+237431us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:31.688638 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.688721 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.241044 (+552323us) raft_consensus.cc:1685] finished
0709 12:25:32.241064 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.241194 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44548,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5522,"thread_start_us":211992,"threads_started":1,"wal-append.queue_time_us":536601}
I20260709 12:25:32.344305 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.391670 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.352468 30709 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:32.353011 30690 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.259s	user 0.002s	sys 0.000s
I20260709 12:25:32.357551 30627 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:32.357759 30583 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.779s	user 0.008s	sys 0.003s
I20260709 12:25:32.313794 30636 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:32.358796 30644 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.333655 30643 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.220064 30360 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.239066 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 695) took 2253 ms. Trace:
W20260709 12:25:32.365967 30269 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.692s	user 0.008s	sys 0.004s
I20260709 12:25:32.287516 30658 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.237552 30689 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:32.218300 30661 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.301196 30628 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:32.239039 30714 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:32.326417 30654 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:32.426959 30654 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.427371 30654 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:32.379115 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 154284us
I20260709 12:25:32.428709 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 966) took 1138 ms. Trace:
I20260709 12:25:32.429322 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.331439 30666 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:32.390834 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:32.393227 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.337845 30705 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:32.385746 30268 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.003s
I20260709 12:25:32.394006 26051 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:25:32.401463 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.403000 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.382091 30665 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:32.411450 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.387475 30531 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:32.414633 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.415489 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.414656 26200 rpcz_store.cc:276] 0709 12:25:29.985532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.094191 (+108659us) service_pool.cc:224] Handling call
0709 12:25:30.134802 (+ 40611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.500809 (+1366007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.238767 (+737958us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400398d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:32.238978 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3088983936}
W20260709 12:25:32.407730 30590 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:25:32.395735 30640 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.447126 30640 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.448400 26045 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:32.449483 30644 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.449560 30640 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.389881 30712 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:32.397749 30709 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.425383 26786 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:32.451951 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.425361 30716 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:32.426041 30702 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
I20260709 12:25:32.426529 30661 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.431425 26198 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:32.384496 30631 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:25:32.434283 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.434190 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 713) took 1346 ms. Trace:
I20260709 12:25:32.439389 26785 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.458244 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.413439 30633 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:32.439726 26019 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 3.108s (SENT)
I20260709 12:25:32.462741 26053 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:32.443190 30717 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:32.463433 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:32.445392 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.444725 30636 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.440852 30268 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.446175 30590 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.446689 30689 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:32.449857 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.420156 30269 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.396802 30530 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.457821 26126 rpcz_store.cc:276] 0709 12:25:31.088222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.228943 (+140721us) service_pool.cc:224] Handling call
0709 12:25:31.274075 (+ 45132us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.452392 (+178317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.452444 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.465782 (+ 13338us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.755716 (+289934us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:31.755759 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.755823 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.433911 (+678088us) raft_consensus.cc:1685] finished
0709 12:25:32.433941 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.434110 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53369,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":31979,"thread_start_us":402268,"threads_started":2,"wal-append.queue_time_us":346271}
I20260709 12:25:32.429245 26788 rpcz_store.cc:276] 0709 12:25:31.290522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.297363 (+  6841us) service_pool.cc:224] Handling call
0709 12:25:31.297796 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.341646 (+ 43850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.341691 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.384970 (+ 43279us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.690884 (+305914us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:31.690942 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.232693 (+541751us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b5400639f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:32.232765 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.428323 (+195558us) raft_consensus.cc:1685] finished
0709 12:25:32.428351 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.428625 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60577,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7974,"spinlock_wait_cycles":99831168,"thread_start_us":265005,"threads_started":1,"wal-append.queue_time_us":481678}
I20260709 12:25:32.438213 30665 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.414885 30583 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.478770 26791 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:32.422394 30658 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.466427 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1623) took 1305 ms. Trace:
I20260709 12:25:32.450417 30712 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:32.465975 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:32.453677 30709 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:32.488405 30666 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.438287 30714 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:32.492554 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52860.1us
I20260709 12:25:32.493031 30709 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.494261 30714 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.494707 26120 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:25:32.494917 30688 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:32.495530 30688 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.474321 26787 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:32.474423 26050 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:32.430526 30628 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.481141 26047 rpcz_store.cc:276] 0709 12:25:31.161029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.422062 (+261033us) service_pool.cc:224] Handling call
0709 12:25:31.449089 (+ 27027us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.477016 (+ 27927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.477050 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.477717 (+   667us) log.cc:844] Serialized 52 byte log entry
0709 12:25:32.122721 (+645004us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:32.315528 (+192807us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.321661 (+  6133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.321722 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.466153 (+144431us) raft_consensus.cc:1685] finished
0709 12:25:32.466180 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.466354 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20801,"prepare.queue_time_us":105546,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":25981,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":50635,"thread_start_us":657416,"threads_started":2,"wal-append.queue_time_us":617967}
W20260709 12:25:32.475289 30686 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.360s	user 0.000s	sys 0.003s
I20260709 12:25:32.570340 26202 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:32.572847 26785 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:32.506280 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.496798 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.534837 26195 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:32.471782 30716 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:32.477876 30725 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.155s
I20260709 12:25:32.574254 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.474695 30537 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:32.566038 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.582198 26053 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:32.477895 30665 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.496909 30709 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.499008 30717 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:32.587654 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.499202 30712 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.591316 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:32.567152 30637 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:32.593354 30653 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.958s	user 0.008s	sys 0.006s
I20260709 12:25:32.577811 30625 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.601666 26203 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:32.497931 30688 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.596001 26047 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:32.577270 30531 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.577692 30397 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.578704 30716 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.590571 26046 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:32.604779 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.624754 26201 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:32.582057 30661 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:32.616222 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.636534 26051 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:32.639926 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.640617 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.600659 30664 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:32.614006 30643 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:32.657332 30723 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:25:32.601876 30637 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.626668 30716 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:32.663398 26201 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:32.689286 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:32.690466 26201 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 12:25:32.667021 30728 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:25:32.622310 30531 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:32.671103 30264 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
I20260709 12:25:32.673723 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.677709 30527 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:32.678826 26053 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:32.679769 26049 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 5
I20260709 12:25:32.661582 30397 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:32.683482 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 699) took 2663 ms. Trace:
I20260709 12:25:32.685143 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.644733 30664 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.694418 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:32.695523 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.691772 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:32.706256 26049 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:25:32.700855 30631 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.708185 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.708469 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.708935 26789 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:32.767242 26051 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.749400 30643 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:32.770216 30643 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:32.758044 30737 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:32.752592 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1624) took 1106 ms. Trace:
I20260709 12:25:32.773826 26194 rpcz_store.cc:276] 0709 12:25:31.648159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.996632 (+348473us) service_pool.cc:224] Handling call
0709 12:25:31.997008 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.140585 (+143577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.140617 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.144654 (+  4037us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.409479 (+264825us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:32.409521 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.409600 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.752330 (+342730us) raft_consensus.cc:1685] finished
0709 12:25:32.752356 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.752522 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21588,"raft.queue_time_us":172245,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9227,"thread_start_us":284832,"threads_started":2,"wal-append.queue_time_us":271130}
W20260709 12:25:32.776584 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.736881 30590 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.700695 30583 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.707669 30527 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.740136 30664 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:32.738094 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:32.750181 30732 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:25:32.737138 30397 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.736922 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:32.745908 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.790956 26783 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:25:32.758479 30717 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.763998 26049 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.703328 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:32.764107 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72461.5us
I20260709 12:25:32.695374 30360 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.763839 30639 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.708953 30264 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.777384 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:32.779906 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.708318 26197 rpcz_store.cc:276] 0709 12:25:30.020345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.234932 (+214587us) service_pool.cc:224] Handling call
0709 12:25:30.235344 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.235690 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.235716 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.236191 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.352257 (+1116066us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:31.352323 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.352424 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.428459 (+1076035us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b240020ba78. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:25:32.682878 (+254419us) raft_consensus.cc:1685] finished
0709 12:25:32.682907 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.683406 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":537,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":13008512,"thread_start_us":1102243,"threads_started":1,"wal-append.queue_time_us":1234373}
W20260709 12:25:32.786811 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1619) took 1638 ms (client timeout 2108 ms). Trace:
I20260709 12:25:32.787956 26045 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:32.749503 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:32.781992 30728 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:32.642235 30661 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.783670 30587 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:32.794716 26194 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.765105 30268 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:32.794978 26048 rpcz_store.cc:269] 0709 12:25:31.144062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.144614 (+   552us) service_pool.cc:224] Handling call
0709 12:25:32.781759 (+1637145us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:32.781950 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2210066176}
I20260709 12:25:32.790120 30269 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:32.785712 30633 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:32.818424 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:32.819099 26127 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.809592 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.819850 26201 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:32.811838 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.801950 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:32.811692 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.822484 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:32.791512 29961 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.810353 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:32.824584 26053 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:32.824409 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:32.826010 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.828996 26123 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:25:32.798951 30730 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
I20260709 12:25:32.832464 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.833850 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.796645 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:32.831430 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.835505 26053 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:32.805208 30637 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:32.786188 30631 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.847193 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.841252 26201 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:32.813848 30655 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.845402 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.798918 30583 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:32.807904 30398 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:32.806965 30643 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.854784 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.863581 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.813678 30587 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:32.864428 26787 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:32.864481 30587 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:32.865309 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.858326 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.865341 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:32.866544 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.858896 26051 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:32.866576 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:32.868409 26201 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.868716 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:32.829592 30639 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:32.827620 30689 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.838804 30709 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:32.883891 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:32.884624 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.891712 26197 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:32.839946 30568 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:32.885232 26201 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:32.832608 30360 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:32.899693 26194 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:32.845285 30725 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:32.905217 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.893293 26122 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.906565 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:32.899036 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:32.909152 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:32.860157 30642 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.004s
W20260709 12:25:32.889061 30733 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:25:32.903820 30631 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:32.911830 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:32.912905 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.913465 26201 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:32.915496 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.915904 30642 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.916484 30639 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:32.917660 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:32.918504 30642 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.918850 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:32.920279 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:32.920854 26201 raft_consensus.cc:2376] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10. Re-sending same reply.
I20260709 12:25:32.921729 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.922897 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.924104 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.925088 26051 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:32.833719 30668 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:25:32.920040 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:32.832643 29961 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.926466 26045 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:32.935976 26121 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:32.936970 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:32.881345 30397 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.883754 30633 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:32.878760 30637 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 12:25:32.943272 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 124454us
I20260709 12:25:32.906616 30360 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:32.954681 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:32.955513 26051 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.950706 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:32.875108 30655 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.925240 30725 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:32.879366 30398 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:32.931169 30666 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:32.948504 30734 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 12:25:32.927695 30530 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.964454 30658 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.968720 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:32.974946 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1059) took 2105 ms (client timeout 1529 ms). Trace:
I20260709 12:25:32.977422 30531 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:32.978883 30636 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:32.884421 30718 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
W20260709 12:25:33.034111 26783 rpcz_store.cc:269] 0709 12:25:30.869275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.876281 (+  7006us) service_pool.cc:224] Handling call
0709 12:25:32.974697 (+2098416us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:32.974850 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4065949952}
I20260709 12:25:32.980772 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:33.038295 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.989423 26122 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:32.878846 30583 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.980158 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1625) took 1331 ms. Trace:
I20260709 12:25:33.018890 30587 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:32.959033 30633 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:33.042143 30633 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.043691 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:32.960260 30360 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:33.020726 30740 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:25:33.052657 30628 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:33.055013 30654 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.627s	user 0.002s	sys 0.007s
I20260709 12:25:33.056943 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 779) took 1038 ms. Trace:
I20260709 12:25:33.057422 26790 rpcz_store.cc:276] 0709 12:25:32.018326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.043402 (+1025076us) service_pool.cc:224] Handling call
0709 12:25:33.056853 (+ 13451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.039381 26203 rpcz_store.cc:276] 0709 12:25:31.648605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.998826 (+350221us) service_pool.cc:224] Handling call
0709 12:25:32.013661 (+ 14835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.612696 (+599035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.979820 (+367124us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54002189d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:32.980098 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347737984}
I20260709 12:25:33.021422 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.038441 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:33.038769 26048 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.065274 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.071447 26051 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:33.036501 30666 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:33.039136 30665 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.087971 26127 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:33.075433 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.089829 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:33.092057 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:33.074791 30741 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:25:33.103233 30716 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.104981 30640 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:33.108969 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.059165 30636 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.109514 30398 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.099454 30264 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.112078 30640 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:33.113155 30666 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:33.116219 26199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1630) took 1466 ms (client timeout 1623 ms). Trace:
I20260709 12:25:33.116591 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:33.116677 26199 rpcz_store.cc:269] 0709 12:25:31.650257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.755802 (+105545us) service_pool.cc:224] Handling call
0709 12:25:33.115990 (+1360188us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:33.116133 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443585792}
I20260709 12:25:33.117177 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.117929 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.118108 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.118248 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.118822 26199 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:33.119015 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.119287 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.119737 26197 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:33.119587 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.121094 26199 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:33.076308 30658 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.100070 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.124696 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:33.127560 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 708) took 1006 ms. Trace:
I20260709 12:25:33.128358 26199 rpcz_store.cc:276] 0709 12:25:32.121045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.119373 (+998328us) service_pool.cc:224] Handling call
0709 12:25:33.127474 (+  8101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.129328 26198 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:33.129895 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.130594 26198 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:33.131110 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.132817 26200 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:33.134086 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:33.138072 26197 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:25:33.138698 26197 pending_rounds.cc:85] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 9
I20260709 12:25:33.139032 26197 pending_rounds.cc:107] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:25:33.139356 26197 raft_consensus.cc:2889] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.055483 30654 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.132603 30527 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.140930 30264 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.141309 30640 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.144600 26782 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:33.145370 30654 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.149878 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:33.159241 30525 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:33.159654 30525 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:33.152863 26787 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:33.160957 30641 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.161079 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.150738 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1628) took 1501 ms. Trace:
I20260709 12:25:33.162544 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.162732 26197 rpcz_store.cc:276] 0709 12:25:31.649598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.122418 (+1472820us) service_pool.cc:224] Handling call
0709 12:25:33.123221 (+   803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.150383 (+ 27162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.150664 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.152169 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:33.153644 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1629) took 1503 ms. Trace:
I20260709 12:25:33.142509 30636 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.141423 30658 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.166865 26203 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:33.167455 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1633) took 1516 ms. Trace:
I20260709 12:25:33.167754 26203 rpcz_store.cc:276] 0709 12:25:31.651250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.166253 (+1515003us) service_pool.cc:224] Handling call
0709 12:25:33.166563 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.167198 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.167380 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.143782 30716 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:33.142288 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1631) took 1482 ms. Trace:
I20260709 12:25:33.169680 26198 rpcz_store.cc:276] 0709 12:25:31.650554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.129955 (+1479401us) service_pool.cc:224] Handling call
0709 12:25:33.130239 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.131273 (+  1034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.132449 (+  1176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.170233 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:33.173120 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:33.145623 30665 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.159250 26790 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.177541 30655 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:33.179641 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1621) took 1539 ms (client timeout 1874 ms). Trace:
I20260709 12:25:33.180485 30652 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:33.181386 30652 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:33.181771 30652 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:33.168223 30658 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.182596 30268 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:33.180078 26195 rpcz_store.cc:269] 0709 12:25:31.639948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.640383 (+   435us) service_pool.cc:224] Handling call
0709 12:25:33.179456 (+1539073us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:33.179581 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575863680}
I20260709 12:25:33.182607 30652 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.186791 30652 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:33.186862 29961 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.150473 30527 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.188794 30587 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.190033 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 775) took 1174 ms. Trace:
I20260709 12:25:33.152424 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:33.192847 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:33.193097 26199 rpcz_store.cc:276] 0709 12:25:31.649904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.129014 (+1479110us) service_pool.cc:224] Handling call
0709 12:25:33.152904 (+ 23890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.153154 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.153193 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.153288 (+    95us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:33.153333 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.153422 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.153563 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.165930 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.198665 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.200678 26122 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:33.191581 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.203662 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.204406 26120 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:33.190433 26786 rpcz_store.cc:276] 0709 12:25:32.015440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.016270 (+   830us) service_pool.cc:224] Handling call
0709 12:25:33.189794 (+1173524us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54001162d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:33.189947 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882799360}
I20260709 12:25:33.203629 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.222721 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:33.239599 26196 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:33.240423 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 716) took 1034 ms. Trace:
I20260709 12:25:33.240736 26196 rpcz_store.cc:276] 0709 12:25:32.206329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.238958 (+1032629us) service_pool.cc:224] Handling call
0709 12:25:33.239362 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.240164 (+   802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.240369 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.243048 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.247625 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:33.248426 26051 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:33.252312 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 99857.2us
I20260709 12:25:33.248862 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:33.269692 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.273528 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.297361 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:33.298537 26785 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:33.300486 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:33.311369 26120 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 9.
I20260709 12:25:33.242097 30643 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.324422 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 719) took 1100 ms. Trace:
I20260709 12:25:33.324860 26199 rpcz_store.cc:276] 0709 12:25:32.224495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.247346 (+1022851us) service_pool.cc:224] Handling call
0709 12:25:33.323887 (+ 76541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.324066 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.324092 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.324145 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:33.324177 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.324253 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.324367 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:33.330319 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1641) took 1647 ms. Trace:
I20260709 12:25:33.336561 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:33.336763 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:33.330771 26202 rpcz_store.cc:276] 0709 12:25:31.682546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.881831 (+199285us) service_pool.cc:224] Handling call
0709 12:25:33.330069 (+1448238us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400301e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:33.330254 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409032064}
I20260709 12:25:33.349298 26790 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:33.336736 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:33.318498 30718 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:33.326264 30720 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:33.363335 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:33.366554 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.370523 26120 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:25:33.366976 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 990979us
I20260709 12:25:33.371181 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.378314 26202 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:33.386200 30243 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:33.388475 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.300889 30398 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:25:33.391388 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.317559 30360 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.390590 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.423324 26120 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:33.370267 30676 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:33.427043 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.441453 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.441498 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 558) took 1323 ms. Trace:
I20260709 12:25:33.296306 30525 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.445500 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.446204 26120 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:33.437316 26202 raft_consensus.cc:2376] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10. Re-sending same reply.
I20260709 12:25:33.451416 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.452459 26120 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:33.452342 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.443814 26050 rpcz_store.cc:276] 0709 12:25:32.117974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.118166 (+   192us) service_pool.cc:224] Handling call
0709 12:25:32.136544 (+ 18378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.564566 (+428022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.441172 (+876606us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b5400173254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:33.441423 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776449408}
W20260709 12:25:33.426669 30664 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.002s
I20260709 12:25:33.382746 30654 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:33.390281 30643 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.392736 30530 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.464280 26197 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:33.392472 30531 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:33.469633 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:33.470062 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.430029 30676 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:33.461443 30728 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:25:33.482187 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.486131 26120 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:33.486442 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.450428 30717 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:33.469789 30742 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.559s	user 0.000s	sys 0.004s
I20260709 12:25:33.464457 30654 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.493253 26125 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:33.496975 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.504637 26202 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
W20260709 12:25:33.497184 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105303us
I20260709 12:25:33.373966 30718 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:33.502419 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.475683 30525 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:33.508194 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65350.3us
I20260709 12:25:33.499995 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:33.477211 30633 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:33.485118 30676 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.517614 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:33.527266 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.511762 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:33.530083 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.532258 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.526592 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:33.468787 30666 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:25:33.538650 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.470388 30664 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.506778 30725 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.508528 30654 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.507321 30398 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.543942 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:33.480940 30531 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:33.524207 30712 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.907s	user 0.006s	sys 0.006s
I20260709 12:25:33.487946 30643 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.477734 30530 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.522704 30665 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:33.546835 30659 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:33.541209 30625 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:33.563414 26194 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:33.578015 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.584717 26125 raft_consensus.cc:2410] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:33.584529 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.588529 30648 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:33.596378 30264 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.597920 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.598608 26202 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 8.
I20260709 12:25:33.599602 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:33.600533 30749 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 12:25:33.600838 30700 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:33.601962 30700 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:33.602516 30700 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:33.603971 30659 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:33.605216 30659 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.607110 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 712) took 1462 ms. Trace:
I20260709 12:25:33.557416 30398 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.607692 30360 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.566123 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:25:33.633703 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 67699.5us
I20260709 12:25:33.582811 30625 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:33.579885 30243 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:33.609784 30718 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.639041 30700 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.614892 30268 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:33.579885 30269 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:33.616617 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 717) took 1399 ms. Trace:
I20260709 12:25:33.663793 26196 rpcz_store.cc:276] 0709 12:25:32.217282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.241632 (+1024350us) service_pool.cc:224] Handling call
0709 12:25:33.246942 (+  5310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.289021 (+ 42079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.289060 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.317124 (+ 28064us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.498954 (+181830us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:33.499012 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.499127 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.616343 (+117216us) raft_consensus.cc:1685] finished
0709 12:25:33.616374 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.616544 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":40851,"thread_start_us":124530,"threads_started":1,"wal-append.queue_time_us":213252}
I20260709 12:25:33.576248 30527 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:33.576714 30763 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:33.618882 30658 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:33.620188 30531 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:33.636970 30643 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.649106 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.686210 26789 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:33.686502 26202 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:33.650676 30530 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.607551 26198 rpcz_store.cc:276] 0709 12:25:32.144418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.170497 (+1026079us) service_pool.cc:224] Handling call
0709 12:25:33.170814 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.171235 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:33.171260 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.171690 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.397187 (+225497us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:33.397246 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.397332 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.606869 (+209537us) raft_consensus.cc:1685] finished
0709 12:25:33.606890 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.607045 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5224,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":69,"thread_start_us":225042,"threads_started":1,"wal-append.queue_time_us":339518}
I20260709 12:25:33.580544 30665 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:33.665874 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 713) took 1504 ms. Trace:
I20260709 12:25:33.639304 30625 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:33.562479 30712 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.643743 30642 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:33.696308 30642 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:33.697682 30642 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.680919 26125 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:33.706974 30712 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:33.661679 30243 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.666987 30527 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.683305 26757 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:33.685303 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1074) took 1270 ms. Trace:
I20260709 12:25:33.688153 26195 rpcz_store.cc:276] 0709 12:25:32.161075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.183912 (+1022837us) service_pool.cc:224] Handling call
0709 12:25:33.184257 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.184469 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:33.381353 (+196884us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.482937 (+101584us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.662821 (+179884us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.662896 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:33.662959 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.663052 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.663232 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1010,"prepare.queue_time_us":476934,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":24584,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":10002,"thread_start_us":462027,"threads_started":3,"wal-append.queue_time_us":223565}
W20260709 12:25:33.712539 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128111us
W20260709 12:25:33.713500 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 228857us
I20260709 12:25:33.709890 26791 rpcz_store.cc:276] 0709 12:25:32.415365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.445077 (+ 29712us) service_pool.cc:224] Handling call
0709 12:25:33.685020 (+1239943us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b5400660154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:33.685228 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31893376}
I20260709 12:25:33.713711 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.718292 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.718209 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.720927 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.721724 26196 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.723327 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.737732 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.686625 30763 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:33.746268 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.747853 26122 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:33.749176 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.752493 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:33.690232 30665 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.754241 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.780777 26123 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:33.769233 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48426.7us
W20260709 12:25:33.774647 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 212662us
I20260709 12:25:33.667354 30268 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.775427 26195 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:33.749234 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.777570 26120 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:33.778159 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:33.711467 30700 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.782539 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.791397 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:33.825667 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76549.5us
I20260709 12:25:33.826087 26052 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:33.801903 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.826987 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.793500 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:33.719403 30527 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.719676 30288 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:33.829710 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47242.5us
I20260709 12:25:33.682916 30269 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.831377 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.835752 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.837080 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:33.842487 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:33.844867 26048 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.848479 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:33.848879 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.851626 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:33.723953 30649 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.692940 30658 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.778699 30648 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:25:33.842265 30288 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.842422 30652 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.835260 30643 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:33.858544 30587 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:33.882390 30288 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.883019 30664 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.860075 30589 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.861480 30147 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.869930 30712 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:33.920658 26202 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:33.943149 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:33.944216 26202 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 9: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:33.928491 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 711) took 1790 ms. Trace:
I20260709 12:25:33.946573 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:33.933595 26196 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:33.951256 26049 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:33.945207 26203 rpcz_store.cc:276] 0709 12:25:32.137947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.168534 (+1030587us) service_pool.cc:224] Handling call
0709 12:25:33.168885 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.169230 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.169267 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.169743 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.427504 (+257761us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:33.927266 (+499762us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.927957 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.928101 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.928165 (+    64us) raft_consensus.cc:1685] finished
0709 12:25:33.928184 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.928408 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":176066,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":14448,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":73,"thread_start_us":727213,"threads_started":2,"wal-append.queue_time_us":440929}
I20260709 12:25:33.889844 26048 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:25:33.960391 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:33.904404 30640 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.006s
W20260709 12:25:33.964480 30764 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:25:33.967469 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.975553 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:33.894888 30587 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:33.982690 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:33.985199 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 153942us
I20260709 12:25:33.986351 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.987954 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:33.985479 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.892577 30643 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:33.900501 30658 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:33.898950 30243 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:33.993374 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:33.906863 30288 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.972373 30640 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.942595 30712 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:33.992614 30587 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:33.983815 30637 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:33.997783 30643 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.994138 30147 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:34.021679 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 239603us
I20260709 12:25:33.998435 30766 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:34.003648 30530 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.039237 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2039) took 1458 ms. Trace:
I20260709 12:25:34.019618 26052 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:34.033495 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.005905 30658 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:34.044175 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.047264 26203 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:34.041481 26199 rpcz_store.cc:276] 0709 12:25:32.580514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.345756 (+765242us) service_pool.cc:224] Handling call
0709 12:25:33.375408 (+ 29652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.375937 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.376005 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.376939 (+   934us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.545974 (+169035us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:33.749144 (+203170us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.749417 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.749514 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.038922 (+289408us) raft_consensus.cc:1685] finished
0709 12:25:34.038956 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.039157 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57388,"prepare.queue_time_us":57862,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":7000,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":25111,"thread_start_us":263321,"threads_started":2,"wal-append.queue_time_us":258610}
I20260709 12:25:33.957773 30763 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:33.998382 30676 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.019841 30625 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.020866 30718 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:25:34.008841 30531 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:25:34.047096 30778 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.040859 30640 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.082489 30640 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.067281 30768 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:34.049839 30268 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.037251 30661 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:34.073941 30288 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.074477 26787 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:25:34.044198 30636 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:34.061931 30654 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.000s
I20260709 12:25:34.111459 26125 raft_consensus.cc:606] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.113746 26125 raft_consensus.cc:614] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:34.118470 30712 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:34.137826 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.140014 30288 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.140105 30568 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:34.116616 30269 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:34.150163 26784 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:34.152014 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.121929 30527 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.152447 30147 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.153407 30775 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.117062 30654 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.154230 30527 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.155614 30654 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.115348 30665 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.156554 30625 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.157171 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1627) took 2507 ms. Trace:
I20260709 12:25:34.153090 30147 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.157555 26200 rpcz_store.cc:276] 0709 12:25:31.649285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.120157 (+1470872us) service_pool.cc:224] Handling call
0709 12:25:33.131848 (+ 11691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.133197 (+  1349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.156988 (+1023791us) spinlock_profiling.cc:243] Waited 7.03 ms on lock 0x7b54002b5f50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:34.157114 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16179456}
I20260709 12:25:34.158193 30665 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.158949 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 972) took 1457 ms. Trace:
I20260709 12:25:34.159152 30147 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.159345 26782 rpcz_store.cc:276] 0709 12:25:32.701279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.701415 (+   136us) service_pool.cc:224] Handling call
0709 12:25:34.158774 (+1457359us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400715a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:34.158894 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811318656}
I20260709 12:25:34.144295 30243 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:25:34.162647 30766 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.163324 30269 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:34.165977 30269 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:34.156466 26124 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:34.142650 30568 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:34.171052 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.173959 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:34.183135 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.187953 26790 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:34.188522 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.185700 26782 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:34.190872 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.190213 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.195178 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.131676 30587 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.198827 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:34.196516 30668 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.196849 30765 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:34.197022 30712 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.192404 30530 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:34.204432 30568 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.205240 30530 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.205739 30568 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.207715 30530 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.153920 30775 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.208848 26788 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:34.210176 30775 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.156354 30778 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:25:34.210511 30587 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.163308 30766 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.213994 30766 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.201138 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.144498 30640 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.145341 30789 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:34.224215 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 180108us
I20260709 12:25:34.196733 30360 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.226150 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.233821 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.237373 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.225565 30766 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.211591 30643 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.254779 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.240547 30712 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:34.257887 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.259617 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.260448 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.261097 26196 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.261174 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.239243 30530 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:34.267642 30676 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.242067 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.274489 26200 raft_consensus.cc:2376] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15. Re-sending same reply.
I20260709 12:25:34.256763 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.276382 26199 raft_consensus.cc:2376] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11. Re-sending same reply.
I20260709 12:25:34.238049 30775 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:34.316900 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.262725 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.318679 26203 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:34.321682 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80586.6us
I20260709 12:25:34.226521 30789 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.233717 30531 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:25:34.324303 26196 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.324409 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.324519 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.320997 30530 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.331065 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:34.317210 30775 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:34.325502 30644 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:34.337930 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.333737 30786 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.339983 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.329910 30583 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.340214 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.345232 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.348618 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.356168 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.356592 26054 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:34.357365 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.357775 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.331168 30531 consensus_queue.cc:1243] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:25:34.360662 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.336324 30775 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:34.345295 30783 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:25:34.336097 30644 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.334744 30777 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.377446 30538 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.345360 30768 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:25:34.378156 30797 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:34.379645 30778 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:34.393024 30778 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:34.393433 30778 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.394404 30778 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.395723 30778 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:34.373948 30642 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:34.399705 30747 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.006s	user 0.007s	sys 0.000s
W20260709 12:25:34.402556 30799 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:34.335433 30530 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.369150 30268 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:34.404147 30268 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.404870 30268 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.406200 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.406543 30530 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:34.408318 30531 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:34.383405 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.410447 30531 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:34.410847 30531 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:34.384757 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.336912 30780 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:34.387717 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:34.359853 30537 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.412523 26203 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:34.413249 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:34.358777 30786 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:34.416953 30786 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:34.417274 30743 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:34.417868 30780 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:34.419777 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2046) took 1770 ms (client timeout 2033 ms). Trace:
I20260709 12:25:34.413564 30541 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:34.421336 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.339949 30665 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:25:34.414472 26199 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:25:34.415131 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:34.340049 30625 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:25:34.413681 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:34.462483 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50369.2us
I20260709 12:25:34.402630 30766 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.424041 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:34.433418 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45774.7us
I20260709 12:25:34.470801 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:34.471344 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.421988 26200 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.436329 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.421067 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 973) took 1711 ms. Trace:
I20260709 12:25:34.422921 26053 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.472390 26121 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.475060 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:34.420167 26201 rpcz_store.cc:269] 0709 12:25:32.649719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.922621 (+272902us) service_pool.cc:224] Handling call
0709 12:25:34.419575 (+1496954us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400398354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:34.419716 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258565888}
I20260709 12:25:34.463703 26124 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.471737 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.415989 30658 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.477169 26126 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.476140 26120 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:34.478277 26196 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:34.478366 26120 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:34.479506 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.472162 26789 rpcz_store.cc:276] 0709 12:25:32.709633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.769329 (+ 59696us) service_pool.cc:224] Handling call
0709 12:25:33.685965 (+916636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.688723 (+  2758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.420899 (+732176us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b5400660150. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:34.421013 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107003136}
I20260709 12:25:34.479710 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:34.480211 26120 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.421895 30777 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:34.486425 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2894) took 1679 ms. Trace:
I20260709 12:25:34.485299 26045 raft_consensus.cc:1217] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.8->[12.9-12.10]   Dedup: 12.9->[12.10-12.10]
I20260709 12:25:34.420291 30643 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.421849 30541 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.489846 30525 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:34.491827 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.492435 30525 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.491976 30541 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.492951 26201 raft_consensus.cc:2376] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12. Re-sending same reply.
I20260709 12:25:34.493399 30525 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.494271 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.490717 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.420215 30780 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.469494 30642 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.500931 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.483717 30538 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:25:34.483964 30264 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:34.482577 30147 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.481316 30768 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:34.486845 26128 rpcz_store.cc:276] 0709 12:25:32.807008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.807139 (+   131us) service_pool.cc:224] Handling call
0709 12:25:32.807525 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.807870 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.807897 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.808352 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.074723 (+266371us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:33.074768 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.074836 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.399055 (+1324219us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:34.481790 (+ 82735us) raft_consensus.cc:1685] finished
0709 12:25:34.481811 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.486362 (+  4551us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6259,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":10621,"spinlock_wait_cycles":357583616,"thread_start_us":227092,"threads_started":1,"wal-append.queue_time_us":491863}
I20260709 12:25:34.517746 26126 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:34.508795 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.490111 30777 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.519819 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.500603 30642 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.411711 30531 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.526952 30712 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:34.527422 30712 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:34.489974 30797 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:25:34.534643 26053 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:34.535869 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.536792 26049 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.524758 30537 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.525087 30642 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.543886 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.556021 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.556408 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.555914 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.544641 26050 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:34.546286 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.558452 26048 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:34.559566 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.562453 26054 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:34.578018 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.581323 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.583016 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.583751 26048 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:34.545992 30789 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.542694 30398 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.556279 30531 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.588001 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:34.588693 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.588903 26201 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:34.589468 26054 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.538087 30786 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.533159 30712 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.590827 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.594987 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.595563 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.598825 26201 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:34.604887 26123 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:34.605589 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.612224 30268 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.612905 30786 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.613291 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.614404 26051 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:34.614764 30268 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.607924 30712 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.637331 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.638535 26049 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:34.638618 26051 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:34.646165 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.665238 26128 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:25:34.652515 30801 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:25:34.676513 26051 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:34.663735 30688 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:34.695814 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.697216 26051 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:34.684587 30268 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.704411 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:34.706957 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.709717 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:34.701872 30803 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:25:34.712760 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.717375 26046 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:34.715900 26051 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.684929 30688 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.724778 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:34.728603 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.733816 30768 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.735026 30768 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.738593 30583 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:34.739171 30583 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:34.740116 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 204321us
W20260709 12:25:34.740705 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2043) took 2151 ms (client timeout 1883 ms). Trace:
W20260709 12:25:34.741114 26194 rpcz_store.cc:269] 0709 12:25:32.589697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.908342 (+318645us) service_pool.cc:224] Handling call
0709 12:25:34.740510 (+1832168us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b5400335254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:34.740638 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392951168}
I20260709 12:25:34.742268 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2865) took 2094 ms. Trace:
I20260709 12:25:34.742635 26047 rpcz_store.cc:276] 0709 12:25:32.648071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.648311 (+   240us) service_pool.cc:224] Handling call
0709 12:25:32.707558 (+ 59247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.709548 (+  1990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.709576 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.743763 (+ 34187us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.844797 (+101034us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:32.880278 (+ 35481us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.897519 (+ 17241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.897638 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.322945 (+1425307us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b180021ade0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:34.418729 (+ 95784us) raft_consensus.cc:1685] finished
0709 12:25:34.418835 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.742111 (+323276us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5400172850. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:34.742214 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27795,"prepare.queue_time_us":28045,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":24701,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":24960,"spinlock_wait_cycles":41645696,"thread_start_us":48685,"threads_started":1,"wal-append.queue_time_us":146264}
I20260709 12:25:34.743904 30628 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:34.744122 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:34.744735 30628 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:34.745518 30797 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:34.745746 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:34.746028 30628 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:34.742657 30770 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:25:34.746542 30628 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.747053 30797 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:34.749406 26787 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:34.754282 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.759011 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:34.707538 30268 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.764602 26200 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [11.10-15.13]
I20260709 12:25:34.766396 30264 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:34.778926 30264 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.766852 30805 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:34.766993 30791 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:34.767468 30786 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:34.772490 30797 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.764858 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:34.747632 30628 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.777738 30583 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:34.782425 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.790373 30665 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:34.790819 30665 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:34.791234 30665 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.780916 30805 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:34.792218 30792 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.577s	user 0.000s	sys 0.006s
I20260709 12:25:34.792558 30805 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:34.792389 26093 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:34.783676 30804 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:25:34.794610 30770 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.797569 30688 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:34.797410 30797 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:34.800825 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.801522 26122 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
W20260709 12:25:34.804698 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.786552 30786 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:34.806303 30786 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:34.807168 30786 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.808386 30763 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.808971 30763 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.811168 30763 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.812582 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.812055 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.786029 30655 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:25:34.815634 30770 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:34.816450 30790 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.606s	user 0.003s	sys 0.002s
I20260709 12:25:34.818044 30628 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:34.819072 30744 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:34.819507 30744 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:34.820362 30744 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.821887 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 714) took 2643 ms. Trace:
I20260709 12:25:34.822273 26197 rpcz_store.cc:276] 0709 12:25:32.178113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.194698 (+1016585us) service_pool.cc:224] Handling call
0709 12:25:33.195024 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.470828 (+275804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.821675 (+1350847us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:34.821833 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602014592}
I20260709 12:25:34.822760 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.823405 26124 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:25:34.824785 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.826632 30766 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:34.827178 30766 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.827755 30075 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:34.829385 30766 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:34.830492 30369 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:34.834997 30268 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:34.835515 26045 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:34.779917 30264 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.836406 26046 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:34.836629 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.837294 26045 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:34.837452 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.839116 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.841213 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.842705 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.838258 26046 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.845538 26048 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:34.847745 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.842653 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:34.853348 26787 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:34.854347 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.856227 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:34.857621 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.858114 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.859750 29961 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:34.861011 30791 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:34.860180 29961 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:34.863273 29961 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.808374 30786 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.864440 30659 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:34.864917 30659 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:34.866019 30659 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.867427 30659 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.789831 30775 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.869202 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:34.870363 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.875952 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.877089 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.877806 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.878230 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.878552 26787 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.879663 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.880264 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.872294 26054 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:34.881037 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:34.873487 26788 raft_consensus.cc:2368] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:34.884332 26791 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:34.886085 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.887367 26121 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:34.892867 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.890439 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.899362 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:34.900064 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:34.894603 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.910092 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:34.910089 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:34.869239 30537 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.874795 30268 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.898382 30778 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:34.885578 30625 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.879607 30789 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.893011 30655 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.893963 30587 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:34.892674 30640 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:25:34.869458 30637 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:25:34.898382 30643 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:34.933431 26049 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:34.929795 26128 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:34.906728 30541 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:34.944119 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:34.945556 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.946844 26045 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:34.885375 30770 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:25:34.957384 26048 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:34.967334 30818 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:34.840186 30269 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:34.972383 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:34.973475 26047 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:34.974216 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:34.975454 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76171.2us
I20260709 12:25:34.979249 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:34.926669 30778 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.982967 26046 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:34.917936 30625 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:34.933995 30587 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:34.990712 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100365us
I20260709 12:25:34.992336 26128 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:34.993927 26051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49f4a1a801de461182f8315a884b9889"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:25:34.935417 30664 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:34.968039 30818 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:34.990684 30625 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.953373 30360 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:34.997067 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 140867us
I20260709 12:25:35.011945 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:35.013096 26018 leader_election.cc:400] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:35.001482 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:35.013692 26018 leader_election.cc:403] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:35.014273 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.972007 30589 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:35.017298 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.986305 30787 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.995394 30778 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.993088 30716 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:35.023059 26785 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:34.998849 30587 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:35.033932 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:35.033869 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.931557 30644 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.020584 30269 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.042321 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:34.995712 30777 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.054931 29961 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.026430 30268 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:35.065030 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:35.018421 30789 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:25:35.007865 30664 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.018945 30643 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:25:35.091310 29961 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:35.093094 30818 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:35.097074 30807 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:35.107658 29961 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.108333 29961 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.110383 29961 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:35.019948 30360 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.112536 26123 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:35.113703 30777 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.113857 30360 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.064534 30637 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.043529 30822 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:35.025468 30589 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:35.119168 30642 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.001s
W20260709 12:25:35.120338 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.121166 30589 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.088967 30644 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:35.144858 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79922us
W20260709 12:25:35.147024 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79092.8us
I20260709 12:25:35.064828 30716 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:35.119632 30642 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:35.151247 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68114.4us
I20260709 12:25:35.160907 26051 raft_consensus.cc:606] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:35.161427 26051 raft_consensus.cc:614] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:35.168567 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:35.159494 26124 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 3 to 10, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:25:35.178612 30750 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.684s	user 0.000s	sys 0.006s
I20260709 12:25:35.178828 30625 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.180734 30147 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:35.181262 30147 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:35.182248 30147 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.183624 30147 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.189597 30775 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.151377 30718 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:35.145247 30815 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:35.193361 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:35.203100 26050 raft_consensus.cc:2368] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13.
I20260709 12:25:35.153789 30268 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:35.235698 26203 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:35.170775 30538 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.023216 30765 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 3 to 10, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:35.265086 30664 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.266929 30527 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:35.267535 30527 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:35.268616 30527 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.270332 30527 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:35.241626 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:35.270956 30531 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.000s
I20260709 12:25:35.271211 26198 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:35.271826 26202 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:25:35.242555 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:35.241307 30636 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:25:35.244721 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.244903 26047 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:25:35.245745 30811 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:35.246322 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.245882 26048 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 3 to 10, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:35.247949 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:35.259836 30744 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.261042 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.282873 26196 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:35.231107 30716 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:35.328418 26201 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:35.333029 26052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:25:35.333916 26052 raft_consensus.cc:606] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.334421 26052 raft_consensus.cc:614] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:35.335091 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.336951 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51439.4us
W20260709 12:25:35.287392 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.273939 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.272184 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:35.283218 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 249412us
I20260709 12:25:35.322710 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.261309 26046 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:35.324611 26791 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:25:35.280637 30812 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:25:35.273447 30636 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:35.337436 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 358288us
I20260709 12:25:35.361495 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:35.362767 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.342250 26203 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:35.318849 30268 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.365156 26201 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:35.355074 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:35.344058 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 327337us
I20260709 12:25:35.355859 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:35.348039 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.351778 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:35.340595 26202 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:25:35.357724 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2065) took 2021 ms. Trace:
I20260709 12:25:35.369671 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:35.358929 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73406.3us
I20260709 12:25:35.370608 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:35.280171 30642 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.328016 30716 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.191603 30822 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.159s
W20260709 12:25:35.343144 30805 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.550s	user 0.007s	sys 0.005s
I20260709 12:25:35.336165 30718 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:35.279959 30665 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.337085 30780 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 12:25:35.376987 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.475039 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:35.441892 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54518.7us
I20260709 12:25:35.378085 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:35.438946 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:35.476526 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.502684 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.510560 26197 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.377712 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:35.514326 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.499339 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:35.368268 26195 rpcz_store.cc:276] 0709 12:25:33.336031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.973863 (+637832us) service_pool.cc:224] Handling call
0709 12:25:33.994093 (+ 20230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.995680 (+  1587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.995727 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.831057 (+835330us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:25:34.831482 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.230834 (+399352us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b540031d920. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b7431c6b8 00007f5b743a592b 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c
0709 12:25:35.230917 (+    83us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:35.265911 (+ 34994us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.357295 (+ 91384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.357392 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.357447 (+    55us) raft_consensus.cc:1685] finished
0709 12:25:35.357464 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.357653 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92160,"prepare.queue_time_us":44492,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":46,"raft.queue_time_us":93966,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":5522,"spinlock_wait_cycles":17529600,"thread_start_us":75273,"threads_started":1,"wal-append.queue_time_us":72344}
I20260709 12:25:35.442473 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:35.506178 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.506811 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.517904 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.519830 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.338076 30817 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:35.520726 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.342996 30537 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:35.521413 26200 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:35.524070 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.524978 26200 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.526723 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:35.527102 26201 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
W20260709 12:25:35.358568 30806 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:35.478138 30369 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.540220 26048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
W20260709 12:25:35.535815 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.543871 26048 raft_consensus.cc:606] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.544277 26048 raft_consensus.cc:614] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:35.379822 30770 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.522619 30716 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:35.358968 30664 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:35.355916 30766 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:35.496342 30360 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:25:35.554117 26197 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
W20260709 12:25:35.595577 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66319.4us
I20260709 12:25:35.652216 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.652944 26197 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:35.519850 30625 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.439591 30777 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:35.459836 30655 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:25:35.360999 30636 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:35.594880 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.439263 30268 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.524775 30642 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.654350 26197 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:35.513257 30789 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:25:35.531404 30817 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.542834 30587 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.694212 26200 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:35.696161 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.366659 30530 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.217s
I20260709 12:25:35.536625 30780 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:35.543380 30786 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.001s
W20260709 12:25:35.698582 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 255627us
I20260709 12:25:35.695246 30766 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.650485 30644 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.505234 30805 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.706118 30777 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:35.649396 30767 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.538157 30830 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:35.537611 30568 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 12:25:35.535004 30628 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.164s
I20260709 12:25:35.687965 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:35.751317 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63412.3us
I20260709 12:25:35.752192 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.544898 30537 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:35.722775 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.725545 26195 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:35.697451 26125 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:35.704487 30838 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:35.705876 30531 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:35.754837 30531 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.756000 30659 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.756541 30659 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.757215 30531 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.757741 30531 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:35.758191 30531 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.758706 30659 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.754923 30838 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:35.730501 26047 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:35.703511 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 176850us
I20260709 12:25:35.760280 30838 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:35.732949 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.603643 30369 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:35.761591 30531 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.762630 30838 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:35.698019 30664 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.764174 26203 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:35.760924 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.766230 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.744540 30777 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:35.767757 26121 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:25:35.747377 30640 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:35.702497 30718 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.769485 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.763327 30369 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:35.743026 30642 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.769464 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:35.751883 30834 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:35.743628 30644 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.769933 30786 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:35.743224 30835 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:25:35.774523 30818 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.773275 30640 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:35.806180 26094 leader_election.cc:400] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13.
I20260709 12:25:35.796420 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1674) took 1296 ms. Trace:
I20260709 12:25:35.824371 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.825201 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:35.825728 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.825776 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:35.826615 26046 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:35.826990 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:35.789214 30369 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.822407 26199 rpcz_store.cc:276] 0709 12:25:34.499959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.725851 (+225892us) service_pool.cc:224] Handling call
0709 12:25:34.726187 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.726414 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:34.985273 (+258859us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.796045 (+810772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.796098 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:35.796145 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.796218 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.796361 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119432,"prepare.queue_time_us":311069,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":118248,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7080,"thread_start_us":962970,"threads_started":3,"wal-append.queue_time_us":555181}
I20260709 12:25:35.786507 30831 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:35.834232 26197 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:35.835920 26051 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:35.835920 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:35.836184 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:35.837728 30829 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:25:35.838025 30369 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.837925 30655 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:35.831110 30640 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.841391 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.842765 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:35.804875 30820 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:35.752485 30636 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.819209 30845 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.866271 26045 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:35.866184 26128 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:35.869534 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:35.837973 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.792865 30625 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:25:35.830055 30665 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.909765 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:35.910243 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:35.910815 26045 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:35.910286 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.848811 30789 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:35.914149 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:35.914781 26129 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:35.914904 30851 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.916172 30854 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:35.916819 30854 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:35.898213 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128851us
I20260709 12:25:35.795192 30642 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.750006 30830 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:35.864571 30831 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:35.908280 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:35.908389 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:35.913473 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:35.928345 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.933844 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:35.904911 30775 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:35.940860 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.941747 26129 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:25:35.897110 30655 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.943385 26202 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.958437 30537 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.962226 30369 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:35.928346 30625 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.933400 30831 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:35.970260 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56284.5us
I20260709 12:25:35.936909 30846 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.941486 30636 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.940301 30791 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:35.940354 30587 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:35.964747 30775 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:35.965047 30766 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:35.973512 30147 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.788s	user 0.013s	sys 0.000s
W20260709 12:25:36.151537 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.094022 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:36.301468 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.272858 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:36.017621 26195 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:35.966444 30789 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:36.294217 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:36.103283 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.314446 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.315099 26785 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:35.973536 30805 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:35.977553 30836 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:25:36.074997 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.138808 26199 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:36.049176 26201 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:35.973290 30817 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.082589 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2180) took 1205 ms. Trace:
I20260709 12:25:35.982250 26202 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:36.316272 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:35.984298 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.083693 30398 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:36.296905 30770 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.093596 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:36.013900 30844 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:25:36.121167 30269 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.074s	user 0.005s	sys 0.009s
W20260709 12:25:35.989979 30841 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:25:36.064065 30538 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.802s	user 0.004s	sys 0.006s
W20260709 12:25:36.016290 30810 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.835s	user 0.003s	sys 0.003s
I20260709 12:25:36.329041 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:36.261646 30744 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.924s	user 0.012s	sys 0.000s
W20260709 12:25:36.013198 30839 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:25:36.333817 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.274063 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 299658us
I20260709 12:25:36.345773 30767 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:25:36.196765 30527 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.926s	user 0.007s	sys 0.006s
I20260709 12:25:36.073578 30655 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.993350 30847 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.197731 30643 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:36.317632 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.233392 30712 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.258819 30845 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:36.278517 30800 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.302089 30815 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:35.987157 30268 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:35.991750 30642 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.429015 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2084) took 1189 ms. Trace:
I20260709 12:25:36.429493 26198 rpcz_store.cc:276] 0709 12:25:35.240039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.340849 (+100810us) service_pool.cc:224] Handling call
0709 12:25:35.341250 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.368510 (+ 27260us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:35.368546 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.438433 (+ 69887us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.724446 (+286013us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:35.724483 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.724544 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.428846 (+704302us) raft_consensus.cc:1685] finished
0709 12:25:36.428865 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.428967 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26756,"raft.queue_time_us":126915,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":25559,"thread_start_us":255804,"threads_started":1,"wal-append.queue_time_us":360345}
I20260709 12:25:36.330839 26121 rpcz_store.cc:276] 0709 12:25:34.877422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.897285 (+ 19863us) service_pool.cc:224] Handling call
0709 12:25:34.897691 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.775353 (+877662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.082206 (+306853us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54001d15d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:36.082501 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967342336}
I20260709 12:25:36.028627 30659 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.207813 30848 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.196461 30857 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.185628 30856 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.087764 30644 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.036017 30846 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:36.440310 30856 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.440888 26052 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:36.249606 30664 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:36.413189 30637 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:36.090775 30681 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.441737 30681 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.441623 30637 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.442811 30856 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.413218 30643 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.443830 30681 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.260788 30854 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.295416 30658 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:36.444859 30658 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:36.445643 30643 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.445767 30658 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.422736 30712 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.446921 30658 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:36.448367 30712 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.448710 30637 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.295410 30775 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.036176 30855 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:36.449577 30855 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.418774 30718 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.451103 30718 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.451848 30855 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.322031 30851 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:36.453104 30851 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.453464 30718 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:36.361807 26053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.004s	sys 0.007s
I20260709 12:25:36.334307 30850 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.044467 30716 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.089924 30777 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.329478 30766 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:36.282505 30862 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:36.078066 30765 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.004s
I20260709 12:25:36.344024 30398 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:36.184207 30640 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.371302 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:36.497412 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 126171us
I20260709 12:25:36.367202 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:36.495759 30862 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:36.499862 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 132855us
I20260709 12:25:36.255939 30831 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.380604 26129 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:36.354722 30770 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.382593 30847 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.363363 30538 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:36.407729 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.510511 26122 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:36.408066 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.378815 30786 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:36.373992 30767 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.377715 30866 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:36.410308 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.413597 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:36.413990 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 225196us
I20260709 12:25:36.420073 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:36.519039 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105543us
I20260709 12:25:36.420428 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.421042 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.421288 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.426039 30264 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:36.426424 26782 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 3 to 10, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:36.427631 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.417073 30802 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:36.427887 30800 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.434898 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.435941 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1675) took 1904 ms. Trace:
I20260709 12:25:36.429592 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:36.431761 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.439280 30857 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.440588 30846 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.010280 30818 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:36.074445 30830 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:36.288728 30791 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:36.455837 26053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.005s	sys 0.007s
I20260709 12:25:36.597998 26194 rpcz_store.cc:276] 0709 12:25:34.531907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.741879 (+209972us) service_pool.cc:224] Handling call
0709 12:25:34.742196 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.742534 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.742561 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.742988 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.985725 (+242737us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:35.528294 (+542569us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.731053 (+202759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.731103 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.015690 (+284587us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:36.424112 (+408422us) raft_consensus.cc:1685] finished
0709 12:25:36.435733 (+ 11621us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.435881 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14475,"prepare.queue_time_us":108424,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":241178,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":10475,"spinlock_wait_cycles":25038720,"thread_start_us":796714,"threads_started":3,"wal-append.queue_time_us":438097}
I20260709 12:25:36.460131 30848 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.422243 30838 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:36.424634 30815 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:36.485699 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 755) took 1678 ms. Trace:
I20260709 12:25:36.494711 30766 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.500424 30805 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:36.373291 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2929) took 1305 ms. Trace:
I20260709 12:25:36.495139 30851 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.501420 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:36.506382 26790 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:36.424364 30859 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 12:25:36.616817 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 115478us
I20260709 12:25:36.507678 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.617918 26129 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:36.486123 30718 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.511029 26123 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:36.432317 30861 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:25:36.515496 26128 raft_consensus.cc:606] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:36.459726 30659 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.524968 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:36.632041 26197 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:36.633062 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.526273 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:36.633801 26129 raft_consensus.cc:2368] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17.
I20260709 12:25:36.633767 26201 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:36.634707 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.635038 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:36.635453 26129 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:36.526901 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:36.635900 26198 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:36.520416 26787 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
W20260709 12:25:36.577347 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57695.8us
I20260709 12:25:36.612670 26120 rpcz_store.cc:276] 0709 12:25:35.067743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.067871 (+   128us) service_pool.cc:224] Handling call
0709 12:25:35.068299 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.068739 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.068772 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.069400 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.358487 (+289087us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b18003eab20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:35.750936 (+392449us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.750975 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.751037 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.367336 (+616299us) raft_consensus.cc:1685] finished
0709 12:25:36.367360 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.373221 (+  5861us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21065,"raft.queue_time_us":10359,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6222,"spinlock_wait_cycles":45249408,"thread_start_us":392261,"threads_started":1,"wal-append.queue_time_us":781769}
I20260709 12:25:36.508401 30744 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:36.575513 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.469609 30850 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.457059 26054 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:36.530263 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:36.601495 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:36.644160 26048 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:36.644611 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.608434 26126 rpcz_store.cc:276] 0709 12:25:34.807068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.808462 (+  1394us) service_pool.cc:224] Handling call
0709 12:25:34.808798 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.815869 (+  7071us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:34.815898 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.257565 (+441667us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:25:35.257999 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.633658 (+375659us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.633699 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.633775 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.480935 (+847160us) raft_consensus.cc:1685] finished
0709 12:25:36.480967 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.485617 (+  4650us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6655,"raft.queue_time_us":7134,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":81758464,"thread_start_us":375199,"threads_started":1,"wal-append.queue_time_us":450020}
W20260709 12:25:36.605386 30877 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:25:36.610141 30766 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:36.659945 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:36.662289 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.604385 30681 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:36.624118 30718 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.669723 26047 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:36.607292 30838 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.629281 26128 raft_consensus.cc:614] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:36.579329 30527 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:36.515651 30767 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:36.630544 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:36.525449 26202 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:36.588023 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 158502us
I20260709 12:25:36.695757 26046 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:36.696278 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:36.518700 30862 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.637290 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:36.642627 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.623479 30868 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:36.646189 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.835546 26120 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:36.647267 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.837330 26129 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:36.648307 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.520210 30866 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.652073 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:36.652681 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.600672 30864 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:36.613511 30831 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.518131 30786 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.612538 30805 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:36.592763 30802 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.694522 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.881592 26122 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:36.617341 30870 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:25:36.616273 30818 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.899127 30818 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:36.889976 26123 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:36.832897 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:36.899916 26047 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:36.901094 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:36.901765 26047 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:36.934540 30802 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:36.834111 30527 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:36.878454 30841 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 12:25:36.838089 26121 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:36.670166 30854 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:36.658995 30744 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:36.853636 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.990053 26120 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:36.991530 26120 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:36.607447 30791 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:36.866163 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 214203us
W20260709 12:25:36.994103 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.994706 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:36.527987 30855 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.170s
I20260709 12:25:36.882681 30805 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:36.889587 30716 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:36.877975 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181796us
I20260709 12:25:36.699529 30637 raft_consensus.cc:2764] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:36.620425 30398 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:36.834064 30851 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:36.834647 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 197394us
I20260709 12:25:36.852147 30868 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.944340 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:36.958323 26127 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:36.868412 30665 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:36.862017 30797 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.962643 26787 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 8
I20260709 12:25:36.876708 30643 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.988932 30862 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:36.995541 26202 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:36.869266 30864 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:36.881757 30780 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:36.870947 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:36.874085 26126 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:36.608259 30815 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.019586 30859 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.173625 30744 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.040400 30659 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.106858 26787 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.9
I20260709 12:25:37.040731 30640 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.183920 26787 raft_consensus.cc:2917] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Aborting config change with OpId 11.9: Aborted: Op aborted by new leader
I20260709 12:25:37.187597 26787 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.136837 30868 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:36.695209 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73086.6us
I20260709 12:25:36.986704 26121 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:36.987126 30854 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.138345 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:37.196542 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 201622us
I20260709 12:25:37.140604 26197 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:37.141114 30830 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.040943 30770 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.198040 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:36.996399 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:36.984886 30531 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.146415 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.201099 26199 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:37.147082 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.203671 26203 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:37.001201 30777 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.159037 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.204921 26785 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:37.018118 30840 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.147400 30643 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.175443 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:37.151360 30780 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:37.109948 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:37.135339 30848 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.135330 30866 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:37.195192 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.169549 30841 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:25:37.135097 30861 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.199872 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:37.237892 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.153079 30665 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:37.209884 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 190920us
I20260709 12:25:37.018427 30075 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:37.213177 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.001s	sys 0.017s
I20260709 12:25:37.223342 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:37.251397 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.001s	sys 0.017s
I20260709 12:25:37.221275 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:37.251662 26195 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:37.230463 26202 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:37.174525 30859 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.223989 30644 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.207340 30854 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:37.216825 30531 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.208494 30640 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.149950 30797 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.248188 30842 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.250739 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.228336 30780 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:37.224442 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:25:37.145984 30851 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.249111 30861 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.257093 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.235502 30846 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.218972 30840 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.234292 30847 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:37.253854 30264 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.674s	user 0.003s	sys 0.007s
W20260709 12:25:37.274495 26782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.000s	sys 0.027s
I20260709 12:25:37.186210 30527 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.224109 30800 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:37.280122 26045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.006s	sys 0.022s
I20260709 12:25:37.249850 30866 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:37.278877 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.937s (SENT) [suppressed 27 similar messages]
I20260709 12:25:37.289119 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2908) took 1195 ms. Trace:
W20260709 12:25:37.333742 26045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.006s	sys 0.022s
I20260709 12:25:37.269004 30625 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:37.255578 30818 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:25:37.270820 30857 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:37.317852 26782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.000s	sys 0.028s
I20260709 12:25:37.255882 30777 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:25:37.272859 30718 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:37.333809 26054 rpcz_store.cc:276] 0709 12:25:36.093549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.093748 (+   199us) service_pool.cc:224] Handling call
0709 12:25:37.288915 (+1195167us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54000f9954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:37.289062 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334217984}
I20260709 12:25:37.208884 30830 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:37.309341 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52349.8us
I20260709 12:25:37.283041 30541 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.256053 30075 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.269330 30587 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:25:37.365341 30862 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:37.365821 30862 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:37.349115 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.366602 30861 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.288957 30786 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.366905 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:37.366768 30862 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.368584 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.369597 30643 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.370182 30862 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:37.371335 30886 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:37.371768 30886 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:37.372345 30665 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.373891 30398 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:37.379597 30859 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.256508 30767 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.273097 30644 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.316934 30884 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:37.355734 30830 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:37.336750 30625 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.356292 26197 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:25:37.357290 30882 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 12:25:37.357576 30874 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.935s	user 0.000s	sys 0.018s
I20260709 12:25:37.289503 30681 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:37.333226 30860 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.278848 30640 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.284000 30842 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.375984 30864 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:37.369632 30827 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.372182 30886 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:37.449245 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 172280us
I20260709 12:25:37.450477 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.368588 30531 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.473142 30818 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.473760 30818 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.382876 30838 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:37.389822 30587 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:37.453248 30767 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.461558 30644 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.463485 30640 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.475908 30818 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.465020 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2935) took 1050 ms. Trace:
I20260709 12:25:37.479720 26127 rpcz_store.cc:276] 0709 12:25:36.414777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.661780 (+247003us) service_pool.cc:224] Handling call
0709 12:25:37.464784 (+803004us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002cabd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:37.464939 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678061312}
I20260709 12:25:37.467556 30797 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.467646 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58316 (request call id 26) took 1558 ms. Trace:
I20260709 12:25:37.481616 26128 rpcz_store.cc:276] 0709 12:25:35.910003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.910125 (+   122us) service_pool.cc:224] Handling call
0709 12:25:37.467344 (+1557219us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54001f6dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:37.467523 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390126976}
I20260709 12:25:37.419749 30643 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:37.483232 30643 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.485965 30643 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.424661 30625 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:37.489464 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2090) took 1302 ms. Trace:
I20260709 12:25:37.490082 30644 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.490688 30644 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.470549 30900 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:37.492663 30900 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:37.493090 30644 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.493945 30842 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.493554 30900 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.471838 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.497303 30075 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:37.498029 30640 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:37.498790 30640 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:37.500588 30643 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.501283 30643 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.501955 30851 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.504007 30643 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.509390 30840 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.499943 30640 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.511107 30681 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.433347 30860 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.517143 30625 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.520462 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:37.523039 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 767) took 1655 ms. Trace:
I20260709 12:25:37.523460 26124 rpcz_store.cc:276] 0709 12:25:35.867954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.869175 (+  1221us) service_pool.cc:224] Handling call
0709 12:25:37.522768 (+1653593us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:37.522923 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3720486144}
W20260709 12:25:37.524822 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74462.7us
I20260709 12:25:37.490011 26200 rpcz_store.cc:276] 0709 12:25:36.186884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.187032 (+   148us) service_pool.cc:224] Handling call
0709 12:25:36.187487 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.194944 (+  7457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.194973 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.198212 (+  3239us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.319302 (+121090us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:36.319331 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.319379 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.357085 (+1037706us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:37.489080 (+131995us) raft_consensus.cc:1685] finished
0709 12:25:37.489107 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.489386 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6957,"raft.queue_time_us":8255,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":11317,"spinlock_wait_cycles":27472128,"thread_start_us":81300,"threads_started":1,"wal-append.queue_time_us":148027}
I20260709 12:25:37.532764 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.538285 30805 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.520006 30860 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.548112 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 2901) took 2049 ms. Trace:
I20260709 12:25:37.548764 26053 rpcz_store.cc:276] 0709 12:25:35.498771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.499339 (+   568us) service_pool.cc:224] Handling call
0709 12:25:35.499685 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.746911 (+247226us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:35.747175 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.599644 (+852469us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:36.599699 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:36.599743 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.547743 (+948000us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400173c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:37.547811 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.548019 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":960,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":60422,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":68,"spinlock_wait_cycles":568079616,"thread_start_us":852052,"threads_started":2,"wal-append.queue_time_us":169515,"wal-append.run_cpu_time_us":1173,"wal-append.run_wall_time_us":1001445}
I20260709 12:25:37.548137 30797 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:25:37.553512 30797 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:37.553944 30797 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:37.555071 30797 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.557354 30625 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.558853 30797 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:37.568444 30644 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:37.569069 30644 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.569494 30644 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:37.572413 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:37.582372 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 303481us
I20260709 12:25:37.588830 30858 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:37.597833 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.598992 26200 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:37.606249 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.609127 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.609944 26048 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:37.611111 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.612298 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.613202 26202 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:37.616678 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.616787 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.618023 26048 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:37.617878 26051 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:37.629336 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.630455 26051 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:37.631587 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.632468 26194 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 8.
I20260709 12:25:37.634232 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.634392 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.637353 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:37.638317 26195 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:37.641388 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.642095 26198 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:37.642302 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.642942 26791 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:37.636889 30860 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:37.645556 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.645835 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:37.641098 30818 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.604488 30858 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:37.647204 30858 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:37.647152 30818 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.649585 30818 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.609201 30817 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.645367 30860 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.649200 30868 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:37.708580 26787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.003s	sys 0.019s
I20260709 12:25:37.708431 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:37.709144 26787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.003s	sys 0.020s
I20260709 12:25:37.709324 26202 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:37.713376 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.714676 26198 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:25:37.710347 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.718501 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:37.722357 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 353839us
I20260709 12:25:37.720057 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.734084 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:25:37.723572 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.736447 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:37.737883 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 99597.4us
I20260709 12:25:37.745857 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:37.744437 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.725183 30818 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:37.760579 30818 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.761103 30818 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.761739 30850 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.723654 30780 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.766187 30780 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.728173 30797 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:37.768205 30797 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.768662 30797 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.745373 30269 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:37.747089 30864 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:25:37.747423 30827 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:37.774492 30827 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:37.774607 30860 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:37.750361 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.748736 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.819986 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.792189 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.831214 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.829290 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.832294 26169 leader_election.cc:400] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17.
I20260709 12:25:37.832635 26169 leader_election.cc:403] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.785830 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:37.785653 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:37.786690 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:37.751837 30644 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.834832 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.835695 26122 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:37.820689 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:37.832574 26783 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:37.837577 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:37.838447 26127 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:37.839258 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1094) took 1418 ms. Trace:
I20260709 12:25:37.839324 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.787304 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:37.800698 26124 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:37.810194 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.932590 26128 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:37.828559 30775 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.842691 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:37.843569 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56357.8us
I20260709 12:25:37.836556 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:37.919428 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.836320 30778 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:37.924293 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 141385us
I20260709 12:25:37.923044 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.925330 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.839775 26789 rpcz_store.cc:276] 0709 12:25:36.421249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.421734 (+   485us) service_pool.cc:224] Handling call
0709 12:25:36.422051 (+   317us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:36.989620 (+567569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.989651 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:36.999738 (+ 10087us) log.cc:844] Serialized 52 byte log entry
0709 12:25:37.382994 (+383256us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:37.629980 (+246986us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.652592 (+ 22612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.838912 (+186320us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54005ce254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:37.838955 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.838998 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:37.839067 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.839196 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":345616,"raft.queue_time_us":212312,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":69011,"spinlock_wait_cycles":559516032,"thread_start_us":865845,"threads_started":3,"wal-append.queue_time_us":398430}
I20260709 12:25:37.776403 30655 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:37.836962 30643 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:37.841820 30780 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.941083 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.711747 30817 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.829277 30640 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.784440 30862 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:37.983033 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:37.984933 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.986233 26783 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:37.988077 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:37.989528 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:37.935103 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 149350us
I20260709 12:25:37.936036 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.990553 26051 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:25:37.842936 26125 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:37.990710 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:37.991117 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 158981us
I20260709 12:25:37.991993 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:37.828513 30802 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.936458 30890 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:37.941851 30855 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:38.002846 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.939133 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:37.948220 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.940151 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.005391 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.929091 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.006376 26129 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:25:37.943133 26121 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:37.943778 26788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1092) took 1523 ms (client timeout 1829 ms). Trace:
W20260709 12:25:38.007355 26788 rpcz_store.cc:269] 0709 12:25:36.420637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.420827 (+   190us) service_pool.cc:224] Handling call
0709 12:25:37.943589 (+1522762us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540056aed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:37.943717 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042102528}
I20260709 12:25:37.946035 30655 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.932934 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.775451 30827 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.968890 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:37.954643 30644 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:37.973076 25951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 275) took 39389 ms (client timeout 15000 ms). Trace:
I20260709 12:25:37.934448 30625 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:37.972934 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:37.941350 30778 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.028689 30269 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:25:37.967507 30777 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.034595 30897 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 12:25:38.034860 30777 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.972635 30767 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:38.036075 30767 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.037119 30777 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.034574 30778 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.013228 30369 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.038564 30767 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.969727 30643 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.040655 30892 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:38.041512 30643 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.041761 30840 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.937033 30895 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:25:38.042335 30840 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:37.976003 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:38.043491 30908 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:25:37.974831 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.976562 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.977766 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:38.044754 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.018127 30655 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.046033 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.979166 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.977180 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.981169 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:37.975884 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.046909 26050 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:38.047381 26198 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:37.933764 30744 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:37.982097 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.047984 30744 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.048415 26053 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:25:37.941071 30898 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 12:25:38.048702 30894 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.990279 26783 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:37.936892 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.049832 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:37.998262 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:38.051069 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:38.051293 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53124.3us
I20260709 12:25:37.945858 30398 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:38.052434 30850 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.052886 30850 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:38.052788 30398 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.053421 30850 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.055167 30398 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.936688 30899 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:38.055872 30899 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.936040 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.005685 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:38.032935 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.032655 26123 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:37.990104 30765 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.986626 30805 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:37.987764 30716 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.993757 30637 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:38.062096 30716 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:38.008505 30896 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:25:38.065133 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1004) took 1103 ms. Trace:
I20260709 12:25:38.065531 26787 rpcz_store.cc:276] 0709 12:25:36.961801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.961950 (+   149us) service_pool.cc:224] Handling call
0709 12:25:36.962371 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.189043 (+226672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.189093 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.189681 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.709577 (+519896us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:37.709619 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.709749 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.064911 (+355162us) raft_consensus.cc:1685] finished
0709 12:25:38.064931 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.065060 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":519555,"threads_started":1,"wal-append.queue_time_us":556445}
I20260709 12:25:37.947156 30775 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:37.995002 30853 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:37.968217 30780 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.033970 30625 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:37.975028 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.074517 26128 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:25:38.075450 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:37.986310 30640 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.041985 30814 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:38.042644 30872 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:37.978605 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:37.840708 30854 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 12:25:38.048483 30744 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:37.964864 30665 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:37.937690 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.011394 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.056324 30899 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:38.008562 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.061146 26123 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:25:38.026247 25951 rpcz_store.cc:269] 0709 12:24:58.583801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.584070 (+   269us) service_pool.cc:224] Handling call
0709 12:25:37.972953 (+39388883us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:30.433333 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:30.738212 (+304879us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:30.776315 (+ 38103us) write_op.cc:435] Acquired schema lock
0709 12:25:30.791560 (+ 15245us) tablet.cc:662] Decoding operations
0709 12:25:30.902485 (+110925us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:30.902622 (+   137us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:30.902684 (+    62us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:25:31.115050 (+212366us) tablet.cc:701] Row locks acquired
0709 12:25:31.115100 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:25:31.115280 (+   180us) write_op.cc:270] Start()
0709 12:25:31.115443 (+   163us) write_op.cc:276] Timestamp: P: 1783599931115211 usec, L: 0
0709 12:25:31.115474 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:25:31.116219 (+   745us) log.cc:844] Serialized 4392 byte log entry
0709 12:25:32.357859 (+1241640us) op_driver.cc:464] REPLICATION: finished
0709 12:25:32.489257 (+131398us) write_op.cc:301] APPLY: starting
0709 12:25:32.489362 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:33.515648 (+1026286us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:33.552709 (+ 37061us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:33.903610 (+350901us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:33.903960 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:25:33.904025 (+    65us) write_op.cc:312] APPLY: finished
0709 12:25:33.966505 (+ 62480us) log.cc:844] Serialized 105 byte log entry
0709 12:25:34.108443 (+141938us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:34.113006 (+  4563us) write_op.cc:454] Released schema lock
0709 12:25:34.118833 (+  5827us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:34.159333 (+ 40500us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:34.159373 (+    40us) write_op.cc:454] Released schema lock
0709 12:25:34.815858 (+656485us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
Metrics: {"mutex_wait_us":77942,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":131066,"apply.run_cpu_time_us":34896,"apply.run_wall_time_us":1670038,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50173,"num_ops":11,"prepare.queue_time_us":393472,"prepare.run_cpu_time_us":20465,"prepare.run_wall_time_us":927582,"raft.queue_time_us":398780,"raft.run_cpu_time_us":13605,"raft.run_wall_time_us":81119,"replication_time_us":1242205,"spinlock_wait_cycles":11080064,"thread_start_us":755825,"threads_started":5,"wal-append.queue_time_us":559430,"wal-append.run_cpu_time_us":23948,"wal-append.run_wall_time_us":3507438}]]}
I20260709 12:25:38.089397 26788 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:38.069571 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:38.028821 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.076099 26198 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:38.090432 26199 raft_consensus.cc:2376] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14. Re-sending same reply.
I20260709 12:25:38.090701 26200 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:38.041239 30892 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.076731 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:38.077584 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.080226 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.017560 30890 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:38.083598 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:38.097797 26046 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
W20260709 12:25:38.084565 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 104805us
I20260709 12:25:38.073164 26201 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:38.028246 30791 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:25:38.073069 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:38.087157 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.058264 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.094439 30658 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:38.095589 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 157965us
I20260709 12:25:38.098920 26195 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:38.072188 30775 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.072589 30531 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.001601 30802 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.111611 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:38.113381 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 122753us
I20260709 12:25:38.114338 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:38.119536 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.120657 26790 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:38.080914 30853 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.115520 30716 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.078574 30765 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.107623 30791 consensus_queue.cc:1243] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:38.125855 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:38.106065 30658 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:38.105733 30835 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.106412 30805 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.185038 30805 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.185496 30805 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:38.187580 30886 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.000s
I20260709 12:25:38.124300 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.127226 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:38.188833 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64663.1us
I20260709 12:25:38.191231 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.188843 26201 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 7.
I20260709 12:25:38.193673 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.129561 30840 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:38.124128 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.155834 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.158035 26203 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.126863 26195 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:38.200875 26788 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:38.201556 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.161127 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:38.202468 30909 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:25:38.162341 30919 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:38.203676 30815 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:38.163012 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104805us
I20260709 12:25:38.204936 26202 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:38.087203 30854 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.107803 30531 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.221359 30531 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.221750 30915 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.222404 30915 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.113210 30269 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.175762 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.178895 30716 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.139858 30868 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:38.184487 30835 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:38.087893 30872 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.096583 30665 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.197741 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.194229 26201 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:38.154768 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:38.206259 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96804.4us
I20260709 12:25:38.207193 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.207974 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.250707 26784 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:38.114259 30712 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:25:38.205228 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.105527 30778 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.223881 30637 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17.
I20260709 12:25:38.224409 30770 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:38.230705 30269 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.231856 30892 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:38.232467 30906 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:25:38.277444 26125 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:38.236397 30716 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:38.237188 30640 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.246073 26203 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:38.195437 30914 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.253202 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.254105 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.213085 30886 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.255316 30527 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.256338 30525 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.254101 30916 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:38.258324 30665 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.229033 30767 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:25:38.256003 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.259763 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:38.260658 30904 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:25:38.333346 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.334724 26783 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:38.335078 26783 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:38.261199 30655 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.335799 26783 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.336913 30802 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.337342 26783 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.337896 30655 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.338423 30853 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.338951 26126 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:38.331144 30269 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.329523 30525 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.340708 30780 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.341863 30525 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.342005 30765 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.343212 30744 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.344530 26791 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:38.344882 30872 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.346267 30744 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.282598 30716 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.263441 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.347208 30369 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:38.270826 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65673.1us
I20260709 12:25:38.273365 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.348083 26197 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:25:38.348359 30858 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.005s
I20260709 12:25:38.348801 30858 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.350358 30860 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.350811 26783 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:38.262409 30915 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.263178 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.167424 30625 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.238430 30824 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:38.291373 26203 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:38.352170 26197 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:38.252480 30835 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:38.271483 30797 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.003s
I20260709 12:25:38.289860 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.294001 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:38.298362 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44359.9us
I20260709 12:25:38.301291 26194 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:25:38.301596 26790 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:38.258849 26124 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:38.250660 30890 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.352046 30858 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.294039 30914 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.273767 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:38.272264 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 82059.1us
I20260709 12:25:38.394291 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:38.384071 30780 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.395275 30780 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.396466 30915 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.396927 30915 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:38.397347 30915 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.397763 30780 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.398968 30712 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.399420 30712 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.399802 30712 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:38.400640 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 126936us
I20260709 12:25:38.401352 30744 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:38.353456 26782 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:38.401849 30744 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:38.383845 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 124157us
I20260709 12:25:38.402303 30744 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.402798 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.404438 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.401495 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:38.350787 30860 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:38.341727 30850 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.380722 30824 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:38.389146 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:38.389997 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 116732us
I20260709 12:25:38.433466 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.385332 26789 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:38.280884 30857 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:38.272646 30770 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:38.302326 30916 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.436326 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.408946 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:38.407133 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.438676 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.439307 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:38.331540 30854 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.331231 30886 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.414019 30766 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.353065 30527 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.450736 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:38.289819 30658 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.454306 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.406828 30914 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.400906 30805 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.424830 30269 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:38.428588 30531 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.281070 30814 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 12:25:38.435158 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.433902 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:38.388202 30853 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.444576 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.444835 26048 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:38.478098 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.445921 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.446940 26047 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:38.446836 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.448092 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.481189 26049 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.404430 30872 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.449167 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:38.449865 26789 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:38.451860 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62788.3us
I20260709 12:25:38.415419 30860 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.391940 30797 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.483572 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:38.486141 26045 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:38.450539 30147 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.487344 26052 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:38.496707 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.450275 30766 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.491752 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.449601 30851 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:38.493896 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:38.494089 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 60247.5us
I20260709 12:25:38.454155 30831 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:38.508450 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.511919 26048 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:38.443199 30770 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.441951 30916 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:38.434168 30780 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:25:38.509996 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:38.512070 26054 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:38.507380 26049 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:38.520926 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.455482 30765 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.455175 30918 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:38.441723 30791 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:25:38.517956 26047 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.453631 30767 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:25:38.514106 30851 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.529551 26120 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:38.484709 30872 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.500013 30840 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.482522 30525 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:25:38.538753 30855 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.001s
I20260709 12:25:38.545220 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.555589 26047 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:38.556142 26054 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:38.556757 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.556949 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.530205 30831 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:38.523289 30766 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.547143 30851 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.526580 30765 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:38.557447 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 103200us
I20260709 12:25:38.557657 26054 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:38.492594 30900 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:38.528960 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.533288 30644 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:38.535012 30854 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.592833 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:38.544775 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:38.499572 30899 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.564757 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:38.548214 30872 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.558410 30655 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:38.548754 30840 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:38.582345 30818 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.821s	user 0.010s	sys 0.000s
I20260709 12:25:38.588613 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.592104 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:38.560175 30917 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:38.546275 30918 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:38.558043 30858 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:38.558724 30855 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:38.596576 26049 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:38.597442 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:38.599428 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54722.5us
W20260709 12:25:38.600046 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.594075 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.617849 26127 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.618011 26128 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:38.612813 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.639293 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:38.604506 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118948us
I20260709 12:25:38.534716 30824 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.601774 30643 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:38.596398 30765 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.644582 30857 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:38.611135 30840 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.599623 30854 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.738257 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.559731 30744 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:38.739580 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 99868.4us
W20260709 12:25:38.737782 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.742548 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.759305 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:38.759934 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.738092 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:38.746279 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.643813 30767 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:25:38.763036 30643 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:38.750748 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.763948 26049 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:38.765338 30926 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:25:38.766963 30658 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:38.769822 30744 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.772368 26123 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:38.773797 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.606678 30797 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.738708 30525 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:38.775480 30525 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:38.609074 30527 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:38.776115 30525 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:38.640282 30818 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.777482 26121 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:38.779203 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:38.779795 26121 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.781322 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.782061 26053 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:38.782987 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.743626 26054 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:38.784108 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.761336 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:38.774540 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:38.761206 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.798547 26121 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:38.787534 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:38.788537 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.757978 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:38.804004 26049 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:38.800133 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.603905 30644 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.808826 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.811540 26128 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
W20260709 12:25:38.811956 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50670.8us
I20260709 12:25:38.806504 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:38.807556 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45271us
I20260709 12:25:38.751174 30824 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.813174 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:38.616887 30655 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.813561 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:38.808858 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.754096 30850 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:38.643501 30866 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 12:25:38.815645 26123 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:25:38.818835 26047 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:25:38.822106 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64235.8us
I20260709 12:25:38.643258 30858 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:38.805689 30925 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
I20260709 12:25:38.821424 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:38.600200 30900 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.847072 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:38.611366 30831 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:38.857599 26047 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:25:38.810598 30815 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.000s
I20260709 12:25:38.811317 30147 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.810122 30767 consensus_queue.cc:1243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:38.868536 26127 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:38.820050 30264 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:38.841893 30797 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:38.869910 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48565.7us
I20260709 12:25:38.870779 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:38.810424 30640 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:38.872907 26198 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:38.803341 30935 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:38.815692 30531 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:38.965823 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 120373us
I20260709 12:25:38.966744 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:38.969185 30929 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.975750 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:38.816031 30805 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.786952 30818 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:38.947350 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 121336us
I20260709 12:25:38.842227 30917 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:38.949854 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.793394 30712 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:38.969795 30929 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:38.956785 30777 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.969103 30928 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
W20260709 12:25:38.883031 30716 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.000s
I20260709 12:25:38.844255 30643 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:39.111618 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:38.972267 30797 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.139446 30814 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.033391 30850 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:39.139922 30797 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.140069 30850 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:39.140847 30850 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.970343 30369 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.142798 30369 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.148759 26201 raft_consensus.cc:2368] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
I20260709 12:25:39.123971 26126 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:25:39.047919 30770 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.000s
I20260709 12:25:39.068383 30917 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:39.138521 30777 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.139309 30818 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:39.149223 30644 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:38.982070 30640 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:39.046938 30264 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:39.114024 30925 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:25:39.110517 30929 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.980268 30147 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.175s
I20260709 12:25:39.165813 30917 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:39.166335 30835 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.003s
I20260709 12:25:38.948429 30857 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.169540 30818 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:39.174988 30900 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.176174 30858 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.123461 30643 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:39.109872 30525 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.168937 30531 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:39.124781 30933 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:39.194903 30854 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.165019 30847 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.169216 30712 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.300823 30531 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.255620 26050 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:25:39.261917 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 224517us
I20260709 12:25:39.149999 30716 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.263594 26054 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:39.173926 30777 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.310513 26054 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:39.276744 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 750) took 1177 ms. Trace:
I20260709 12:25:39.312493 26200 rpcz_store.cc:276] 0709 12:25:38.101452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.106454 (+  5002us) service_pool.cc:224] Handling call
0709 12:25:38.106839 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:38.299623 (+192784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.299658 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:38.381404 (+ 81746us) log.cc:844] Serialized 52 byte log entry
0709 12:25:38.495074 (+113670us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:38.495115 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.495181 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.275876 (+780695us) raft_consensus.cc:1685] finished
0709 12:25:39.275925 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.276404 (+   479us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":947,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5382,"thread_start_us":292004,"threads_started":2,"wal-append.queue_time_us":181109}
I20260709 12:25:39.294663 30264 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.200028 30765 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.167639 30770 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.148573 30814 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.188690 30818 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.343307 26053 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:39.297051 30625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.010s	sys 0.049s
W20260709 12:25:39.344262 30625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.010s	sys 0.050s
I20260709 12:25:39.344671 30951 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.345510 30625 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.180307 30640 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.193691 30643 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:39.353691 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2962) took 1414 ms. Trace:
I20260709 12:25:39.354372 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:39.271847 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.189067 30835 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.356149 30777 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.266481 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:39.359875 30777 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.360872 26126 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:39.362212 30640 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.363335 30777 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.309427 30716 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:39.215806 30851 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.000s
I20260709 12:25:39.372355 30851 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:39.373396 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1718) took 1246 ms (client timeout 1506 ms). Trace:
W20260709 12:25:39.373943 26198 rpcz_store.cc:269] 0709 12:25:38.126412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.153721 (+ 27309us) service_pool.cc:224] Handling call
0709 12:25:39.372950 (+1219229us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.373186 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565318016}
W20260709 12:25:39.377818 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.378625 26045 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:39.381258 30531 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.220561 30850 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:39.217720 30872 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.000s
I20260709 12:25:39.180987 30899 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:39.189420 30915 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.792s	user 0.012s	sys 0.001s
I20260709 12:25:39.407059 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 801) took 1627 ms. Trace:
I20260709 12:25:39.409720 30835 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.220816 30929 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.196943 30933 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:39.194504 30935 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:39.221593 30948 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
W20260709 12:25:39.199993 30766 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.000s
I20260709 12:25:39.466231 30766 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:39.329720 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2924) took 1697 ms (client timeout 2018 ms). Trace:
W20260709 12:25:39.468616 26051 rpcz_store.cc:269] 0709 12:25:37.633520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.634098 (+   578us) service_pool.cc:224] Handling call
0709 12:25:39.329086 (+1694988us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.329451 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811422080}
I20260709 12:25:39.334121 30525 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.335862 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.337387 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 775) took 1506 ms. Trace:
I20260709 12:25:39.339330 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 2959) took 1412 ms. Trace:
I20260709 12:25:39.340070 30765 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.354265 26125 rpcz_store.cc:276] 0709 12:25:37.939303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.002489 (+ 63186us) service_pool.cc:224] Handling call
0709 12:25:39.353080 (+1350591us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.353435 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628988672}
I20260709 12:25:39.365469 30770 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.207293 30847 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:39.368625 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 744) took 1292 ms (client timeout 1581 ms). Trace:
I20260709 12:25:39.370402 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1696) took 2219 ms. Trace:
I20260709 12:25:39.375995 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.377835 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.345706 30951 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.375996 30851 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.404776 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.407135 30915 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.407626 26782 rpcz_store.cc:276] 0709 12:25:37.779515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.779989 (+   474us) service_pool.cc:224] Handling call
0709 12:25:37.780304 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.402095 (+621791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.406197 (+1004102us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54004501d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.406682 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317522176}
I20260709 12:25:39.418746 30835 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:39.419325 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1098) took 1265 ms. Trace:
I20260709 12:25:39.412886 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:39.400756 30946 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:39.433591 30075 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:39.437354 30814 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:39.393409 30625 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.458267 26203 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.461666 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:39.462771 30802 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:39.465034 30864 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:39.472747 26120 rpcz_store.cc:276] 0709 12:25:37.831627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.831910 (+   283us) service_pool.cc:224] Handling call
0709 12:25:37.832162 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.542998 (+710836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.336586 (+793588us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54001860d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.337094 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579396480}
I20260709 12:25:39.473664 26124 rpcz_store.cc:276] 0709 12:25:37.927620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.935662 (+  8042us) service_pool.cc:224] Handling call
0709 12:25:39.338732 (+1403070us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400185bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.339006 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616249088}
I20260709 12:25:39.530212 26126 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:39.488340 26129 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:39.487650 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 755) took 1051 ms. Trace:
I20260709 12:25:39.496775 26199 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:39.497892 30850 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.495416 26790 rpcz_store.cc:276] 0709 12:25:38.153893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.154491 (+   598us) service_pool.cc:224] Handling call
0709 12:25:39.418713 (+1264222us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54003c5854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.418962 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112189312}
W20260709 12:25:39.491284 26194 rpcz_store.cc:269] 0709 12:25:38.075883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.076500 (+   617us) service_pool.cc:224] Handling call
0709 12:25:39.368197 (+1291697us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.368400 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410774912}
I20260709 12:25:39.506148 26054 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:39.503463 30935 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.507788 26198 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.507500 30872 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.508530 26201 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
W20260709 12:25:39.508177 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 500433us
W20260709 12:25:39.509047 30827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.012s	sys 0.011s
I20260709 12:25:39.509701 30766 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.511672 26045 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:39.557437 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:39.557938 30860 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.050s	user 0.015s	sys 0.006s
I20260709 12:25:39.558243 26045 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:39.558821 30369 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:39.558468 30860 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.559528 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:39.516829 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.504812 30915 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:39.560843 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 812) took 1580 ms (client timeout 2067 ms). Trace:
W20260709 12:25:39.561093 30911 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.538s	user 0.007s	sys 0.015s
I20260709 12:25:39.518630 30802 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:39.561393 26786 rpcz_store.cc:269] 0709 12:25:37.980576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.045772 (+ 65196us) service_pool.cc:224] Handling call
0709 12:25:39.560365 (+1514593us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400409a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.560640 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718807808}
I20260709 12:25:39.563091 30802 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.501279 30075 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:39.519297 30933 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.564270 30847 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.564855 30527 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:39.520385 30778 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.523775 26195 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:25:39.525133 30953 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:39.528595 26053 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
W20260709 12:25:39.495401 30914 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.993s	user 0.005s	sys 0.016s
I20260709 12:25:39.481330 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:39.531000 26200 rpcz_store.cc:276] 0709 12:25:38.433081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.335049 (+901968us) service_pool.cc:224] Handling call
0709 12:25:39.484639 (+149590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.533257 30951 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.491604 26196 rpcz_store.cc:276] 0709 12:25:37.151130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.151363 (+   233us) service_pool.cc:224] Handling call
0709 12:25:37.175495 (+ 24132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.420930 (+245435us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:37.435441 (+ 14511us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.990292 (+554851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.990347 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:37.990398 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.369696 (+1379298us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400217fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.369803 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.370149 (+   346us) inbound_call.cc:177] Queueing success response
0709 12:25:39.469274 (+ 99125us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b48000b1b58. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
Metrics: {"mutex_wait_us":277843,"prepare.queue_time_us":28172,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":280984,"raft.run_cpu_time_us":14299,"raft.run_wall_time_us":1477296,"spinlock_wait_cycles":572843264,"thread_start_us":274230,"threads_started":1,"wal-append.queue_time_us":486036,"wal-append.run_cpu_time_us":2914,"wal-append.run_wall_time_us":1424083}
I20260709 12:25:39.536826 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:39.541011 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45902.8us
W20260709 12:25:39.541607 26046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 2925) took 1563 ms (client timeout 2024 ms). Trace:
I20260709 12:25:39.543284 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 754) took 1110 ms. Trace:
W20260709 12:25:39.543637 30824 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.718s	user 0.004s	sys 0.016s
I20260709 12:25:39.529841 30864 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:39.576529 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.544597 30946 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.550088 26194 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:39.550150 30872 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:39.574186 26046 rpcz_store.cc:269] 0709 12:25:37.978100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.978946 (+   846us) service_pool.cc:224] Handling call
0709 12:25:39.541238 (+1562292us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b5400229a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.541433 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117391616}
I20260709 12:25:39.554692 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:39.563587 30398 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:39.522056 30625 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.565310 30527 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:39.550184 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2106) took 1574 ms. Trace:
I20260709 12:25:39.590780 26202 rpcz_store.cc:276] 0709 12:25:37.975801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.976334 (+   533us) service_pool.cc:224] Handling call
0709 12:25:39.549763 (+1573429us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400396a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.549972 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368427264}
I20260709 12:25:39.570333 30817 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.570214 30075 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.591714 30527 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:39.571273 30797 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:39.554147 30827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.012s	sys 0.012s
I20260709 12:25:39.572674 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1111) took 1153 ms. Trace:
I20260709 12:25:39.574522 26199 rpcz_store.cc:276] 0709 12:25:38.432820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.377133 (+944313us) service_pool.cc:224] Handling call
0709 12:25:39.543203 (+166070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.579137 30892 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:39.580349 30802 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:39.594197 30892 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:39.556090 30929 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:39.582362 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148946us
W20260709 12:25:39.582773 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 526722us
I20260709 12:25:39.582973 30864 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.593958 26787 rpcz_store.cc:276] 0709 12:25:38.418991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.440384 (+ 21393us) service_pool.cc:224] Handling call
0709 12:25:38.450880 (+ 10496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.492430 (+ 41550us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:38.492674 (+   244us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.411543 (+918869us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b1800379480. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:25:39.438946 (+ 27403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.439062 (+   116us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:39.439223 (+   161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.572168 (+132945us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b54005ce254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.572225 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.572448 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47697,"prepare.queue_time_us":3058,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":33,"raft.queue_time_us":949054,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":31495,"spinlock_wait_cycles":112258304}
I20260709 12:25:39.596796 26197 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:39.551456 30919 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:39.588045 30398 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:39.597847 30919 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.598661 30398 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.599375 26046 pending_rounds.cc:85] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 8
I20260709 12:25:39.599784 26046 pending_rounds.cc:107] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:25:39.571533 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1680) took 1127 ms. Trace:
I20260709 12:25:39.600124 26046 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:39.582387 30842 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:39.600646 30655 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.584724 26125 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:39.601826 30398 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:39.584877 30917 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.600409 26050 rpcz_store.cc:276] 0709 12:25:38.443976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.444159 (+   183us) service_pool.cc:224] Handling call
0709 12:25:39.571286 (+1127127us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54000f8a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.571473 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787402496}
I20260709 12:25:39.621129 30643 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.633421 26203 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:39.586975 30914 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.640411 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:39.648262 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.648648 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.649165 26202 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:39.650460 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.651819 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.652827 26196 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:39.650429 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.656108 26051 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:39.660409 26050 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:25:39.653892 26194 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:39.596804 30892 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:39.668355 30840 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.006s
I20260709 12:25:39.668584 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.669385 26787 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:39.668962 30840 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.671598 30840 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:39.672394 26047 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:39.672652 30838 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.673941 26054 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:39.659236 30860 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.660470 30919 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.667096 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:39.683012 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.681316 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:39.685742 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.688103 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:39.684617 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:39.689632 26202 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:39.678128 30917 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:39.677313 30959 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:39.700145 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:39.706207 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64092.9us
I20260709 12:25:39.706957 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:39.705041 26045 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:39.712414 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.712728 30797 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.713397 26789 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:39.716722 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:39.717765 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.721324 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80910.1us
I20260709 12:25:39.723956 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:39.722571 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.715909 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 598) took 1111 ms. Trace:
I20260709 12:25:39.725641 26053 rpcz_store.cc:276] 0709 12:25:38.604042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.570437 (+966395us) service_pool.cc:224] Handling call
0709 12:25:39.570946 (+   509us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:39.571489 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.571518 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:39.572211 (+   693us) log.cc:844] Serialized 52 byte log entry
0709 12:25:39.629894 (+ 57683us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:39.629935 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.630002 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.715669 (+ 85667us) raft_consensus.cc:1685] finished
0709 12:25:39.715695 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.715843 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1661,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":18234,"thread_start_us":55723,"threads_started":1,"wal-append.queue_time_us":67959}
I20260709 12:25:39.716076 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.726257 26169 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:39.740413 30527 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.740051 30959 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.743747 30959 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.749953 26198 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:39.728154 26169 leader_election.cc:403] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.752400 26202 raft_consensus.cc:2368] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:39.733562 26199 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:39.755535 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 756) took 1317 ms. Trace:
I20260709 12:25:39.755868 26198 rpcz_store.cc:276] 0709 12:25:38.433326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.375514 (+942188us) service_pool.cc:224] Handling call
0709 12:25:39.750931 (+375417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.757292 26054 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.756989 30892 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.770320 26782 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:39.755069 30643 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.768676 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.776669 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.784058 26791 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:39.784623 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.785730 30917 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:39.787333 30767 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:39.787761 30767 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.788111 30767 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:39.797868 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:39.802085 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:39.759315 30933 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.804677 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:39.808276 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:39.810637 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:39.811656 26120 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:39.814203 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2970) took 1078 ms. Trace:
I20260709 12:25:39.814594 26127 rpcz_store.cc:276] 0709 12:25:38.735749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.900435 (+164686us) service_pool.cc:224] Handling call
0709 12:25:38.900834 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.953630 (+ 52796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.953685 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.953767 (+    82us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:38.953820 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.813778 (+859958us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.813840 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.814049 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120731520}
W20260709 12:25:39.815845 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.779579 30914 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.829828 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:39.829861 26198 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:39.833688 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107662us
I20260709 12:25:39.770090 30818 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.826295 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:39.838006 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.848630 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:39.849484 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:39.850570 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:39.851271 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.856300 30922 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.857424 30780 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:39.857954 30780 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.858383 30780 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:39.861057 30963 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:39.861670 30933 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.872931 26128 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:39.874527 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 2965) took 1266 ms. Trace:
I20260709 12:25:39.874889 26128 rpcz_store.cc:276] 0709 12:25:38.607887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.812469 (+204582us) service_pool.cc:224] Handling call
0709 12:25:38.812800 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.873515 (+1060715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.874247 (+   732us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001f6dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.874454 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374194560}
I20260709 12:25:39.877054 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 600) took 1209 ms. Trace:
I20260709 12:25:39.877468 26046 rpcz_store.cc:276] 0709 12:25:38.667773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.598511 (+930738us) service_pool.cc:224] Handling call
0709 12:25:39.599000 (+   489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:39.601157 (+  2157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.601186 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:39.601843 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:25:39.637995 (+ 36152us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:39.676827 (+ 38832us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.680363 (+  3536us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.680574 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.680637 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.808159 (+127522us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b18002d6ca0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:39.876824 (+ 68665us) raft_consensus.cc:1685] finished
0709 12:25:39.876849 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.876995 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3045,"prepare.queue_time_us":172935,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":2338,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":4797,"spinlock_wait_cycles":12713600,"thread_start_us":35672,"threads_started":1,"wal-append.queue_time_us":152068}
I20260709 12:25:39.879571 30917 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10.
I20260709 12:25:39.857076 30922 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.882041 30922 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:39.886413 25948 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 8 to 10, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 10 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.892194 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1727) took 1460 ms. Trace:
I20260709 12:25:39.894549 26197 rpcz_store.cc:276] 0709 12:25:38.431281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.483315 (+ 52034us) service_pool.cc:224] Handling call
0709 12:25:39.891907 (+1408592us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400301954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.892109 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520382848}
I20260709 12:25:39.896198 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 753) took 1636 ms. Trace:
W20260709 12:25:39.894094 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.896749 26195 rpcz_store.cc:276] 0709 12:25:38.259380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.380188 (+120808us) service_pool.cc:224] Handling call
0709 12:25:39.416910 (+1036722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.566499 (+149589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.895929 (+329430us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400335750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.896130 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300033920}
I20260709 12:25:39.897926 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:39.898106 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.898934 26195 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:39.899747 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:39.899951 26124 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:39.900409 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:39.901084 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:39.902830 30916 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.369s	user 0.015s	sys 0.007s
I20260709 12:25:39.902882 30075 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.903501 30916 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.904043 26128 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:39.857023 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 594) took 1254 ms. Trace:
I20260709 12:25:39.896140 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.906211 26051 rpcz_store.cc:276] 0709 12:25:38.602286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.470271 (+867985us) service_pool.cc:224] Handling call
0709 12:25:39.470803 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.656565 (+185762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.856705 (+200140us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54001b3fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.856919 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403753856}
I20260709 12:25:39.906868 26195 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 18 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:25:39.846616 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:39.833400 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1728) took 1395 ms. Trace:
I20260709 12:25:39.925240 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:39.926685 26129 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14.
W20260709 12:25:39.851406 30945 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.703s	user 0.002s	sys 0.000s
I20260709 12:25:39.932613 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.933422 26197 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.918782 26201 rpcz_store.cc:276] 0709 12:25:38.431534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.552413 (+1120879us) service_pool.cc:224] Handling call
0709 12:25:39.552916 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.554500 (+  1584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.554558 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.555237 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.711437 (+156200us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b18003a37e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:39.752196 (+ 40759us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:39.752252 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.752326 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.817120 (+ 64794us) raft_consensus.cc:1685] finished
0709 12:25:39.817148 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.826874 (+  9726us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":298,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":22798080,"thread_start_us":40577,"threads_started":1,"wal-append.queue_time_us":237589}
I20260709 12:25:39.935062 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:39.938709 26782 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:39.939872 30864 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:39.948750 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 112324us
I20260709 12:25:39.952796 26202 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:39.953504 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 161928us
I20260709 12:25:39.954487 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:39.955739 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:39.958966 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:39.973634 26045 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:39.974802 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:39.975242 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:39.975565 26045 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.976277 30817 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:39.977154 30817 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:39.977329 30903 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:39.977655 30817 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:39.978199 30817 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.978099 30903 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.979496 30946 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:39.960599 30855 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:39.916046 30917 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:39.980549 30868 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.980733 30855 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.981240 30855 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.981017 30917 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.981411 30959 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:39.981684 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1726) took 1550 ms (client timeout 1984 ms). Trace:
W20260709 12:25:39.982170 26203 rpcz_store.cc:269] 0709 12:25:38.431000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.440661 (+  9661us) service_pool.cc:224] Handling call
0709 12:25:39.981461 (+1540800us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400163d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:39.981619 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133738368}
I20260709 12:25:39.983629 30917 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:39.983965 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66770.2us
I20260709 12:25:39.981251 30868 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:39.984936 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:39.937115 30851 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:39.986382 30922 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:39.987017 30868 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:39.973678 30934 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.825s	user 0.002s	sys 0.000s
I20260709 12:25:39.989070 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:39.989440 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:39.989861 26050 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.991683 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1692) took 1500 ms. Trace:
I20260709 12:25:39.960690 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:39.992678 26051 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:39.993891 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:39.995193 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:39.961630 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:39.962692 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45054us
W20260709 12:25:39.964030 30940 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.815s	user 0.001s	sys 0.000s
I20260709 12:25:39.967263 30838 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:40.036667 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:40.040423 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55625.6us
I20260709 12:25:40.041224 30917 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:40.041713 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:40.041908 30917 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:40.043308 30917 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.937722 25948 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 9, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:39.982139 30959 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.047492 30933 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:39.967988 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:39.994792 30767 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:39.975917 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 597) took 1372 ms. Trace:
I20260709 12:25:39.962397 30818 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.000746 26051 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:25:39.975845 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:39.992100 26052 rpcz_store.cc:276] 0709 12:25:38.491225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.496915 (+  5690us) service_pool.cc:224] Handling call
0709 12:25:38.497239 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:38.504490 (+  7251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.504540 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:38.510739 (+  6199us) log.cc:844] Serialized 52 byte log entry
0709 12:25:38.753394 (+242655us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:38.753441 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.753512 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.972659 (+1219147us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:39.991168 (+ 18509us) raft_consensus.cc:1685] finished
0709 12:25:39.991192 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.991467 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8499,"raft.queue_time_us":6904,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8604,"spinlock_wait_cycles":12094336,"thread_start_us":209335,"threads_started":1,"wal-append.queue_time_us":346693}
I20260709 12:25:40.008380 30915 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:40.009356 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:40.009021 30855 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.057668 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.010638 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:40.015172 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.019212 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:40.024940 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:40.025930 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:40.026702 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:40.024945 26787 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:39.996847 26789 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:40.032426 30851 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:40.033111 26201 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:40.033946 26124 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:40.034365 26129 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.044117 30637 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:40.046931 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1331856890001291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:25:40.048943 26195 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:39.961496 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:40.050343 26045 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:40.059845 26786 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:40.062999 26197 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:40.086591 26203 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.053284 26050 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:40.054545 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:40.088513 26202 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:40.088687 26790 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:40.060850 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.055191 26782 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:40.090605 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.058313 30959 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:40.091348 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115718us
I20260709 12:25:40.092584 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:40.062023 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65074.1us
I20260709 12:25:40.098467 30527 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:40.098587 26788 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:40.099359 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:40.099519 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55439us
I20260709 12:25:40.099738 30868 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:40.100116 30075 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:40.100672 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81574.2us
I20260709 12:25:40.100025 26053 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:25:40.139396 26054 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:40.102833 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.103336 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.140781 26126 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:40.106812 26049 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:40.056038 26047 rpcz_store.cc:276] 0709 12:25:38.603783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.484004 (+880221us) service_pool.cc:224] Handling call
0709 12:25:39.530567 (+ 46563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.672864 (+142297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.975658 (+302794us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:39.975849 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290302848}
I20260709 12:25:40.108971 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.112527 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.142275 26128 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.112639 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.112996 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:40.113317 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.116701 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.118805 26052 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:40.120549 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:40.121336 26123 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:40.121997 26094 leader_election.cc:400] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
I20260709 12:25:40.122255 30855 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.049142 30933 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.124788 30398 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:40.102391 25948 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 7, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.133071 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:40.136586 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:40.087939 30637 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:40.146489 26094 leader_election.cc:403] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:25:40.166568 26050 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:40.153505 26121 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:40.174038 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.183686 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.172726 26046 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.183373 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:40.183018 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.192888 26203 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:40.187404 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:40.139046 30868 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:40.190794 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69040.6us
I20260709 12:25:40.138985 30527 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:40.173903 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:40.191077 25948 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:40.198570 26127 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:40.199049 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:40.201555 26052 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:40.203550 26198 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.168910 30637 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:40.202947 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.164641 30903 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:40.209622 26051 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.173957 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.214746 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:40.215783 26197 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.215830 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:40.217423 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:40.217973 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.215008 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:40.219918 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.222121 26045 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:40.198335 30818 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
I20260709 12:25:40.224717 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:40.164482 30780 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.218591 25948 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:40.228281 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:40.230885 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.231805 26047 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:40.217159 30916 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.198293 30864 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:25:40.197343 30398 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.256961 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 9.
I20260709 12:25:40.217321 30855 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:40.218914 30933 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.249667 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:40.259586 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 347641us
I20260709 12:25:40.164503 30817 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:40.265712 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91763.1us
I20260709 12:25:40.266753 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.271099 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:40.273562 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1694) took 1661 ms. Trace:
W20260709 12:25:40.291843 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.297181 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 101476us
I20260709 12:25:40.258613 30931 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:40.308558 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.287997 25948 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 9, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 9 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.312780 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:40.314009 26054 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:40.285526 30855 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:40.274570 30959 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:25:40.318795 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:40.320014 30959 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.320586 30959 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:40.321985 30959 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.323661 30959 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:40.332405 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105903us
I20260709 12:25:40.332561 30527 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.334599 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:40.335090 26051 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:25:40.336282 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.274010 26048 rpcz_store.cc:276] 0709 12:25:38.508669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.512579 (+  3910us) service_pool.cc:224] Handling call
0709 12:25:38.512936 (+   357us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:38.513354 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:38.513386 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:38.518441 (+  5055us) log.cc:844] Serialized 78 byte log entry
0709 12:25:39.758266 (+1239825us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b180028c960. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:39.852342 (+ 94076us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:39.852413 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.852580 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.132303 (+279723us) raft_consensus.cc:1685] finished
0709 12:25:40.132347 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.170114 (+ 37767us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2205,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":13301,"spinlock_wait_cycles":11133184,"thread_start_us":66219,"threads_started":1,"wal-append.queue_time_us":1611536}
I20260709 12:25:40.340394 26201 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:40.342614 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.344673 30935 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:40.268842 30903 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:25:40.347645 26053 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:40.353852 26050 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:40.364694 30860 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.372351 30933 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:40.373448 30851 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:40.374393 30915 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:40.376248 30922 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:40.391175 26126 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:40.296425 30840 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:40.338307 30931 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:40.396468 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 129835us
W20260709 12:25:40.397115 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57622.6us
I20260709 12:25:40.397900 30851 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:40.398866 30915 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.399844 30922 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:40.385159 30933 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.285822 30827 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 12:25:40.407006 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 146010us
W20260709 12:25:40.413736 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 369144us
I20260709 12:25:40.411679 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:40.409083 26127 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:40.409525 26128 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:40.379814 30767 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:40.409016 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:40.415999 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:40.355427 30797 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:40.410285 25948 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.382988 30860 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:40.395779 30840 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:40.410382 30915 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:40.410149 30922 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:40.427580 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 129303us
I20260709 12:25:40.410284 30851 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:40.426801 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.413996 30933 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.430869 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:40.430462 30855 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:40.352094 30868 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.430341 30916 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:40.441105 30946 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.445894 30855 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:40.456998 30855 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:40.456451 30931 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.460356 30946 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.462815 30946 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:40.463626 30946 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:40.464483 30946 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:40.467512 30398 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.469828 26169 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14.
I20260709 12:25:40.470171 26169 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:40.471647 30838 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:40.472175 30075 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14.
I20260709 12:25:40.472486 30838 consensus_queue.cc:1243] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:40.473093 30838 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.473466 30838 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:40.474066 30892 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.000s
I20260709 12:25:40.475378 26198 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:40.476466 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:40.476786 26167 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.813s (SENT)
I20260709 12:25:40.477326 26128 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:40.477372 26127 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.476504 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:40.448444 30817 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.456349 30802 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:40.423739 30959 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:25:40.467206 30946 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.450567 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.447302 25948 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
W20260709 12:25:40.424062 30864 raft_consensus.cc:670] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } [suppressed 2 similar messages]
I20260709 12:25:40.464898 30851 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.439445 30840 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:40.456195 30857 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.471796 30855 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.484170 25948 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:40.491770 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.496915 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.497468 26049 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:40.497618 26128 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:40.499408 30780 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:40.494151 30860 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.501725 30860 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.501971 30922 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.503862 30922 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:40.505935 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:40.484937 26202 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:40.474447 30838 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.512987 30838 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:40.514276 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71562.2us
I20260709 12:25:40.489473 30767 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:40.511406 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56662.7us
I20260709 12:25:40.496714 30857 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.524575 30918 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:40.511384 30716 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.524653 30892 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:40.525394 30918 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:40.525588 30892 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.526198 30892 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.525612 30716 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.528355 30716 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.521327 30765 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:40.529129 30765 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.525236 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.532908 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:40.533679 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:40.534680 30824 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:40.535115 30959 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:40.534614 30892 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.535403 30824 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:40.535733 30959 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:40.536017 30824 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.537268 30892 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.536863 30959 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.539847 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.542861 30716 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:40.545892 30855 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.548755 30851 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.552435 30915 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.534775 30943 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:25:40.554093 30915 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.552186 25948 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:40.554939 30824 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.556417 25948 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 9, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:40.557745 30767 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:40.582245 30767 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.583065 30824 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:40.508822 30791 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.584479 30791 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:40.585729 30791 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.587615 30791 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.564105 26046 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:40.561654 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:40.589480 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67139.6us
I20260709 12:25:40.591495 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:40.560494 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:40.594238 30818 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:40.594901 30818 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.595392 30818 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:40.596590 30398 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:40.597884 30398 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.598399 30398 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:40.511651 30817 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:40.601794 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.605650 25948 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:40.571584 26127 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:25:40.609349 26127 pending_rounds.cc:85] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
I20260709 12:25:40.609835 26127 pending_rounds.cc:107] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:25:40.610172 26127 raft_consensus.cc:2889] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:40.534616 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.532002 30765 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.576212 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.612469 30765 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:40.612645 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78112.1us
I20260709 12:25:40.613180 30765 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:40.613765 30765 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:40.565311 30918 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:40.616596 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.620777 30957 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:40.621799 30957 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:40.622308 25948 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.574633 30919 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:40.635138 26121 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:40.635568 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44187.5us
W20260709 12:25:40.636768 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 121640us
I20260709 12:25:40.599553 30398 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.637797 26093 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.638144 30398 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.636909 30831 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:40.640133 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:40.642328 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.641999 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.646417 26128 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:40.649977 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 110213us
I20260709 12:25:40.640411 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.654066 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.640196 30840 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.681579 25948 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 10, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 10 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:40.640657 30831 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:40.705542 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.712108 26128 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:25:40.645134 30824 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:25:40.696645 30957 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.641199 30637 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:40.728943 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 298139us
W20260709 12:25:40.730072 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.733875 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:40.697160 30919 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:40.756193 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:40.737646 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 468227us
I20260709 12:25:40.731040 25948 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.789382 26128 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:40.752990 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 302502us
W20260709 12:25:40.815672 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 172989us
I20260709 12:25:40.819856 26019 leader_election.cc:400] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:40.730322 30916 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:40.722306 30864 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:25:40.816987 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:40.790485 30527 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:40.840571 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:40.841605 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50731.6us
I20260709 12:25:40.844327 26124 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:40.850540 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:40.754983 30797 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:40.849822 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:40.858095 26788 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:25:40.842238 25948 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 9, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 9 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.737066 30957 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.848826 30946 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:40.868552 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:40.876754 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57288.8us
I20260709 12:25:40.877611 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.878890 26129 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:40.882905 30916 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.883407 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.885208 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.886097 26786 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:40.887811 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:40.867528 30840 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.846585 30943 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:25:40.927590 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.932711 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:40.913870 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.926584 30797 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.938717 25948 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:40.939217 30943 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:40.940696 30980 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:40.941190 30980 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:40.941443 30972 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:25:40.942180 30980 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.943660 30903 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.944108 30903 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:40.945097 30903 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.929606 30824 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:40.901226 30959 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:40.933169 26790 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:40.940181 25948 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 9 to 10, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 10 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.863706 30933 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.894148 30917 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:40.969990 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:40.948522 30981 raft_consensus.cc:2764] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:40.960340 30919 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:40.895210 30857 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:40.965794 30986 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.971989 26124 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:25:40.956547 30959 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:40.972589 26791 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:40.976184 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:40.943821 30980 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:40.992777 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:40.991750 26787 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:40.993974 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:41.002882 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.004179 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:40.891794 30527 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:25:41.006760 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.010728 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:40.996654 25948 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:40.974084 30802 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.973922 30917 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:41.055076 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:40.988453 30780 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:40.988853 30817 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:41.064775 30824 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.064376 30817 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.886968 30831 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:40.998353 30922 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:25:41.035403 30978 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:25:41.053354 30986 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.060801 30802 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:40.988668 30918 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:41.072635 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101697us
W20260709 12:25:41.106767 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.970599 30946 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.058161 30917 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.088642 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.087599 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:41.083596 30860 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 12:25:41.069206 30983 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:41.161996 26782 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:41.088178 26791 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:41.136955 25948 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 5 to 7, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 7 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:25:41.177923 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:41.190285 26127 raft_consensus.cc:2368] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15.
I20260709 12:25:41.068917 30637 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:41.191872 26169 leader_election.cc:400] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15.
I20260709 12:25:41.192221 26169 leader_election.cc:403] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:25:41.090292 30977 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 12:25:41.074709 30855 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:41.185199 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 368315us
I20260709 12:25:41.081178 30981 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:41.090811 30838 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.003s
I20260709 12:25:41.087319 30986 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.080611 30527 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.070536 30817 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:41.093309 30802 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.192660 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.068279 30791 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.077373 30985 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:41.220141 26202 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:25:41.222493 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1736) took 1293 ms. Trace:
I20260709 12:25:41.095038 30918 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:41.182435 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.017s
I20260709 12:25:41.083087 30818 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.088363 30780 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.088507 30824 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.083552 30922 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.213972 30992 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.217240 30933 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:41.232651 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.018s	sys 0.017s
I20260709 12:25:41.230875 26200 rpcz_store.cc:276] 0709 12:25:39.928665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.969463 (+ 40798us) service_pool.cc:224] Handling call
0709 12:25:39.969855 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.049532 (+ 79677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.049568 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.086017 (+ 36449us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.518117 (+432100us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:40.518194 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.518317 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.089530 (+571213us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:41.222155 (+132625us) raft_consensus.cc:1685] finished
0709 12:25:41.222180 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.222425 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40078,"raft.queue_time_us":26965,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3089,"spinlock_wait_cycles":52104448,"thread_start_us":388172,"threads_started":1,"wal-append.queue_time_us":620054}
I20260709 12:25:41.196133 30919 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:41.202179 30765 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.588s	user 0.015s	sys 0.000s
W20260709 12:25:41.237349 26125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.012s
I20260709 12:25:41.226397 30986 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.201174 30981 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:41.265012 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 394001us
W20260709 12:25:41.248436 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 320905us
I20260709 12:25:41.254621 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:41.204737 30855 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:41.259043 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81417.3us
I20260709 12:25:41.228842 30946 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:41.209125 30398 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
W20260709 12:25:41.260800 26125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.004s	sys 0.012s
I20260709 12:25:41.206952 30983 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:41.073128 30959 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.267838 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:41.271045 30818 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.272395 26202 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:41.221159 30985 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:41.302711 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 292075us
I20260709 12:25:41.257853 30994 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.261222 30957 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:41.275097 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 634984us
I20260709 12:25:41.262355 30992 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:41.274660 30915 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.720s	user 0.014s	sys 0.000s
W20260709 12:25:41.278313 30851 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.004s
I20260709 12:25:41.305409 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.306347 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.224519 30527 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:41.266988 30981 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.315661 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.315891 26127 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:41.316334 26122 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:41.316457 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.318948 26046 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.326503 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.328862 26126 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:41.330506 26202 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
W20260709 12:25:41.331312 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 217297us
I20260709 12:25:41.264307 30765 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.344698 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:41.349515 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81759.1us
I20260709 12:25:41.266728 30986 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.354192 26129 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:41.309584 30838 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:41.355656 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.356618 26048 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:41.350780 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:41.354131 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:41.360220 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55252.4us
I20260709 12:25:41.312196 30916 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.359436 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 95594.5us
I20260709 12:25:41.366143 26124 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:25:41.371241 26046 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:41.372905 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:41.376372 30998 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.365097 30716 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.327208 30917 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.309333 30791 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:25:41.423884 26199 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:41.423969 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.376366 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:41.358036 30959 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:41.428578 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1428436840001268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260709 12:25:41.397352 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.361227 30838 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:41.430836 30716 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.434409 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58157us
I20260709 12:25:41.431241 30979 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:41.437800 30780 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.437844 31005 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:41.438786 31005 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:41.439347 30527 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.439808 31005 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.446959 30765 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:41.389739 31004 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:41.437079 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:41.389515 30824 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:41.457024 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 102959us
I20260709 12:25:41.395476 30917 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:41.461668 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:41.451650 30780 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.473023 30838 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.395516 30992 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.393606 30922 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:41.478083 30840 raft_consensus.cc:2764] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:25:41.433993 30994 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:41.374389 30916 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.494678 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:41.481213 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 83369.4us
I20260709 12:25:41.475188 30986 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:41.499243 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 171353us
I20260709 12:25:41.456082 30791 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:41.501410 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:41.509864 30903 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.454794 30716 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:41.512395 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.512884 30838 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.514829 30716 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:41.515331 30791 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.515784 30791 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.516148 26121 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:41.518630 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.519999 26048 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:25:41.520988 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.523195 26129 raft_consensus.cc:2368] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:41.524415 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.392620 30855 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.523175 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.527642 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:41.521374 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:41.528051 26127 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:41.380187 30918 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:41.526553 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.529592 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:41.387516 30985 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:41.535337 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.531576 30840 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:41.536705 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:41.537528 26051 raft_consensus.cc:2376] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12. Re-sending same reply.
I20260709 12:25:41.538586 30527 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:41.539311 30527 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:41.499089 30780 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.540362 30527 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.536156 26054 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.536901 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:41.491657 30765 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:25:41.545840 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.497946 30994 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:41.548862 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.547663 26127 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:41.557122 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.558531 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:41.560537 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.561944 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.585569 26125 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:41.585884 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.596110 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44045.4us
I20260709 12:25:41.614742 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:41.697384 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:41.700173 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.700902 26046 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 9: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:41.701150 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:41.701982 26052 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:41.700001 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:41.702920 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:41.703733 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:41.704008 30980 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.005s
I20260709 12:25:41.705610 26048 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:41.708606 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.542639 30780 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:41.625504 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:41.709755 30943 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.710135 26049 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.710536 30943 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:41.659559 31013 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:25:41.712771 30916 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.712908 30943 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.659989 31009 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:41.690714 26051 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:25:41.618611 30827 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:41.546679 30824 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:41.693109 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 190890us
I20260709 12:25:41.708627 26054 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
W20260709 12:25:41.693768 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 172679us
I20260709 12:25:41.392613 30983 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.525995 30917 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:41.752884 30980 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:41.695423 26050 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:41.727252 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.729028 30943 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:25:41.728996 30840 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:41.733278 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.736038 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.763770 26199 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:41.736296 30838 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.767343 30838 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.767966 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:41.708688 30985 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:41.768963 30840 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.733682 30918 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:41.736783 30857 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:41.737491 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.772121 26202 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:25:41.737565 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:41.739269 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.741056 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.744025 30716 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.744967 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:41.769820 30985 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.778482 26195 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.745857 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:41.746364 30892 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.746990 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:41.747275 30075 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:41.753974 30957 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:25:41.715670 31009 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.791330 30979 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:41.791862 30838 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:41.793040 30716 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:41.781401 31000 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.794551 30979 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:41.795147 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:41.795936 26200 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:41.777670 26048 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:25:41.782763 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55575.3us
I20260709 12:25:41.797494 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.783361 26020 leader_election.cc:400] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:41.798287 26020 leader_election.cc:403] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:41.728363 31008 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:41.786929 26197 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.788547 30918 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.798202 26049 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:41.799913 26790 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:25:41.786671 30857 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.818269 26093 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:41.820377 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.820182 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.820647 30986 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:41.787617 30855 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:41.822176 30855 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.822602 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.784633 30791 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.785830 26121 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:41.833047 26124 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:41.822548 30855 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.778890 30916 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:25:41.817397 30980 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:41.839761 30818 raft_consensus.cc:2764] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:41.825047 30791 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.843321 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.832652 30838 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.844586 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.844732 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.845752 26786 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:25:41.844623 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.885360 26200 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:41.829408 30986 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.846696 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:41.818060 30946 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:41.829295 30780 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.848474 31001 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:41.884944 26786 pending_rounds.cc:85] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 13
I20260709 12:25:41.906672 26786 pending_rounds.cc:107] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:25:41.907069 26786 raft_consensus.cc:2889] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:41.849818 30919 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.586s	user 0.003s	sys 0.013s
I20260709 12:25:41.830596 30935 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.801332 31009 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.852746 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.796783 30992 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.860832 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:41.865157 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 111304us
I20260709 12:25:41.864158 30917 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:41.806397 30985 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:41.883342 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:41.787242 30892 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.886534 26197 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:41.887270 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:41.918002 26202 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:41.886293 26785 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:41.846056 30857 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:41.920625 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 172306us
I20260709 12:25:41.862185 30922 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.862896 30980 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:41.913453 26125 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:41.818799 31008 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.846169 31000 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:25:41.923802 26787 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:41.906339 31001 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:41.860589 30824 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:41.911473 30935 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:41.914037 30919 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15.
I20260709 12:25:41.930985 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.937587 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.937602 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:41.856170 30994 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.938925 30994 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.939087 31001 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.940609 31001 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.934027 30716 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:41.944552 30981 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.584s	user 0.015s	sys 0.011s
I20260709 12:25:41.945026 30981 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.947901 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:41.907632 30838 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.909113 30986 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.963548 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.909324 30946 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:41.962952 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.970191 26200 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:41.947360 30935 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.971405 26126 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:41.931241 30922 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.972563 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.972890 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:41.977702 26127 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:41.973760 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.981750 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.958557 30985 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:41.985299 26200 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:41.962697 30992 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.963074 30075 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:41.989180 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:41.991925 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:41.992740 26197 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:42.000763 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150065us
I20260709 12:25:42.001910 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.011001 26198 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:25:42.015228 31032 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.015828 31032 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.018507 31032 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.021595 30791 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:42.022647 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.023535 26200 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:42.024578 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.025424 26202 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:41.986146 30860 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:41.946862 30959 consensus_queue.cc:1175] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 13 status { last_received { term: 13 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 13 index: 10 } }
I20260709 12:25:41.985730 30985 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:42.034209 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.029623 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.037096 26126 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 8.
I20260709 12:25:42.043812 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:42.044629 26129 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.043566 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.052096 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.003533 30075 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:42.023243 30716 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.057093 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.057854 26048 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:42.056028 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.060045 26054 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:42.029250 30916 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.061998 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.063570 26787 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:42.033378 30818 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.033937 31025 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.039964 30985 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.069957 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.071336 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.075292 26050 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:42.075639 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:41.989459 30917 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.076977 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:41.987733 30980 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.060552 30716 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.083484 30979 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:42.063622 30922 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.083938 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:42.092950 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46849us
I20260709 12:25:42.093753 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.084517 30527 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.085501 26786 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:42.086092 26788 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:42.099818 26126 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:42.077219 30992 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.070358 30916 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:41.994645 30824 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.078344 30981 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:42.107707 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.082214 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53842.7us
I20260709 12:25:42.103514 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.107949 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.055207 30791 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.110409 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.115137 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.117908 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:42.103466 31027 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:25:42.128918 26049 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:42.129384 30994 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.129892 30994 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.131604 30916 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.076926 31025 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.133157 26782 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:42.134025 31025 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.136773 30946 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:42.137730 31022 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:25:42.139207 31023 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:25:42.141005 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.141251 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.142550 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.143289 26787 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14.
I20260709 12:25:42.143419 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.144084 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.144645 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.144721 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.145213 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.145364 26791 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.146261 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.147068 30918 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.148510 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.149345 26788 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:25:42.113709 31021 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:25:42.113936 31033 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.132053 30994 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.095829 30716 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.091527 30979 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:42.156082 26783 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:42.144675 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.156373 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.159221 26782 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:42.101915 30831 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:42.169071 26791 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:42.169598 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.171149 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.162312 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:42.163399 26168 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:42.172470 26168 leader_election.cc:403] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:42.174589 26129 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:42.122161 30922 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.154078 31026 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:42.182278 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.182540 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.178800 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.183063 26054 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:42.183281 26790 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.152407 30780 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13.
I20260709 12:25:42.188820 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.160772 30981 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.164324 30979 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:42.196477 30986 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.177387 31009 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.197331 31009 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.199515 31009 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.151903 31033 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.184008 30918 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.192560 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.202137 31033 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.153293 30985 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.204828 30985 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.168768 31005 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:42.168773 30840 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:42.213165 26054 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:42.217036 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.219590 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.220552 26050 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:42.178889 30916 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.179589 30892 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.206962 30981 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.232796 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.182587 31026 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.233736 26790 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:42.180392 31008 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.188988 31024 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:42.252514 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:42.223315 30986 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:42.164471 30075 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.233450 30946 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.234509 30838 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.235739 30985 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.250401 31026 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.249388 31028 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.274011 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.222611 30922 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.257265 30986 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.290525 31028 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.292516 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.292788 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.293404 26790 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12. Re-sending same reply.
I20260709 12:25:42.293596 26791 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14.
I20260709 12:25:42.296206 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.296207 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.298583 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.262362 31009 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:42.299485 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.299978 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.300033 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:42.304302 26129 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:42.306320 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.307359 26122 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:25:42.306149 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.309239 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.310654 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.295452 30986 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:42.295580 30918 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12.
I20260709 12:25:42.257079 31024 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.300042 30916 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.313983 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.315461 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:42.313087 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 96189.6us
I20260709 12:25:42.317607 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.318102 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:42.319265 26124 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:42.319430 26787 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 10: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:42.301651 31045 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.321079 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 3.012s (SENT) [suppressed 36 similar messages]
I20260709 12:25:42.322360 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:42.309260 30981 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:42.340008 26786 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:42.350679 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.325052 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:42.349120 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.356828 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.359921 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.313791 30840 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:42.369872 26126 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:25:42.381351 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56365.4us
I20260709 12:25:42.382556 26129 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:42.385479 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.388078 30981 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:42.388569 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.389408 31028 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.389204 30981 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.390697 30979 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.393767 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.394515 26045 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:42.388140 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.396039 26784 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:42.396453 26128 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:42.370801 30780 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.396381 31033 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.397854 31033 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.398280 31033 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.398312 30979 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:42.399685 30855 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.000s
W20260709 12:25:42.400261 31037 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:25:42.369340 31044 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:42.402724 31044 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:42.404392 30922 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.372972 30791 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:42.413316 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53496.6us
I20260709 12:25:42.414603 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:42.415467 30922 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.415944 30922 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.326215 30916 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.416126 30780 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:42.418404 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44769.6us
I20260709 12:25:42.418598 30916 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.420383 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.421545 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:42.428673 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.430974 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:42.436847 26047 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:42.437656 26199 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:42.439379 26201 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:42.440493 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.441156 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:42.442111 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.443043 26048 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:42.444304 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.447058 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.449534 26194 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:42.449990 26200 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:42.450495 26047 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:42.451045 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.451692 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:42.452384 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.455066 26051 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:25:42.455345 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.456061 26053 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:42.456139 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.457247 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.416396 30922 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.421005 31005 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:42.464208 26046 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:25:42.465693 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.466572 26788 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:42.469945 26791 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:42.460863 26784 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:42.464114 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.461200 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.419818 31033 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.449292 30780 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:42.355624 31045 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.477085 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.410719 30855 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.481798 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:42.483520 26201 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:42.487234 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:42.488989 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56514.2us
I20260709 12:25:42.403987 31044 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.491410 31001 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.491786 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:42.492705 31049 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:25:42.496940 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.507135 26788 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:42.508128 31026 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.508222 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:42.514547 26788 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:25:42.515506 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.500394 31061 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.474900 31056 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.502812 26052 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:42.476540 31048 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:25:42.503587 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.482986 30780 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:42.505497 26201 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:42.485333 30075 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.505841 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.455706 30840 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.497973 31045 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.529989 31001 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.532416 31055 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:42.522559 31005 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:42.591318 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 87827.1us
I20260709 12:25:42.527848 30780 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:42.596212 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 90422us
I20260709 12:25:42.613802 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.602818 30855 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.607709 30922 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.608858 26054 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:42.609073 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:42.593474 31058 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.073s	user 0.001s	sys 0.004s
I20260709 12:25:42.593803 30916 raft_consensus.cc:2764] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:42.616721 26167 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14.
I20260709 12:25:42.595635 31056 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:42.621264 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:42.618235 30918 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.598290 30791 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.598487 30824 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.626834 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.634930 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.635838 30918 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.636713 30918 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.637712 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.638087 26201 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:42.638813 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.639005 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.639827 30824 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:42.524819 31061 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.640192 26787 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:25:42.641445 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.628332 31026 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.630574 31056 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.632493 30791 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.610119 30716 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:42.648828 30716 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.649901 31056 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.651131 30791 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.652765 31001 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:42.653949 31067 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:25:42.655138 30935 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.669s	user 0.014s	sys 0.000s
I20260709 12:25:42.663163 31061 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.665505 30986 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:42.629202 31066 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.007s	sys 0.012s
I20260709 12:25:42.667734 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.678754 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 7.
I20260709 12:25:42.668967 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.655650 30935 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.670189 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.671713 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.672072 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.674119 30922 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.639012 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.675825 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.680346 26202 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:25:42.701642 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.700565 26197 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:42.702517 26784 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:42.703897 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.692077 26790 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:42.692224 26787 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:25:42.708273 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:42.708890 26195 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.710871 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.712064 26125 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:42.677882 30919 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.680537 30716 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.713470 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.715211 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.716637 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:25:42.717875 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.701189 31001 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.707906 31026 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.719866 30765 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.724750 30765 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.725173 30765 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.713409 30824 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.731210 26784 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:42.732214 30838 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:42.732559 30527 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:42.732744 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 629291us
I20260709 12:25:42.734831 31001 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.735316 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.736419 30831 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:42.737318 31001 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.738230 26787 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:25:42.738596 26787 pending_rounds.cc:85] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 10
I20260709 12:25:42.738834 26787 pending_rounds.cc:107] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:25:42.739089 26787 raft_consensus.cc:2889] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.742274 31000 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.743525 30979 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.718859 30919 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.747467 30919 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.765807 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:42.770089 26195 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:42.760463 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:42.771853 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.772688 26784 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:42.775462 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.776207 26203 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:42.779875 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:42.780716 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.781252 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:42.782110 26122 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:42.783102 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.784919 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.786459 26790 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:42.788564 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:42.797973 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.799612 26122 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.799955 26123 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:42.790532 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:42.800175 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.801007 26129 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:42.787498 31064 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:25:42.816278 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:42.780867 30838 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.820184 30860 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:42.820598 30998 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:42.821316 30998 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:42.744177 30979 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.821266 30860 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:42.798214 30992 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:42.781183 30919 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 12:25:42.817225 30943 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.003s	sys 0.011s
W20260709 12:25:42.844714 31059 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.114s	user 0.000s	sys 0.003s
I20260709 12:25:42.757367 31000 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.815646 30527 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:42.845182 31026 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:42.861909 26784 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:25:42.861577 30943 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.004s	sys 0.011s
I20260709 12:25:42.871330 26790 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:42.869798 31026 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.872609 30860 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:42.879741 31068 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:42.881232 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2137) took 1030 ms. Trace:
I20260709 12:25:42.881672 26198 rpcz_store.cc:276] 0709 12:25:41.850492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.860524 (+ 10032us) service_pool.cc:224] Handling call
0709 12:25:42.881035 (+1020511us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b5400399c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:42.881167 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104160640}
I20260709 12:25:42.874171 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.884490 26201 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:42.843595 30838 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.850387 30992 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:42.893352 31078 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:42.888278 31026 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:42.901368 30818 raft_consensus.cc:2764] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:42.903378 30979 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.889186 30935 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.911182 30992 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.918982 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:42.919749 26198 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:42.922993 26122 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:42.933166 26789 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:42.916437 30986 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.935242 30838 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:42.935751 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.936249 31000 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:42.885955 30998 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:42.993096 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:42.994028 26046 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.937762 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 803) took 1018 ms. Trace:
I20260709 12:25:42.939026 31041 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:42.944823 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1038) took 1101 ms. Trace:
I20260709 12:25:42.945204 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:42.997867 26127 raft_consensus.cc:606] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.998376 26127 raft_consensus.cc:614] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:42.946419 26079 heartbeater.cc:646] Failed to heartbeat to 127.24.71.190:42395 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.24.71.190:42395 timed out after 15.000s (SENT)
I20260709 12:25:42.951900 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:42.953431 30981 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:42.955403 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:43.013175 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57873.1us
I20260709 12:25:42.960902 31026 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:42.962307 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:42.962913 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:42.971464 26791 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:42.982767 26122 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:42.996611 26783 rpcz_store.cc:276] 0709 12:25:41.842929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.843055 (+   126us) service_pool.cc:224] Handling call
0709 12:25:42.944637 (+1101582us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:42.944756 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593389312}
I20260709 12:25:42.986131 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:42.986266 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:42.987960 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.988945 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.989715 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:42.989221 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:42.993348 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:43.018316 26050 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:43.018749 26047 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:43.019467 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:43.019665 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:43.020668 26050 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:42.995662 26121 rpcz_store.cc:276] 0709 12:25:41.919797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.920001 (+   204us) service_pool.cc:224] Handling call
0709 12:25:41.920446 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.934545 (+ 14099us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:42.019980 (+ 85435us) log.cc:844] Serialized 14 byte log entry
0709 12:25:42.074212 (+ 54232us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.081646 (+  7434us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.081839 (+   193us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:42.081887 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.936950 (+855063us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b18003093a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419
0709 12:25:42.937453 (+   503us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b54003688d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:42.937498 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.937541 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:42.937560 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.937704 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2096,"prepare.queue_time_us":85391,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":18552,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":7167,"spinlock_wait_cycles":36561024,"thread_start_us":130444,"threads_started":2,"wal-append.queue_time_us":89724}
I20260709 12:25:42.941188 30765 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.011660 26052 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:43.027781 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:43.028652 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:25:43.028820 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 66597.9us
I20260709 12:25:43.022182 26120 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:43.031771 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.033246 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:43.033847 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44172.8us
I20260709 12:25:43.034814 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.036533 26050 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:43.036468 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 162363us
I20260709 12:25:43.039755 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.041152 26046 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:43.047845 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:43.049948 26198 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:43.041827 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.051389 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:43.054322 26196 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:43.058003 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.014899 31041 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:43.064216 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:43.027356 30838 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:43.062474 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:43.030058 30527 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:43.067518 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 184199us
I20260709 12:25:43.032850 30919 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:43.076210 26053 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 12:25:43.066804 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:43.066717 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:43.067999 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.042001 30986 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:43.091289 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.105387 26197 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.088194 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.053413 30892 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:43.041999 31032 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:42.999966 30824 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:25:43.117266 26045 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:43.062073 30857 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.064069 31000 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.065436 31053 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:43.065814 30716 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:25:43.041450 31055 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:25:43.075335 30780 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:25:43.128465 30716 consensus_queue.cc:1243] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:25:43.128674 31000 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:43.137190 31034 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:25:43.061764 31001 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:25:43.137919 31071 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 12:25:43.138518 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70626.2us
I20260709 12:25:43.139568 30992 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:43.140390 31072 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:43.121752 31032 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:43.169369 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:43.141388 31077 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:43.141999 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:43.176137 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:43.142697 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.143436 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.143664 26047 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:43.144162 31051 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:25:43.147169 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:43.147794 26049 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:43.148661 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:43.150429 26045 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:43.150697 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:43.182365 26784 raft_consensus.cc:2368] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:43.151889 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:43.106364 31056 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:43.137584 30935 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:43.158926 26053 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:43.191944 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:43.110674 30838 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:43.197233 30838 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:43.202628 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:43.159646 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.158330 31000 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.183023 26120 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:43.184464 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.186321 26047 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:43.190104 26197 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:43.194296 30919 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:43.191479 30824 raft_consensus.cc:2764] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:43.118726 30892 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.205024 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.173424 31032 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:43.182093 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:43.236047 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:43.242712 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58366.9us
I20260709 12:25:43.240789 30922 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:43.191814 31056 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.167589 30827 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:25:43.167786 30994 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:25:43.214676 30981 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:43.248932 31079 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:43.256631 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:43.258395 26195 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:43.205230 30992 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:43.248704 30922 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:43.247066 30892 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.247532 31054 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.278312 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:43.257400 31056 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.279502 26127 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:43.289182 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.290472 26197 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:43.260512 31079 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:43.209792 31041 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.303287 30765 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:43.267292 30992 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.286669 30780 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.305303 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.307641 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.315599 26196 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:43.226356 30527 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.317545 31041 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:43.314059 31079 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:43.321727 31079 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:43.323843 31084 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:25:43.286330 31054 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:43.332607 26196 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:43.339677 31033 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:43.349252 26122 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:43.351533 30857 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:43.356369 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.360432 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.361173 26197 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:43.362166 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3019) took 1046 ms. Trace:
I20260709 12:25:43.362520 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:43.305374 31083 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:25:43.380216 26125 rpcz_store.cc:276] 0709 12:25:42.315880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.316124 (+   244us) service_pool.cc:224] Handling call
0709 12:25:42.316406 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.713125 (+396719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.343652 (+630527us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:43.362100 (+ 18448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909421952}
I20260709 12:25:43.380856 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:43.421782 26201 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:43.423905 26202 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 12:25:43.426256 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.433506 26197 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:43.319506 30780 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:43.440984 26196 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:43.419878 30998 raft_consensus.cc:2764] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:43.421141 30860 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.458480 31094 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:43.459077 31094 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:43.459564 31094 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:43.463910 30981 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:43.466809 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 307258us
W20260709 12:25:43.466979 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 269135us
W20260709 12:25:43.487056 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60897.2us
I20260709 12:25:43.487901 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.490662 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.494107 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:43.447506 31051 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:43.496496 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:43.500438 26791 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:43.504896 26783 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:43.464370 31103 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.455237 30860 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:43.540565 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:43.563719 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75904.8us
I20260709 12:25:43.566660 26120 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:25:43.465567 30994 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:43.507668 26784 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:43.554689 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64134us
I20260709 12:25:43.496434 31108 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.575083 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:43.574048 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 330455us
I20260709 12:25:43.573486 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.585471 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.575977 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:43.591602 30075 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:43.569790 30860 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.594177 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:43.594885 26201 raft_consensus.cc:2376] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15. Re-sending same reply.
I20260709 12:25:43.590711 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:43.596663 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:43.569392 30994 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:43.572036 31109 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:43.613399 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.614346 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.509624 31095 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 12:25:43.588735 30922 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:43.590704 30857 raft_consensus.cc:2764] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:43.593202 30979 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:43.606616 31054 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:43.597906 30919 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.643579 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52941.2us
W20260709 12:25:43.651690 31088 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:43.589025 31034 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:43.656869 31082 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:43.659072 30994 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.663161 30780 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:43.602914 30831 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:25:43.665072 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:43.635733 31073 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:43.590763 31051 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:43.607558 31109 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:43.616873 30985 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.599375 31101 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:43.651731 31093 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:43.715180 26201 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:25:43.572278 31103 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:43.720405 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52556.6us
I20260709 12:25:43.669764 30979 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:43.680536 30827 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:43.732651 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 119350us
I20260709 12:25:43.598178 30981 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:43.739360 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.661806 31056 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.747476 31056 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:43.748685 30980 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:43.748579 30979 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:43.752744 31096 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:43.755154 31128 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:43.756912 31128 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:43.665464 30919 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.718083 31073 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:43.744622 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.735410 30994 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.770152 26201 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:43.760561 31073 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.779309 26048 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:43.797326 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52820.4us
I20260709 12:25:43.798558 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.802245 26199 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:43.805485 31127 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:43.765295 31113 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:43.813428 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.817677 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:43.821986 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:43.843701 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:43.846297 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 270319us
I20260709 12:25:43.856849 26018 leader_election.cc:400] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:43.857281 26018 leader_election.cc:403] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:43.810257 31101 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.805751 30992 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:43.806425 31127 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.855149 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:43.868649 26199 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.842064 30827 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:43.796329 30892 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.860833 31124 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:43.871496 31090 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
I20260709 12:25:43.863719 31114 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:43.882982 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:43.849906 31128 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:25:43.886941 26198 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:43.865024 30992 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:43.914449 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:43.841562 31113 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:43.889904 31032 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.005s
I20260709 12:25:43.874315 31127 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:43.937970 31118 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:25:43.941172 31129 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:25:43.909427 31101 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.944900 31072 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.739s	user 0.012s	sys 0.000s
I20260709 12:25:43.945021 30892 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:43.931661 31125 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12.
I20260709 12:25:43.931137 31102 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:43.932062 31121 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.945423 30992 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:43.948058 31000 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.698s	user 0.008s	sys 0.003s
I20260709 12:25:43.958940 30919 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:43.959069 31139 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:43.914709 30527 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:25:43.977653 31072 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.001714 31113 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:43.959559 31101 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.002537 30992 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.003924 31101 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:43.992257 31137 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.004720 31137 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.006909 31137 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.007421 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:43.961778 31108 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:43.954077 31032 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.009471 31127 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.010680 31032 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:44.011443 26785 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:43.975826 31103 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:43.987497 31000 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:43.991642 31102 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:25:43.997457 31122 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.991907 31139 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:43.999284 31080 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.850s	user 0.004s	sys 0.000s
W20260709 12:25:44.000448 31039 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.873s	user 0.000s	sys 0.003s
I20260709 12:25:43.991870 30943 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:44.012989 26790 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:44.022632 26199 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:44.020098 30919 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 12:25:44.025192 31041 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.000s
I20260709 12:25:44.025111 31000 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:44.016201 31094 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.002s
I20260709 12:25:44.026188 31094 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.026659 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3029) took 1258 ms. Trace:
I20260709 12:25:44.015388 31102 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:44.027127 26129 rpcz_store.cc:276] 0709 12:25:42.768172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.822502 (+ 54330us) service_pool.cc:224] Handling call
0709 12:25:42.846655 (+ 24153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.846975 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.847003 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.854892 (+  7889us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.033835 (+178943us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:43.033872 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.033939 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.998941 (+965002us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b240028f1a8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:25:44.026360 (+ 27419us) raft_consensus.cc:1685] finished
0709 12:25:44.026388 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.026593 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29738,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":6651,"spinlock_wait_cycles":9784704,"thread_start_us":160072,"threads_started":1,"wal-append.queue_time_us":215035}
I20260709 12:25:44.028621 31094 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.028927 31139 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.015695 30943 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.029693 30943 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.015199 31122 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:44.023053 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 170562us
I20260709 12:25:44.014344 31103 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.033231 31054 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.032624 31105 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.037704 31054 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.038005 31105 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.041260 31105 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:44.043309 31079 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.721s	user 0.006s	sys 0.007s
W20260709 12:25:44.061591 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.068172 30918 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:44.069214 31134 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.064781 31054 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:44.069962 31134 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.070930 31141 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.073143 31134 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.078709 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.079542 26785 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.067667 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1759) took 1021 ms. Trace:
I20260709 12:25:44.081099 26789 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:44.081598 26194 rpcz_store.cc:276] 0709 12:25:43.046223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.046355 (+   132us) service_pool.cc:224] Handling call
0709 12:25:43.077412 (+ 31057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.077660 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:43.124457 (+ 46797us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.002501 (+878044us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b1800286120. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:25:44.067239 (+ 64738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.067307 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:44.067377 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.067461 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.067597 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46580,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":3501,"raft.queue_time_us":2993,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":107,"spinlock_wait_cycles":60181888,"thread_start_us":64427,"threads_started":1}
I20260709 12:25:44.071512 31141 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.035738 31103 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.104869 31124 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.107822 31103 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.108512 26784 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:25:44.116214 26788 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
W20260709 12:25:44.116690 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 204755us
I20260709 12:25:44.122165 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.123827 26046 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:44.091696 30860 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.110288 31122 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.126688 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.129218 26052 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 12:25:44.116677 30985 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:44.126354 30983 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:44.137449 31129 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.139107 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.131157 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:44.140730 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.143543 30860 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.125095 31079 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:44.145288 31079 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.149720 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.154318 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:44.157555 26786 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:44.160825 26785 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:44.135324 30943 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.167052 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.166077 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.183864 30959 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:44.184361 30959 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:44.185218 30959 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.186658 30959 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.173674 30983 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:25:44.188503 31146 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:44.192052 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.164990 30860 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:44.151526 31079 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.193980 26786 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:44.172972 31129 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:44.206519 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:44.207878 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.209708 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.209612 26786 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:44.211421 26786 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:44.211447 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.213457 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:44.216368 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.216802 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:44.219358 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.221940 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:44.222788 26047 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.221707 26052 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:44.220846 26054 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:25:44.224898 30943 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:44.226040 26194 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:44.226696 26783 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:44.227267 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:44.228451 30791 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.228770 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:44.229060 30791 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:44.229880 30918 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:44.230054 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:44.230156 30791 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.223328 31009 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:44.231618 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:44.231833 31009 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:44.233117 31009 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.233801 30994 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:44.234429 31109 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.235671 31109 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:44.236392 31122 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:44.222901 30986 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:44.237012 31122 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.240520 31034 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:44.241101 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:44.241405 26047 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:44.241127 26786 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:44.242237 26786 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:44.242810 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.242969 26786 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.243427 26785 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:44.244096 26786 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.246344 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:44.246963 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:44.219918 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.227998 26194 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:44.239908 31122 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.250998 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:44.246029 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:44.261745 26200 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:44.261765 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.262940 26048 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:44.252725 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:44.273917 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:44.281032 30986 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:44.281332 26791 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:44.266242 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.264818 26196 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:44.208767 31129 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.286046 31129 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:44.286620 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.289784 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.197304 31000 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:25:44.261849 31103 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:44.293305 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.252250 30918 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.197744 31146 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:25:44.299736 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:44.310736 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.301854 26786 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:44.302785 31005 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:44.313562 31005 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.316165 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:44.294888 30943 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:44.315492 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 49358.3us
I20260709 12:25:44.306529 31009 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.315033 30831 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:25:44.288508 31108 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:25:44.335582 26785 pending_rounds.cc:85] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 11
I20260709 12:25:44.329267 30943 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.340277 30943 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.340700 31051 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:44.339980 26785 pending_rounds.cc:107] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:25:44.343327 26785 raft_consensus.cc:2889] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:44.298132 30527 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.224227 31073 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:44.306615 31143 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 12:25:44.347074 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48368.7us
I20260709 12:25:44.329187 31149 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:44.341217 31051 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.306193 30979 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:25:44.348381 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:44.338665 31147 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:44.338930 31053 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.349351 31009 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.358738 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47108.9us
I20260709 12:25:44.348889 31122 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.334231 31131 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:25:44.365023 31086 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.098s	user 0.001s	sys 0.003s
I20260709 12:25:44.359913 30527 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:44.358314 31149 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:44.362666 31103 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:44.358886 31141 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.363795 30986 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:25:44.376787 31009 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:44.396832 31142 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:25:44.397181 31123 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
W20260709 12:25:44.398465 31148 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:44.364452 31114 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:25:44.389876 31108 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:44.406185 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44426.5us
I20260709 12:25:44.387131 31103 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.387416 31149 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:44.374890 31053 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.383427 31122 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.419373 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:44.441327 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3033) took 1533 ms. Trace:
I20260709 12:25:44.414398 31114 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:44.441767 26128 rpcz_store.cc:276] 0709 12:25:42.908018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.978923 (+ 70905us) service_pool.cc:224] Handling call
0709 12:25:42.979313 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.979735 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.979771 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.980305 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.154830 (+174525us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:43.154959 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.155171 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.245726 (+1090555us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:44.433905 (+188179us) raft_consensus.cc:1685] finished
0709 12:25:44.433927 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.441256 (+  7329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2045,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7792,"spinlock_wait_cycles":2697600,"thread_start_us":173928,"threads_started":1,"wal-append.queue_time_us":280332}
I20260709 12:25:44.416802 31103 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.414176 31108 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:44.449913 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.450598 26045 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:44.438575 30986 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:44.472936 31155 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:44.473479 31155 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:44.474407 31155 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.431169 31146 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:44.431819 31141 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:25:44.461077 31131 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:44.488739 31147 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.506865 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.507408 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:44.507778 26047 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:25:44.507963 26052 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.508723 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:44.509732 26054 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:44.521831 31158 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:44.522091 31091 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.164s	user 0.007s	sys 0.005s
I20260709 12:25:44.501379 31147 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.522562 31158 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:44.523689 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 853) took 1377 ms. Trace:
W20260709 12:25:44.524353 31145 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:25:44.524521 31147 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:44.526198 31024 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:44.532838 31051 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.536135 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.537065 26791 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:44.536403 31122 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.538225 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.538269 31130 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:25:44.541646 26791 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:44.524154 26787 rpcz_store.cc:276] 0709 12:25:43.146472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.147754 (+  1282us) service_pool.cc:224] Handling call
0709 12:25:43.148086 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.148477 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.148504 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.148984 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.270434 (+121450us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:43.270491 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.270603 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.488635 (+1218032us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:44.523402 (+ 34767us) raft_consensus.cc:1685] finished
0709 12:25:44.523426 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.523631 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1161,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":88700800,"thread_start_us":121043,"threads_started":1,"wal-append.queue_time_us":187619}
I20260709 12:25:44.481498 31005 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:44.578249 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.578914 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.591101 26168 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:25:44.595245 26168 leader_election.cc:403] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:44.447335 30527 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.494983 31131 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:44.481940 30986 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.523540 31158 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.609567 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.611747 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.616928 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.617662 26129 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:44.617580 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.590132 31160 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.623737 26127 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:44.621526 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.626468 26125 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:44.627399 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.628121 26120 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:44.603433 31073 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.630380 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.661469 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:44.662221 26125 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.590507 31123 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:25:44.602471 31159 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:44.613842 31028 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:44.673120 26129 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:44.615984 31024 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:25:44.597833 31005 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:25:44.593681 31108 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.686626 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.629822 31166 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:44.690971 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.693040 31009 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:44.693717 31161 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:44.695753 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:44.695919 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.697412 26785 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:25:44.698668 30860 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 12:25:44.698820 26785 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:44.699118 30860 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:44.699290 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1764) took 1636 ms. Trace:
I20260709 12:25:44.699942 26785 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:44.701125 31009 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:44.701833 30943 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.701740 31009 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.704989 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1760) took 1658 ms. Trace:
I20260709 12:25:44.705206 31166 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.705497 26195 rpcz_store.cc:276] 0709 12:25:43.046575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.046715 (+   140us) service_pool.cc:224] Handling call
0709 12:25:43.258142 (+211427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.261991 (+  3849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.704757 (+1442766us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400398850. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:44.704917 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413843840}
I20260709 12:25:44.707695 31166 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:44.708410 31087 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.372s	user 0.001s	sys 0.005s
I20260709 12:25:44.693747 30918 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.705546 31009 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.699944 26198 rpcz_store.cc:276] 0709 12:25:43.063188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.063346 (+   158us) service_pool.cc:224] Handling call
0709 12:25:43.063766 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.916870 (+853104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.699019 (+782149us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b5400163d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:44.699226 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892649344}
I20260709 12:25:44.672065 31028 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:44.704082 30943 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.719230 26782 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:44.723359 26120 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:44.718287 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 852) took 1572 ms. Trace:
I20260709 12:25:44.716169 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:44.725801 26790 rpcz_store.cc:276] 0709 12:25:43.146174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.158341 (+ 12167us) service_pool.cc:224] Handling call
0709 12:25:44.718042 (+1559701us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b5400355a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:44.718206 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745724800}
I20260709 12:25:44.712572 31096 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:44.727419 31096 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:44.713222 30918 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:44.728402 26120 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:44.727655 31028 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.729290 31028 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:44.728358 31096 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:44.730541 31109 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:44.730965 31109 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.631807 31160 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:44.712177 30959 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.004s
W20260709 12:25:44.746572 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 158561us
I20260709 12:25:44.754603 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.755550 26783 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:44.794389 26054 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:44.731352 31109 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:44.731154 31096 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.799178 26047 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:44.808929 26790 raft_consensus.cc:1217] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[8.8-12.11]   Dedup: 9.9->[10.10-12.11]
I20260709 12:25:44.809516 26790 pending_rounds.cc:85] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 9
I20260709 12:25:44.803786 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.738885 31005 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.809784 26790 pending_rounds.cc:107] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:44.810276 26790 raft_consensus.cc:2889] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:44.810704 26051 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:44.630013 31131 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.816256 26054 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:25:44.819423 26046 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:44.820600 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:44.740023 31147 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:25:44.823793 26046 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:44.827207 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:44.816066 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 225084us
I20260709 12:25:44.830082 26049 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:44.830358 31160 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.831459 30986 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:44.847410 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:44.817956 31005 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.737771 31159 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:44.872758 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.849407 31131 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.879146 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.880815 26049 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:44.879307 26126 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:44.879361 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:44.887704 26122 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:44.884769 30986 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:44.890784 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:44.894347 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.895295 26052 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:44.898978 26051 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:25:44.899322 26051 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 8
I20260709 12:25:44.899551 26051 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.9
I20260709 12:25:44.918299 26051 raft_consensus.cc:2917] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Aborting config change with OpId 11.9: Aborted: Op aborted by new leader
I20260709 12:25:44.918766 26051 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:44.903434 31005 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:44.924403 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.925215 26124 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:44.904965 30986 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:44.924788 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 327735us
I20260709 12:25:44.958947 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:44.962100 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:44.963001 26124 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.973752 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:44.975925 26125 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:44.936213 31005 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:45.000202 31123 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:45.001992 31123 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:45.007406 31123 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:45.009233 31000 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:45.009878 31000 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:45.010983 31000 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.013152 31000 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:45.024729 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.038947 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.039909 26121 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
I20260709 12:25:45.041203 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.041394 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:45.042825 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.043614 26121 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:45.044648 30943 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:45.043666 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:45.049599 30943 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:45.050426 30943 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:45.024576 30986 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:45.058586 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.051456 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:45.062428 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.064146 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:45.066690 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.067418 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 480557us
I20260709 12:25:45.071338 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:45.072175 26052 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
W20260709 12:25:45.036068 31149 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.597s	user 0.006s	sys 0.003s
I20260709 12:25:45.074750 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:45.079041 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.080543 26125 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:45.108125 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:45.109033 26049 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:45.075537 30935 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:45.097669 31129 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.811s	user 0.014s	sys 0.000s
I20260709 12:25:45.115087 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.078997 31149 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.133827 26049 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:45.135875 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:45.137877 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.133956 31159 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:45.136961 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:45.155409 31026 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:45.156517 31034 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:45.155987 31026 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.157859 31026 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:45.156847 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.123569 31129 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.163041 31129 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.163843 26791 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:45.114542 30935 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.130714 31179 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:25:45.169253 31103 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.004s
I20260709 12:25:45.169386 30979 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.179018 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.157101 31034 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:45.171326 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:45.180660 26123 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 10.
I20260709 12:25:45.187681 26128 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:45.194591 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.201740 26126 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:45.202246 26126 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:45.202713 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.203060 26126 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.209105 26126 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:45.179906 31179 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.212877 31179 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:45.218478 31165 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:45.219328 31165 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.219856 31165 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:45.227718 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.180797 30979 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:45.231966 31189 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:25:45.232087 30979 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.234258 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.199035 30922 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:45.179548 31103 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.238399 31103 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:45.239602 31028 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.002s
I20260709 12:25:45.239980 30922 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.242367 30922 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.243598 30935 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.244549 31149 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.250931 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:45.257076 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.259541 26126 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:45.266861 26787 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:25:45.265967 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 191354us
W20260709 12:25:45.268904 26019 connection.cc:256] RPC call timeout handler was delayed by -1.2079717330002495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 12:25:45.270526 26019 connection.cc:256] RPC call timeout handler was delayed by -1.0729717330000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 12:25:45.271893 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.273983 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:45.275031 26200 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:45.274173 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:45.279305 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:45.280217 26197 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:45.208604 31109 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.285204 30857 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.285811 30857 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.287164 31109 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.287899 30992 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:45.279456 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:45.288033 26783 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:25:45.297358 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:45.307111 26048 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:45.303875 26194 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:25:45.300415 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.311286 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.368753 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.381434 26197 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:45.381886 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.383692 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.414278 26169 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 8 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 8. Current term is 9.
W20260709 12:25:45.415339 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108092us
I20260709 12:25:45.416697 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:45.417063 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.386412 26123 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:45.288146 30857 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:45.381418 31109 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.301756 31155 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:45.459447 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 184156us
I20260709 12:25:45.413321 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:45.493108 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.508035 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 91413.3us
I20260709 12:25:45.288509 31159 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:45.509057 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:45.521361 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 108298us
I20260709 12:25:45.522519 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:45.523244 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:45.534478 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.535549 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:45.500177 31200 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.552393 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 204772us
I20260709 12:25:45.553877 26121 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:45.384347 30992 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.524646 30986 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.514803 31008 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:45.524924 31114 raft_consensus.cc:2764] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:25:45.581446 26121 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:25:45.541069 31168 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:45.584960 26048 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:45.585502 30986 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.550047 30780 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:45.578737 30838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.228s	user 0.025s	sys 0.029s
I20260709 12:25:45.554541 30857 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:45.554653 31199 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:45.537257 31159 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.581341 31181 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.591825 26121 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:45.589252 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 624) took 1300 ms (client timeout 1695 ms). Trace:
W20260709 12:25:45.592967 30838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.243s	user 0.026s	sys 0.029s
I20260709 12:25:45.578406 31008 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:45.601214 31008 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.602530 31008 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:45.592708 30780 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.604298 30780 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:45.605692 30986 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.605414 31034 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.606292 30986 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:45.607697 31034 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.608342 30857 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:45.609340 30857 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.611339 31053 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.613934 30780 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.597661 31199 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.617311 31199 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:45.620651 31181 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.622467 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1053) took 2123 ms. Trace:
I20260709 12:25:45.623263 30780 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.627584 30992 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:45.506713 31109 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.630739 31109 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.631176 31159 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.540558 31160 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:25:45.574918 31204 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:45.586905 31104 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:45.634387 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.654346 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.622931 26782 rpcz_store.cc:276] 0709 12:25:43.498670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.504450 (+  5780us) service_pool.cc:224] Handling call
0709 12:25:44.718950 (+1214500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.724762 (+  5812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.622229 (+897467us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400355a50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:45.622375 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2644327168}
W20260709 12:25:45.599689 26048 rpcz_store.cc:269] 0709 12:25:44.289336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.289664 (+   328us) service_pool.cc:224] Handling call
0709 12:25:45.297150 (+1007486us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:25:45.589139 (+291989us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4697472}
I20260709 12:25:45.646188 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:45.585507 31082 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.844s	user 0.007s	sys 0.025s
I20260709 12:25:45.657073 26121 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:45.643764 26200 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:25:45.657343 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.658289 26123 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:45.623903 31181 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:45.653620 31206 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:25:45.587678 31200 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:45.659991 31082 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.922s	user 0.008s	sys 0.025s
I20260709 12:25:45.654142 31104 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:45.687638 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 177013us
W20260709 12:25:45.691493 26789 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 850) took 2542 ms (client timeout 1781 ms). Trace:
W20260709 12:25:45.691524 26788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 840) took 3237 ms (client timeout 1716 ms). Trace:
W20260709 12:25:45.691972 26789 rpcz_store.cc:269] 0709 12:25:43.145589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.148328 (+  2739us) service_pool.cc:224] Handling call
0709 12:25:44.007336 (+859008us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280f8c 00007f5b71280ef6 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:25:45.688227 (+1680891us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54004501d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:45.688364 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177146496}
W20260709 12:25:45.692116 26788 rpcz_store.cc:269] 0709 12:25:42.451802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.515216 (+ 63414us) service_pool.cc:224] Handling call
0709 12:25:45.688408 (+3173192us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54003101d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:45.688523 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3678234496}
I20260709 12:25:45.655673 31204 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.689064 31104 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:45.680366 31082 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:45.695921 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 158080us
I20260709 12:25:45.692425 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:45.702672 31192 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:25:45.706596 31165 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:45.709012 31026 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 12:25:45.709511 31026 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:45.711269 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.712613 31026 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:45.716720 26789 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:45.736804 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:45.735723 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 197972us
I20260709 12:25:45.741503 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.742234 26123 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:25:45.743546 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 208121us
I20260709 12:25:45.742753 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:45.733800 31109 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.748903 31212 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.742667 31160 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:25:45.748814 31204 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:45.762341 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.783779 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:45.789268 26051 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:45.831215 26048 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:45.786440 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:45.760614 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:45.833113 26123 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:45.789978 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:45.787586 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:45.829974 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:45.779456 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1763) took 2721 ms (client timeout 2012 ms). Trace:
W20260709 12:25:45.835543 26203 rpcz_store.cc:269] 0709 12:25:43.057910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.059345 (+  1435us) service_pool.cc:224] Handling call
0709 12:25:45.779071 (+2719726us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:45.779267 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5731651456}
I20260709 12:25:45.791394 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:45.864543 26200 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:45.864737 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:45.866864 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 115211us
I20260709 12:25:45.751951 31111 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:45.811735 31098 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.803522 30791 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.760309 31200 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.834075 31034 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:45.869848 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.873728 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:45.727228 31199 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.876019 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.876255 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.905417 26203 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:45.905580 26199 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:45.875978 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.877074 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.811211 31102 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:45.878368 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:45.750258 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:45.878796 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.940403 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.785571 31120 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.766462 31125 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:45.875907 31165 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:45.880466 31026 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.753440 31109 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.830294 31008 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.892557 26127 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:45.892629 26048 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:45.893492 31213 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:45.894888 31217 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.895372 31111 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:45.902505 30918 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.896317 31056 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:45.899299 31104 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:45.910351 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.954030 30824 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:45.954056 31200 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:45.759531 31082 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.944211 31125 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:45.821012 31032 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:25:45.941246 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:45.942539 26196 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:25:45.998301 31032 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.945031 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 199508us
I20260709 12:25:46.003960 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:45.946336 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68047.5us
I20260709 12:25:45.870055 31160 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:45.868739 31054 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.870553 31181 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:45.759977 31212 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:45.950248 31102 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.966847 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:45.896396 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:45.868959 31204 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.999040 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.998431 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:45.906988 31098 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.906729 30791 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.006448 30824 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:46.007149 30838 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:46.029343 30857 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:25:46.009017 26050 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:45.947336 31034 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.047501 31034 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:45.966058 26201 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:45.957630 31026 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.955029 31120 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.998211 31129 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.014706 31200 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:46.026111 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 129807us
I20260709 12:25:46.028755 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:46.037062 30824 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:46.037627 31098 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.040271 31198 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:25:46.048907 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3042) took 1158 ms. Trace:
I20260709 12:25:46.050479 26124 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:45.964288 31008 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.013191 31160 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:45.971539 31217 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:45.973578 30918 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.048579 31099 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:46.063485 26126 rpcz_store.cc:276] 0709 12:25:44.890254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.892144 (+  1890us) service_pool.cc:224] Handling call
0709 12:25:44.899321 (+  7177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.260531 (+361210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.048672 (+788141us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54001f95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:46.048849 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655981952}
I20260709 12:25:46.040073 31108 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.051024 31034 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.013438 31032 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.015817 31028 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:25:46.020819 31054 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.034379 31204 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.024515 31195 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:25:46.024735 31181 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.075081 26782 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:46.068804 31160 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.088017 26785 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:46.073907 26123 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:46.071843 30780 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:46.075640 31099 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.104192 26202 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:46.064208 31129 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.079651 31200 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:46.131122 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.098387 30780 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.053462 31111 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.075577 31214 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:25:46.081529 31034 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.114701 31099 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:46.134544 31155 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:46.159526 31155 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.160989 31204 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:46.162889 31155 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.163663 31198 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:46.164295 31198 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.167016 31198 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.149482 31032 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:46.150111 31219 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:46.152016 31109 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.153324 31212 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.172449 31034 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.169075 30918 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:46.092356 31215 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:25:46.103870 31103 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:25:46.109460 31159 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:25:46.108894 31122 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:25:46.067600 31125 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:46.215436 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.219548 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2263) took 1616 ms. Trace:
I20260709 12:25:46.220999 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:46.220342 26128 rpcz_store.cc:276] 0709 12:25:44.603025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.604347 (+  1322us) service_pool.cc:224] Handling call
0709 12:25:44.604685 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.188650 (+583965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.219316 (+1030666us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:46.219487 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1340008320}
I20260709 12:25:46.138523 30780 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:46.210089 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 199130us
I20260709 12:25:46.134737 31199 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:46.215332 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1065) took 1836 ms. Trace:
I20260709 12:25:46.225962 26791 rpcz_store.cc:276] 0709 12:25:44.378572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.586212 (+207640us) service_pool.cc:224] Handling call
0709 12:25:44.589507 (+  3295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.167377 (+577870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.214992 (+1047615us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:46.215239 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320207616}
I20260709 12:25:46.133311 31224 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:46.227989 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:46.228299 31224 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.231202 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:46.220337 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.235561 26757 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:46.195593 31128 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.239190 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:46.243150 26791 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:46.243741 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:46.208717 31000 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.210266 31102 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.213927 31212 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:46.234292 31104 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.246500 31224 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.247005 31109 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:46.247174 31032 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.254565 31033 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:46.285537 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 188363us
W20260709 12:25:46.286134 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 190601us
I20260709 12:25:46.287642 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.303736 26791 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:46.256335 31212 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:46.259173 30791 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.247694 31128 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.352603 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:46.262812 31125 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.256516 31155 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 12:25:46.322511 31008 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:46.254832 31000 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.362727 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:46.361053 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 814) took 1673 ms (client timeout 2168 ms). Trace:
W20260709 12:25:46.372102 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 140945us
I20260709 12:25:46.255415 31159 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 12:25:46.363343 31120 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.321189 31109 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:46.351526 31166 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.345192 31224 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:46.244911 31200 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:46.382476 26127 rpcz_store.cc:269] 0709 12:25:44.688017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.688166 (+   149us) service_pool.cc:224] Handling call
0709 12:25:46.360845 (+1672679us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:46.360982 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749479296}
I20260709 12:25:46.385964 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:46.386570 31195 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:46.406286 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43655.8us
I20260709 12:25:46.354784 31227 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.324105 31225 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 12:25:46.373715 30922 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:46.394189 31214 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:46.427852 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.404780 31109 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.403940 31217 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.411450 31120 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.445899 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:46.448473 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 89310.5us
W20260709 12:25:46.420100 31221 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:25:46.409356 31054 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.417191 31181 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.392815 31204 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:25:46.390389 31008 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:25:46.451565 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:46.453882 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.455444 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:46.456990 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2272) took 1838 ms (client timeout 1651 ms). Trace:
I20260709 12:25:46.457084 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:46.412659 31230 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:46.465971 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:46.467243 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:46.473230 26122 rpcz_store.cc:269] 0709 12:25:44.619030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.627186 (+  8156us) service_pool.cc:224] Handling call
0709 12:25:46.456783 (+1829597us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:46.456914 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597211520}
I20260709 12:25:46.472043 26128 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:46.479068 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.482535 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.412727 31166 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.485030 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.422040 31231 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:25:46.419066 31224 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.497041 26051 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:46.519153 31224 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.450254 31214 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.521550 31214 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.522850 31231 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:46.501034 26048 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:46.501999 31233 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:46.488463 31181 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:25:46.509228 31000 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.417889 31212 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.512980 31128 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.480332 31229 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.514973 31227 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:46.515482 31204 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:46.516950 31155 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.525568 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70281.3us
I20260709 12:25:46.540679 26051 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:46.528721 26128 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:46.475886 31230 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.474987 31217 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:46.476086 31129 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.532568 26121 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:25:46.576613 26121 pending_rounds.cc:85] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 10
I20260709 12:25:46.576649 26045 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:46.577133 26121 pending_rounds.cc:107] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:25:46.577404 26121 raft_consensus.cc:2889] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:46.343540 31082 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 12:25:46.472777 31054 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:25:46.583570 31000 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:46.584110 31000 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:46.585210 31000 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.542141 26048 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 12:25:46.543008 31156 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:46.543659 31227 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:46.457751 31120 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.548810 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.528262 31214 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.530234 31231 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:46.553753 26049 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: [10.8-11.9]
I20260709 12:25:46.531322 31199 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:46.558495 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.591325 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:46.604110 26047 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:46.592636 26051 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:46.606309 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.606987 26051 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
W20260709 12:25:46.620307 31222 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.407s	user 0.000s	sys 0.001s
I20260709 12:25:46.623359 31095 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:46.623879 31095 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:46.625749 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3038) took 1829 ms. Trace:
I20260709 12:25:46.626184 26120 rpcz_store.cc:276] 0709 12:25:44.795956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.796205 (+   249us) service_pool.cc:224] Handling call
0709 12:25:46.457469 (+1661264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.457676 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.457704 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.457757 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:46.457793 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.457857 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.625521 (+167664us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540013e550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:46.625678 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3819725696}
I20260709 12:25:46.627761 30918 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:46.629561 31120 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.544090 31155 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.633713 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.591845 31156 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:46.544767 31229 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.575079 31230 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.599700 31231 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:46.648953 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 385049us
I20260709 12:25:46.593403 31217 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.632717 31056 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.628178 30918 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:46.649457 31034 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.697260 26782 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:46.699147 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.704998 26789 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
W20260709 12:25:46.700156 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.694294 31221 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:25:46.635816 31199 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:46.603525 31214 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.697793 30959 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:46.746004 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.746588 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1074) took 1764 ms. Trace:
I20260709 12:25:46.747116 26783 rpcz_store.cc:276] 0709 12:25:44.982623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.984526 (+  1903us) service_pool.cc:224] Handling call
0709 12:25:44.984937 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.306693 (+321756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.746272 (+1439579us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54004f79d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:46.746525 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696420736}
I20260709 12:25:46.704015 31056 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.761294 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:46.762009 26051 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:46.769392 31217 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.770066 31166 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.773511 31229 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.775846 31214 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:46.778053 31099 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.001s
I20260709 12:25:46.778652 31099 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.752111 31156 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.781504 31156 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:46.783381 31160 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.682s	user 0.008s	sys 0.004s
I20260709 12:25:46.783994 31160 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.785495 26050 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:46.785681 31199 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.726104 31034 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.790192 31056 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:46.707093 31095 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.791069 31034 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:46.791388 31095 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:46.793352 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.800702 31234 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:25:46.751462 31231 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.805217 31026 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:46.820785 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:46.821597 26202 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.822094 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:46.822628 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.008s	sys 0.016s
I20260709 12:25:46.822849 26201 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:46.823073 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.008s	sys 0.016s
W20260709 12:25:46.823561 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 570826us
I20260709 12:25:46.824808 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.825543 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:46.812144 31224 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.815541 31112 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.745633 30959 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:46.838647 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:46.839496 26051 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:46.839656 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.840368 26194 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:46.842545 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.846306 26201 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:25:46.848941 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.825148 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:46.830638 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 305584us
I20260709 12:25:46.851159 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:46.852046 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.852442 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:46.852773 26197 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:25:46.836998 30780 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.594s	user 0.013s	sys 0.000s
I20260709 12:25:46.855562 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.856952 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:46.818639 31034 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:46.864605 26120 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:46.863780 26051 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:46.868448 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.869583 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.869715 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.868517 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.870296 26782 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
I20260709 12:25:46.870613 26789 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:46.870527 26128 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:46.872670 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.873039 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.837924 31026 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.866904 31032 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:46.883046 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.884938 26195 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
I20260709 12:25:46.876725 31243 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.859823 31231 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.876554 31244 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.888268 31244 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.889161 31099 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.890719 31244 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:46.891047 31108 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.810s	user 0.013s	sys 0.001s
I20260709 12:25:46.751087 31227 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:46.900311 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.901342 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.902482 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:46.902881 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:46.905860 31128 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:46.906456 31128 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:46.909588 31241 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:46.910001 31160 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.910259 31241 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.910846 31241 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:46.914229 31125 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 12:25:46.884610 31033 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:46.916797 31033 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:46.916391 31109 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.917783 31033 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.919788 31109 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:46.867172 31166 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.877029 26787 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEARNER]: Advancing to term 14
I20260709 12:25:46.877920 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:46.878320 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.879901 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.860309 31225 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:46.932539 26786 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:46.887122 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:46.921399 26048 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:46.876667 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:46.938871 26202 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:25:46.943504 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.841642 31224 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.881332 31032 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:46.948136 31225 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.849990 31112 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.915800 31230 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:46.869728 30780 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:46.997658 26167 connection.cc:256] RPC call timeout handler was delayed by -1.3161772530002054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
I20260709 12:25:46.884977 31026 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.880107 31034 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.947818 26050 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:25:47.019809 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.907745 31128 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:46.969254 31155 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.039935 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.079208 26051 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:46.880816 30791 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:46.946583 26129 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:25:47.043108 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.886520 30959 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.058027 31054 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.058712 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:47.063932 31195 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
I20260709 12:25:47.060070 26202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:25:47.090662 26202 raft_consensus.cc:606] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.091192 26202 raft_consensus.cc:614] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:47.018250 31251 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.062631 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.068979 26120 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:47.072073 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:47.074822 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 152478us
I20260709 12:25:46.928536 31243 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.059453 31056 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.056309 31032 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.038676 31166 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.056496 31225 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.087649 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.130693 26195 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:47.087707 26053 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:25:47.093953 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.095994 31199 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.124248 31056 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.123106 30791 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.123267 31195 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:47.082265 31237 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.122373 31248 consensus_queue.cc:1175] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a981c9a7972c4e99a0903924c64fbb5c" responder_term: 15 status { last_received { term: 12 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.10-14.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:47.086439 26049 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:47.159088 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:47.168952 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.095433 31054 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.128491 30959 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:47.165337 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 262492us
W20260709 12:25:47.167376 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73548.7us
I20260709 12:25:47.245920 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.229269 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:25:47.164387 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 92390.9us
I20260709 12:25:47.170102 26045 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:47.248095 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.249563 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:47.248437 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:47.259286 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.232803 31109 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:47.176409 31238 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.267313 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.274519 26049 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:47.266443 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.177285 31195 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:47.173446 31224 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.270321 30780 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.271552 31026 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.174243 31230 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.280395 26120 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:47.297004 26783 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:47.298318 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:47.159252 31251 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.273119 31159 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.316929 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.317582 26049 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:47.311455 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.318220 26120 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:47.319303 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.319519 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.209846 31237 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.312626 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:47.324796 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:47.291214 30824 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:47.322470 26783 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:47.326491 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.294847 31224 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:47.329900 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:47.330495 26120 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:25:47.328362 31028 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:47.349538 26786 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:47.352764 26786 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:47.352475 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.355846 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.357035 26049 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:47.354420 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:47.360332 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.316114 30831 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.315333 31128 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.323184 31159 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:47.332897 31262 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:47.309206 31195 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:47.332036 31218 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.302s	user 0.008s	sys 0.004s
I20260709 12:25:47.327630 31008 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:47.371436 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.308877 31244 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:25:47.334398 31251 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:47.372391 30998 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:47.377960 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.335337 31223 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:47.362299 31123 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.402230 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.427266 26049 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:47.375185 31159 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:47.408646 26020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.651s (SENT) [suppressed 18 similar messages]
I20260709 12:25:47.358219 30838 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:47.435395 30838 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:47.436000 30838 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:47.438647 31258 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:25:47.439615 31260 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 12:25:47.446619 31255 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:25:47.378353 31261 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:25:47.448724 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:47.449400 31056 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:47.455570 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.457326 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.457950 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:47.458647 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:47.412765 30998 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.460793 30998 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.468782 31195 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.472536 31266 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.473179 31266 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.473661 31266 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:47.418763 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.363529 31112 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.477708 31112 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:47.478253 31112 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.459859 31034 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:47.449450 26125 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.458930 26122 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:47.491545 26122 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:47.379942 31128 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.493454 26124 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:47.493656 31112 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.494644 31128 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.375402 30831 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.498068 26120 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:47.498979 30831 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.501655 31212 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.505208 30838 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:47.378566 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 130189us
I20260709 12:25:47.508975 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.509778 31224 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:47.510282 31224 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:47.511195 31231 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.000s
I20260709 12:25:47.511108 31224 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.422462 31223 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.512426 31223 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.512998 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:47.424178 31123 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.514467 31123 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:47.516166 31243 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.518872 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:47.520937 26129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 822) took 2347 ms (client timeout 1719 ms). Trace:
W20260709 12:25:47.521423 26129 rpcz_store.cc:269] 0709 12:25:45.173060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.233866 (+ 60806us) service_pool.cc:224] Handling call
0709 12:25:47.520722 (+2286856us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:47.520839 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3810556288}
I20260709 12:25:47.523175 26129 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:25:47.523538 26129 pending_rounds.cc:85] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 9
I20260709 12:25:47.524453 26197 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:47.524530 31223 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.527076 31223 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.528955 26125 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
W20260709 12:25:47.529804 31254 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 12:25:47.531102 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2291) took 1062 ms. Trace:
W20260709 12:25:47.533284 31256 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.292s	user 0.000s	sys 0.000s
I20260709 12:25:47.533443 31229 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.534078 31229 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:47.534870 31252 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
I20260709 12:25:47.537487 31229 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:47.538508 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 129904us
I20260709 12:25:47.542405 31237 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:47.547416 31241 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.636s	user 0.005s	sys 0.005s
I20260709 12:25:47.548962 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.549139 31082 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:47.549566 31156 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.006s
I20260709 12:25:47.550019 31229 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.550241 26121 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:47.553956 30838 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.559113 30791 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.531518 26126 rpcz_store.cc:276] 0709 12:25:46.469030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.483609 (+ 14579us) service_pool.cc:224] Handling call
0709 12:25:46.490172 (+  6563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.651103 (+160931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.651152 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.697637 (+ 46485us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.026672 (+329035us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:47.026737 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.026833 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.529177 (+502344us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:47.530813 (+  1636us) raft_consensus.cc:1685] finished
0709 12:25:47.530839 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.531039 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44179,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7035,"spinlock_wait_cycles":27601792,"thread_start_us":435618,"threads_started":2,"wal-append.queue_time_us":343527}
I20260709 12:25:47.562413 31000 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.562891 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:47.563659 26783 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:47.566782 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.568670 31129 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:47.569123 31129 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.569473 31129 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.573662 31166 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.574120 31229 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:47.576414 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.014s	sys 0.034s
W20260709 12:25:47.576823 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.015s	sys 0.034s
W20260709 12:25:47.577320 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 1.19132e+06us
I20260709 12:25:47.578011 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.523782 26129 pending_rounds.cc:107] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:25:47.578887 26129 raft_consensus.cc:2889] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.549626 31082 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:47.580622 31082 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.581863 31082 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:47.582377 31229 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:47.583402 31099 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:47.583792 31099 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:47.584743 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.584657 31099 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.585381 31166 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.586061 31166 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.590962 31166 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.593024 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:47.598824 26783 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:47.593691 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.580539 26128 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:25:47.600864 31099 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.602829 31230 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:47.603364 31230 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:47.604301 31230 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.605685 31140 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:47.605971 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.542469 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:47.613376 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70985.1us
I20260709 12:25:47.614312 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.615666 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:47.459990 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:47.623945 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.628777 31165 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.631728 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.636865 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53857.6us
W20260709 12:25:47.636884 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58925.4us
I20260709 12:25:47.631657 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.645653 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:47.647351 26050 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:25:47.648635 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.649292 26050 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:47.650218 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.651959 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.653021 26047 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:47.654237 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.655735 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.658648 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:47.659426 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.657678 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:47.660337 26045 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:47.664469 31165 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.658866 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:47.667016 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.668097 26123 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:25:47.668424 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:47.670533 26046 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:47.547957 31241 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:47.671275 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.674746 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.678447 31112 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:47.680027 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.682231 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:47.632761 31248 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:47.692543 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.633339 31223 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:47.632826 30791 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:47.688603 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.689831 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.690090 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.707715 26201 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:47.658149 31056 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:47.708914 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.709833 26201 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:47.691504 26194 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:47.624626 31109 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:47.632908 31243 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:47.692020 26199 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:25:47.697685 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.530992 31159 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.703629 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:47.727473 26047 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:25:47.704941 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.705840 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.678931 31112 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:47.713354 26123 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:47.719251 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.719537 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.719313 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:47.720755 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 401274us
I20260709 12:25:47.733017 26195 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:47.606297 31140 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.675595 31166 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.708201 31223 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.722760 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.723686 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.755867 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.757609 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.728889 26200 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.759711 26201 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.628618 31217 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:25:47.633057 31237 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:25:47.632613 31095 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:25:47.636808 30831 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:25:47.732061 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.778311 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.778883 26203 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:47.735826 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.638657 31156 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:25:47.700939 31244 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:47.696265 31231 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.693063 31248 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.740806 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.803401 26202 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:25:47.746390 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:47.764810 31223 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.679515 31104 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.764268 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.725198 31243 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.771086 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:47.771916 26124 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:25:47.772162 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:47.773344 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.774629 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.812331 26194 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:47.821120 26195 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:47.735816 31112 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:47.734958 31168 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:47.780623 26199 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:47.743678 31128 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:47.784085 31244 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:47.788396 31109 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.851297 31244 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.793552 31140 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:47.793849 26201 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:47.843242 31168 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:47.795184 31054 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:47.720485 30857 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:25:47.854902 26195 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:47.742502 30791 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:47.858407 30791 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:47.805830 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 479473us
I20260709 12:25:47.745361 31166 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:47.807592 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.862052 31166 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:47.770258 31034 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:47.816321 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:47.823127 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52129.5us
I20260709 12:25:47.810628 31248 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:47.662499 31033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.015s
I20260709 12:25:47.838447 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:47.839583 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.691979 30998 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:47.853807 31168 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.859676 26050 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:25:47.866755 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.814306 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 11.
W20260709 12:25:47.811839 30959 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.001s
I20260709 12:25:47.880637 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:47.777064 31165 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:47.885213 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 77665.9us
I20260709 12:25:47.915400 26195 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:47.824728 31243 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.927585 26197 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:47.988054 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:47.928423 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:47.989162 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.933161 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94831.4us
I20260709 12:25:47.991578 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:47.993747 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 113242us
I20260709 12:25:47.995101 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:47.841115 30986 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.000161 26791 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:25:48.002736 26126 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:48.002848 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.004904 26122 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:47.840891 31128 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:47.809463 31231 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:48.027248 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:48.016893 26786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.016s	sys 0.017s
W20260709 12:25:48.030479 26786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.016s	sys 0.017s
I20260709 12:25:48.030797 26127 raft_consensus.cc:2368] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
W20260709 12:25:47.924360 31032 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.758s	user 0.011s	sys 0.001s
I20260709 12:25:48.002422 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:47.916991 31198 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.033574 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.033818 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:47.926054 31033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.009s	sys 0.015s
I20260709 12:25:48.043365 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:48.045117 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.986917 31102 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:48.081439 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:48.083189 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.005405 30838 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:48.085995 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:48.088763 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.089983 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:25:48.117437 26122 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:48.114470 26196 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:48.111187 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:48.119940 26050 raft_consensus.cc:2376] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12. Re-sending same reply.
I20260709 12:25:48.122427 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.033501 30959 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:48.039119 31128 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:48.135390 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.035439 31271 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.082412 31225 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.005s
I20260709 12:25:47.989835 31270 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.210s
W20260709 12:25:48.140585 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.141273 26197 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:48.091017 31238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.018s	sys 0.019s
I20260709 12:25:47.931803 31129 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.151495 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:48.137349 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.168289 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.169942 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:47.923130 31034 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.108930 31102 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:48.130789 31224 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:48.190798 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:48.193449 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:48.116990 30838 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:48.079430 31217 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:25:48.135707 31274 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:48.218146 26201 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:48.163446 31238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.019s	sys 0.020s
W20260709 12:25:48.229015 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.005s	sys 0.012s
W20260709 12:25:48.229475 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.005s	sys 0.012s
I20260709 12:25:48.044569 31198 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:48.032707 30918 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:48.206727 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.237442 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.238351 26202 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:48.239750 26202 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:48.245292 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:48.245770 26196 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [11.10-17.14]
I20260709 12:25:48.246320 26786 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:48.247455 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:48.037425 30986 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.253415 31129 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.254428 30986 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.140036 30959 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:48.207291 26054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.005s	sys 0.009s
W20260709 12:25:48.207531 26782 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.008s	sys 0.007s
W20260709 12:25:48.162822 31123 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.002s
I20260709 12:25:48.208717 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.042598 31032 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:48.212549 26046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.009s	sys 0.056s
W20260709 12:25:48.275854 26046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.010s	sys 0.056s
I20260709 12:25:48.205204 31241 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:48.226264 26200 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:48.206286 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:48.162573 31104 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:25:48.277808 26122 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:48.219288 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.262459 26196 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:48.265117 26121 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:25:48.281257 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.004s	sys 0.012s
W20260709 12:25:48.281795 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.012s
W20260709 12:25:48.282101 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 599882us
I20260709 12:25:48.284381 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:48.269222 26782 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.008s	sys 0.008s
I20260709 12:25:48.297398 26127 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:25:48.268833 26054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.005s	sys 0.009s
W20260709 12:25:48.283547 26049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.005s	sys 0.008s
W20260709 12:25:48.298978 26049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.005s	sys 0.008s
I20260709 12:25:48.201011 31224 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:48.306953 31224 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:48.261590 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 586871us
I20260709 12:25:48.201174 31102 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.283493 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:48.215910 30838 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.287436 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.317708 26196 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:25:48.268735 31200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.003s	sys 0.011s
I20260709 12:25:48.287384 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.333348 26200 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:48.334074 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:48.162846 31271 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:48.228783 30981 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.011s	sys 0.006s
I20260709 12:25:48.220067 31159 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.189706 31225 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.232074 31229 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.150758 31033 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.337637 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:48.339972 26019 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.569s (SENT)
I20260709 12:25:48.270076 30959 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.349027 26787 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:48.302953 26199 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
W20260709 12:25:48.351610 30981 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.011s	sys 0.006s
W20260709 12:25:48.332950 31200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.003s	sys 0.011s
I20260709 12:25:48.379573 26757 leader_election.cc:400] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
W20260709 12:25:48.380474 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.397673 26757 leader_election.cc:403] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:48.382928 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:48.369231 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:48.351461 31237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.017s	sys 0.002s
I20260709 12:25:48.387344 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:48.394030 26786 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 12: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:48.354661 30918 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:48.405344 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65425.9us
I20260709 12:25:48.401170 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.405210 26791 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:48.406951 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.407245 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:48.408015 26783 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:48.410070 26129 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.018s
W20260709 12:25:48.410476 26129 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.006s	sys 0.018s
I20260709 12:25:48.407881 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:48.256670 31274 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.280943 31123 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.415619 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:48.382788 30780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.011s	sys 0.011s
W20260709 12:25:48.396543 30831 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.001s	sys 0.021s
I20260709 12:25:48.408608 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:48.393404 31247 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.000s	sys 0.021s
I20260709 12:25:48.412699 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:48.393196 31033 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:48.405889 31237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.018s	sys 0.002s
I20260709 12:25:48.333417 31238 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:48.411568 31285 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:48.467629 26123 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:25:48.425941 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:48.447183 26053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.009s	sys 0.016s
I20260709 12:25:48.420090 30998 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.422528 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:48.420485 30780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.011s	sys 0.011s
W20260709 12:25:48.420683 30831 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.001s	sys 0.022s
I20260709 12:25:48.368323 31251 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 12:25:48.406452 31000 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.843s	user 0.011s	sys 0.001s
I20260709 12:25:48.468708 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
W20260709 12:25:48.422374 31247 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.000s	sys 0.022s
W20260709 12:25:48.471154 26754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.014s	sys 0.010s
I20260709 12:25:48.420622 30857 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:48.476029 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.484413 31255 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:25:48.447054 26203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.003s	sys 0.022s
W20260709 12:25:48.446376 31230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.038s	sys 0.035s
I20260709 12:25:48.479724 26791 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.485944 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:48.499980 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:48.467253 31227 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:48.467100 31285 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:48.471540 31231 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:48.508638 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.014s	sys 0.010s
I20260709 12:25:48.424284 31276 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:48.582485 31276 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.582928 31032 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:48.583002 31276 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.583781 31032 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:48.584527 31032 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.587257 31032 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:48.478787 26053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.010s	sys 0.017s
W20260709 12:25:48.515686 26048 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.000s	sys 0.029s
W20260709 12:25:48.588928 26048 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.000s	sys 0.030s
I20260709 12:25:48.482802 31238 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:48.590107 31238 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:48.591668 31082 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.009s	user 0.011s	sys 0.000s
I20260709 12:25:48.591233 31238 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:48.593719 31238 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:48.519136 26203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.004s	sys 0.022s
I20260709 12:25:48.592425 31291 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.558068 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:48.511744 31270 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 15
W20260709 12:25:48.604321 31279 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.202s	user 0.004s	sys 0.000s
W20260709 12:25:48.605132 31099 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.004s	user 0.012s	sys 0.000s
W20260709 12:25:48.606246 25948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 286) took 36337 ms (client timeout 15000 ms). Trace:
W20260709 12:25:48.537715 31230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.040s	sys 0.035s
I20260709 12:25:48.607488 26047 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:48.607954 31285 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:48.609614 31285 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:48.609917 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:48.610672 26047 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:48.611896 26197 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:48.612190 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:48.614466 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:48.618049 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:48.616822 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:48.620323 31273 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.592s	user 0.000s	sys 0.004s
I20260709 12:25:48.559248 31227 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.621104 31227 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.621891 31140 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:48.621944 26047 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:48.622500 31140 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.618724 26786 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.628587 31122 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.521153 31287 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:48.513940 30857 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.478693 31288 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
W20260709 12:25:48.632320 31244 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.780s	user 0.020s	sys 0.000s
I20260709 12:25:48.581547 31034 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:48.622448 31033 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:48.557827 31241 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.128s
I20260709 12:25:48.714928 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:48.798823 26129 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.802109 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1102) took 1030 ms. Trace:
I20260709 12:25:48.803987 26783 rpcz_store.cc:276] 0709 12:25:47.771581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.771825 (+   244us) service_pool.cc:224] Handling call
0709 12:25:48.801812 (+1029987us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540057a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:48.802022 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334907648}
W20260709 12:25:48.785197 31289 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:48.784626 31271 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:48.788584 31225 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:48.747395 31224 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.771754 30838 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.816918 26200 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:48.818069 31095 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:48.818442 30838 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:48.607025 25948 rpcz_store.cc:269] 0709 12:25:12.258237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.258510 (+   273us) service_pool.cc:224] Handling call
0709 12:25:13.490056 (+1231546us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b743adf3a 00007f5b7a163e9c 00007f5b7a26204e 00007f5b787232a4 00007f5b78723232 00007f5b78723161 00007f5b787230dc 00007f5b78722392 00007f5b726e048c 00007f5b726df8c6
0709 12:25:48.568599 (+35078543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:41.561036 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:41.709345 (+148309us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:41.730652 (+ 21307us) write_op.cc:435] Acquired schema lock
0709 12:25:41.765613 (+ 34961us) tablet.cc:662] Decoding operations
0709 12:25:41.982125 (+216512us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:42.004168 (+ 22043us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:42.022719 (+ 18551us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:25:42.348561 (+325842us) tablet.cc:701] Row locks acquired
0709 12:25:42.348616 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:25:42.348884 (+   268us) write_op.cc:270] Start()
0709 12:25:42.349082 (+   198us) write_op.cc:276] Timestamp: P: 1783599942348789 usec, L: 0
0709 12:25:42.349128 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:25:42.350205 (+  1077us) log.cc:844] Serialized 7865 byte log entry
0709 12:25:42.730111 (+379906us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b1800195280. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:43.226374 (+496263us) op_driver.cc:464] REPLICATION: finished
0709 12:25:43.636624 (+410250us) write_op.cc:301] APPLY: starting
0709 12:25:43.698257 (+ 61633us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:44.148150 (+449893us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:44.160441 (+ 12291us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:45.123805 (+963364us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:45.235385 (+111580us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 12:25:45.235513 (+   128us) write_op.cc:312] APPLY: finished
0709 12:25:45.249581 (+ 14068us) log.cc:844] Serialized 180 byte log entry
0709 12:25:45.569136 (+319555us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:45.586881 (+ 17745us) write_op.cc:454] Released schema lock
0709 12:25:45.594901 (+  8020us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:45.625568 (+ 30667us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:45.625600 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3306,"reported_tablets":47,"spinlock_wait_cycles":1128273664,"child_traces":[["op",{"apply.queue_time_us":334796,"apply.run_cpu_time_us":60691,"apply.run_wall_time_us":2017126,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87847,"num_ops":20,"prepare.queue_time_us":97995,"prepare.run_cpu_time_us":23750,"prepare.run_wall_time_us":870809,"raft.queue_time_us":249065,"raft.run_cpu_time_us":13073,"raft.run_wall_time_us":353592,"replication_time_us":819706,"spinlock_wait_cycles":10909440,"thread_start_us":709523,"threads_started":5,"wal-append.queue_time_us":469430,"wal-append.run_cpu_time_us":44362,"wal-append.run_wall_time_us":2408129}]]}
I20260709 12:25:48.707710 31244 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.707949 31283 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.789799 31056 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.063s	user 0.012s	sys 0.000s
I20260709 12:25:48.820328 31123 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.820111 31283 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.820582 31056 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:48.821503 31244 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.767202 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3022) took 1082 ms. Trace:
W20260709 12:25:48.772583 31255 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:25:48.690807 30857 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.777534 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2188) took 1078 ms. Trace:
I20260709 12:25:48.778249 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3009) took 1136 ms. Trace:
I20260709 12:25:48.815099 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3010) took 1173 ms. Trace:
W20260709 12:25:48.791209 31223 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.955s	user 0.011s	sys 0.000s
I20260709 12:25:48.752164 30959 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.818451 31224 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:48.811903 31243 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.778s	user 0.012s	sys 0.000s
I20260709 12:25:48.819330 31140 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:48.764554 31198 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:48.952054 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1788) took 1062 ms. Trace:
I20260709 12:25:48.864987 31225 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:48.952666 26200 rpcz_store.cc:276] 0709 12:25:47.889961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.343929 (+453968us) service_pool.cc:224] Handling call
0709 12:25:48.367330 (+ 23401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.817446 (+450116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.951692 (+134246us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54002b5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:48.951965 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033477504}
I20260709 12:25:48.953353 31225 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.691875 31287 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.956274 31243 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.958024 31287 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:48.958452 31243 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:48.780202 31266 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.306s	user 0.010s	sys 0.002s
W20260709 12:25:48.791141 31168 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.002s
I20260709 12:25:48.822484 26120 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:48.788815 31166 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.926s	user 0.012s	sys 0.001s
W20260709 12:25:48.823159 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.982092 26202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.018s	sys 0.052s
I20260709 12:25:48.822413 26052 rpcz_store.cc:276] 0709 12:25:47.684960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.685433 (+   473us) service_pool.cc:224] Handling call
0709 12:25:47.720083 (+ 34650us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:47.841463 (+121380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.841495 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:47.879619 (+ 38124us) log.cc:844] Serialized 52 byte log entry
0709 12:25:48.475514 (+595895us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:48.475578 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.593887 (+118309us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400030054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:48.593959 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.766786 (+172827us) raft_consensus.cc:1685] finished
0709 12:25:48.766816 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.767113 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16731,"raft.queue_time_us":17318,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":54048,"spinlock_wait_cycles":239280000,"thread_start_us":442080,"threads_started":1,"wal-append.queue_time_us":778043}
W20260709 12:25:48.982985 26202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.018s	sys 0.052s
W20260709 12:25:48.786964 31112 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.908s	user 0.007s	sys 0.004s
W20260709 12:25:48.797029 31248 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.810s	user 0.006s	sys 0.006s
I20260709 12:25:48.834492 26047 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:48.984182 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 641) took 1098 ms. Trace:
I20260709 12:25:48.984730 26047 rpcz_store.cc:276] 0709 12:25:47.886070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.616295 (+730225us) service_pool.cc:224] Handling call
0709 12:25:48.984064 (+367769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.865913 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2305) took 1097 ms. Trace:
I20260709 12:25:48.875123 26791 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:48.747043 31274 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:48.878966 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 472387us
I20260709 12:25:48.819293 30838 raft_consensus.cc:2764] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:48.862327 26203 rpcz_store.cc:276] 0709 12:25:47.698904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.788478 (+ 89574us) service_pool.cc:224] Handling call
0709 12:25:47.788911 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.789288 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.789320 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.789779 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.597866 (+808087us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:48.597924 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.598011 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.777286 (+179275us) raft_consensus.cc:1685] finished
0709 12:25:48.777308 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.777470 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1296,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":98,"thread_start_us":807740,"threads_started":1,"wal-append.queue_time_us":982671}
I20260709 12:25:48.867280 26049 rpcz_store.cc:276] 0709 12:25:47.641900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.645352 (+  3452us) service_pool.cc:224] Handling call
0709 12:25:47.680763 (+ 35411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.706413 (+ 25650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.706449 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.725989 (+ 19540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.319261 (+593272us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:48.319308 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.319378 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.814766 (+495388us) raft_consensus.cc:1685] finished
0709 12:25:48.814802 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.815017 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25880,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":12199,"thread_start_us":543720,"threads_started":1,"wal-append.queue_time_us":763400}
I20260709 12:25:48.900555 26198 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
W20260709 12:25:48.908550 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.007s	sys 0.020s
W20260709 12:25:48.824975 31008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.018s
I20260709 12:25:48.899746 26786 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:48.912210 26129 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
W20260709 12:25:48.949997 31284 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 12:25:48.951412 31212 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.449s	user 0.012s	sys 0.009s
I20260709 12:25:48.868366 26053 rpcz_store.cc:276] 0709 12:25:47.641594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.645274 (+  3680us) service_pool.cc:224] Handling call
0709 12:25:47.664626 (+ 19352us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:47.665054 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.665084 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:47.665822 (+   738us) log.cc:844] Serialized 78 byte log entry
0709 12:25:48.588609 (+922787us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:48.627975 (+ 39366us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.628328 (+   353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.628417 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.777973 (+149556us) raft_consensus.cc:1685] finished
0709 12:25:48.777996 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.778179 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55750,"prepare.queue_time_us":112370,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":3309,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":52292,"thread_start_us":922244,"threads_started":1,"wal-append.queue_time_us":1108404}
I20260709 12:25:48.963371 26200 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:48.973143 26785 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:48.893920 31199 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.006s	sys 0.025s
I20260709 12:25:48.832921 31283 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.818533 31095 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:48.999336 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:49.086716 26128 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:25:49.005218 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.008s	sys 0.020s
I20260709 12:25:49.087101 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 637) took 1202 ms. Trace:
I20260709 12:25:49.015805 26791 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:49.087592 26047 rpcz_store.cc:276] 0709 12:25:47.884905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.998947 (+1114042us) service_pool.cc:224] Handling call
0709 12:25:49.087033 (+ 88086us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.088284 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 895) took 1058 ms. Trace:
I20260709 12:25:49.088951 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:49.018019 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.007s	sys 0.016s
W20260709 12:25:49.090257 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.007s	sys 0.017s
W20260709 12:25:49.018415 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.015s	sys 0.009s
W20260709 12:25:49.035014 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.090950 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.015s	sys 0.009s
I20260709 12:25:49.062366 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 896) took 1030 ms. Trace:
I20260709 12:25:49.091400 26786 rpcz_store.cc:276] 0709 12:25:48.075801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.008104 (+932303us) service_pool.cc:224] Handling call
0709 12:25:49.061544 (+ 53440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.061854 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.061899 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.061957 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:49.062032 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.062114 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.062284 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:49.064605 26121 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.008s	sys 0.024s
W20260709 12:25:48.998270 31128 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.000s
I20260709 12:25:49.082819 26129 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:25:48.876032 31223 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:49.083228 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.016s	sys 0.015s
I20260709 12:25:49.083456 26200 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:49.085614 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:49.098004 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:49.098201 26052 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:48.962790 31225 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.090438 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 639) took 1205 ms. Trace:
I20260709 12:25:49.134487 26047 rpcz_store.cc:276] 0709 12:25:47.885476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.088605 (+1203129us) service_pool.cc:224] Handling call
0709 12:25:49.090360 (+  1755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.136307 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3028) took 1393 ms. Trace:
W20260709 12:25:49.136883 30918 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.010s
I20260709 12:25:48.986068 26123 rpcz_store.cc:276] 0709 12:25:47.768612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.769297 (+   685us) service_pool.cc:224] Handling call
0709 12:25:47.769658 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.468118 (+698460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.829592 (+361474us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b5400369cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:48.865838 (+ 36246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604518784}
I20260709 12:25:49.136684 26047 rpcz_store.cc:276] 0709 12:25:47.742582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.135403 (+1392821us) service_pool.cc:224] Handling call
0709 12:25:49.135733 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.135902 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.135928 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.135995 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.136028 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.136085 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.136221 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:49.092671 26121 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.008s	sys 0.025s
W20260709 12:25:49.092588 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 679934us
W20260709 12:25:49.087347 31199 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.006s	sys 0.026s
I20260709 12:25:49.139880 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:49.140347 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.016s	sys 0.015s
I20260709 12:25:48.997102 31248 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:49.140918 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 857421us
I20260709 12:25:49.141428 26790 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:49.141607 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:49.088904 26791 rpcz_store.cc:276] 0709 12:25:48.030212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.998185 (+967973us) service_pool.cc:224] Handling call
0709 12:25:49.015619 (+ 17434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.087944 (+ 72325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.088197 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.142374 31315 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:49.142540 31248 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.142874 31315 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:49.143304 31315 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.997332 31112 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.143671 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 898) took 1054 ms. Trace:
I20260709 12:25:49.144747 31283 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:49.144963 31238 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:49.145475 31112 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:49.146430 31227 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.000s
I20260709 12:25:49.146593 31238 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:49.147171 31238 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:49.146927 31227 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.149231 31056 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.149348 31227 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:49.082304 31286 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.148160 31238 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.098069 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 889) took 1083 ms. Trace:
I20260709 12:25:49.144106 26791 rpcz_store.cc:276] 0709 12:25:48.088923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.142777 (+1053854us) service_pool.cc:224] Handling call
0709 12:25:49.143130 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.143331 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.143360 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.143410 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:49.143443 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.143501 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.143613 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.150394 31286 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.150812 26785 rpcz_store.cc:276] 0709 12:25:48.014403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.014562 (+   159us) service_pool.cc:224] Handling call
0709 12:25:49.097864 (+1083302us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54001162d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.098005 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2167570688}
I20260709 12:25:49.152060 26791 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:49.153141 31286 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:48.961956 31276 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.098749 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1784) took 1215 ms. Trace:
I20260709 12:25:49.154136 26052 rpcz_store.cc:276] 0709 12:25:47.883115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.003595 (+1120480us) service_pool.cc:224] Handling call
0709 12:25:49.098695 (+ 95100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:49.007623 31008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.007s	sys 0.019s
W20260709 12:25:49.061427 31102 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.004s
I20260709 12:25:49.098755 26049 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:49.173966 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3026) took 1436 ms. Trace:
I20260709 12:25:48.989949 31168 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.113374 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:49.118016 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3021) took 1434 ms. Trace:
I20260709 12:25:48.981647 31266 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:48.996430 31166 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.061818 31274 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.099704 31128 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.176841 26054 rpcz_store.cc:276] 0709 12:25:47.683355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.683481 (+   126us) service_pool.cc:224] Handling call
0709 12:25:47.683807 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.298390 (+614583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.298417 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.298901 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.670097 (+371196us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:48.670143 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.670212 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.117792 (+447580us) raft_consensus.cc:1685] finished
0709 12:25:49.117813 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.117955 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22603,"raft.queue_time_us":46292,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":8021,"thread_start_us":962364,"threads_started":2,"wal-append.queue_time_us":708648}
W20260709 12:25:49.124883 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.179404 31102 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:49.179476 31270 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.000s
I20260709 12:25:49.183931 31270 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:25:49.184263 31223 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.096534 31159 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.185039 31102 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.184980 31159 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.187476 31128 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.189026 31165 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:49.189549 31165 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:49.189751 31276 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:49.190042 31168 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.190618 31165 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.191854 31165 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:49.192095 31159 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:49.193481 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 846) took 1377 ms (client timeout 1532 ms). Trace:
W20260709 12:25:49.193945 26128 rpcz_store.cc:269] 0709 12:25:47.815873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.816031 (+   158us) service_pool.cc:224] Handling call
0709 12:25:49.193297 (+1377266us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.193414 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2774684416}
I20260709 12:25:49.195345 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.196668 31274 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.197293 31166 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:49.198323 26125 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.129798 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 897) took 1041 ms. Trace:
I20260709 12:25:49.199492 26122 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:49.199656 26786 rpcz_store.cc:276] 0709 12:25:48.088490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.098461 (+1009971us) service_pool.cc:224] Handling call
0709 12:25:49.129084 (+ 30623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.129331 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.129373 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.129431 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.129479 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.129550 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.129713 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:49.177409 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.017s
I20260709 12:25:49.130714 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:49.133849 31285 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.011s
I20260709 12:25:49.138768 30959 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:49.149902 31056 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:49.096333 26045 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.369s	user 0.012s	sys 0.020s
W20260709 12:25:49.204201 26045 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.477s	user 0.012s	sys 0.020s
I20260709 12:25:49.143862 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:49.160532 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:25:49.205511 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61679.9us
I20260709 12:25:49.206449 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:49.207686 26126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.013s	sys 0.006s
W20260709 12:25:49.208149 26126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.014s	sys 0.006s
I20260709 12:25:49.174432 26047 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:49.174446 26049 rpcz_store.cc:276] 0709 12:25:47.737305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.064988 (+1327683us) service_pool.cc:224] Handling call
0709 12:25:49.173875 (+108887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.177085 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 1782) took 1294 ms. Trace:
I20260709 12:25:49.209141 31212 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:49.209810 31212 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:49.210147 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.005s	sys 0.013s
I20260709 12:25:49.209848 31237 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.181876 26789 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:49.173369 31156 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.359s	user 0.029s	sys 0.043s
W20260709 12:25:49.201861 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71230.6us
W20260709 12:25:49.210536 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.005s	sys 0.013s
I20260709 12:25:49.210861 31128 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:49.211620 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:49.202953 26784 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.196062 26128 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:49.199522 26048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.003s	sys 0.015s
I20260709 12:25:49.209056 26053 rpcz_store.cc:276] 0709 12:25:47.882464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.082801 (+1200337us) service_pool.cc:224] Handling call
0709 12:25:49.177023 (+ 94222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.124881 31095 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:49.214023 26754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.005s	sys 0.013s
W20260709 12:25:49.214303 26048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.004s	sys 0.015s
W20260709 12:25:49.214684 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.005s	sys 0.013s
W20260709 12:25:49.200834 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.000s	sys 0.017s
I20260709 12:25:49.222432 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.223779 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.227211 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:49.207347 30981 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.001s	sys 0.017s
I20260709 12:25:49.228967 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.239037 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:49.228948 26124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.020s	sys 0.014s
I20260709 12:25:49.236706 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:49.220227 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.246068 26054 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:49.246758 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:49.246918 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.247387 26785 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.206308 31159 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:49.210436 31212 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.206898 31274 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:49.246532 26124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.021s	sys 0.015s
I20260709 12:25:49.254191 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:49.254561 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:49.255306 26049 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
I20260709 12:25:49.255376 26053 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:25:49.240553 26786 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:25:49.212112 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 726224us
I20260709 12:25:49.226137 31243 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.222303 31283 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:49.254253 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3072) took 1259 ms. Trace:
W20260709 12:25:49.253784 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.006s	sys 0.015s
I20260709 12:25:49.261610 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.257838 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:49.276312 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.007s	sys 0.015s
I20260709 12:25:49.263950 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:49.276073 26786 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:49.239007 26128 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:49.277345 26052 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:49.266615 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.205317 30857 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.266535 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:49.248047 31156 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.439s	user 0.029s	sys 0.043s
W20260709 12:25:49.273216 26194 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.016s	sys 0.008s
W20260709 12:25:49.310382 26194 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.016s	sys 0.008s
I20260709 12:25:49.310837 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 894) took 1294 ms. Trace:
I20260709 12:25:49.268330 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:49.249374 31166 raft_consensus.cc:2764] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 12:25:49.280169 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 780198us
I20260709 12:25:49.317036 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:49.244745 31251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.008s	sys 0.020s
I20260709 12:25:49.278764 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:49.289475 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.324427 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:49.294263 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2203) took 1076 ms. Trace:
I20260709 12:25:49.325726 26201 rpcz_store.cc:276] 0709 12:25:48.217455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.219033 (+  1578us) service_pool.cc:224] Handling call
0709 12:25:49.293889 (+1074856us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400163354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.294076 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436448000}
I20260709 12:25:49.297708 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1807) took 1014 ms. Trace:
I20260709 12:25:49.326534 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:49.225452 31237 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:25:49.301290 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 888) took 1287 ms. Trace:
I20260709 12:25:49.330148 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:49.245291 30981 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.001s	sys 0.018s
I20260709 12:25:49.277566 31168 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.311311 26783 rpcz_store.cc:276] 0709 12:25:48.016709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.819809 (+803100us) service_pool.cc:224] Handling call
0709 12:25:48.820115 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.963175 (+143060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.963222 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:48.963282 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:48.963326 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.310500 (+347174us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54005d20d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.310566 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.310760 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328387840}
I20260709 12:25:49.315137 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2189) took 1615 ms. Trace:
W20260709 12:25:49.318154 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49952.9us
I20260709 12:25:49.319721 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.344089 26199 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:49.398655 26791 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:49.256608 31095 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:49.331585 31223 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:25:49.336220 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:49.256136 31308 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
W20260709 12:25:49.280732 31247 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.015s	sys 0.009s
W20260709 12:25:49.299377 31278 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.711s	user 0.000s	sys 0.008s
I20260709 12:25:49.263914 31293 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.327006 26203 rpcz_store.cc:276] 0709 12:25:48.282872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.065091 (+782219us) service_pool.cc:224] Handling call
0709 12:25:49.295109 (+230018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.295405 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.295447 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.295593 (+   146us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.295663 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.295767 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.297441 (+  1674us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400163350. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.297617 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321070464}
W20260709 12:25:49.329372 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62922.9us
I20260709 12:25:49.332340 26197 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:49.329669 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:49.414294 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84702.6us
I20260709 12:25:49.315191 31283 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:49.385077 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:49.323047 31297 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:49.226505 30831 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:25:49.330346 26784 rpcz_store.cc:276] 0709 12:25:48.013433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.013799 (+   366us) service_pool.cc:224] Handling call
0709 12:25:48.044714 (+ 30915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.216448 (+1171734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.300927 (+ 84479us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400355a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.301202 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2660594560}
I20260709 12:25:49.340354 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:49.345112 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1787) took 1461 ms. Trace:
I20260709 12:25:49.332559 31286 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
W20260709 12:25:49.332742 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.009s	sys 0.025s
W20260709 12:25:49.443842 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.012s	sys 0.027s
I20260709 12:25:49.343983 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3074) took 1345 ms. Trace:
I20260709 12:25:49.445116 26125 rpcz_store.cc:276] 0709 12:25:47.998204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.998419 (+   215us) service_pool.cc:224] Handling call
0709 12:25:47.998879 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.198915 (+1200036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.343453 (+144538us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.343899 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2757865344}
I20260709 12:25:49.446346 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.446583 26052 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
W20260709 12:25:49.385257 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60099.9us
W20260709 12:25:49.340639 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2186) took 1643 ms (client timeout 1911 ms). Trace:
I20260709 12:25:49.447481 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.400866 26783 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:49.341935 26196 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:25:49.332913 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.343297 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1782) took 1481 ms. Trace:
W20260709 12:25:49.387818 31195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.006s	sys 0.025s
W20260709 12:25:49.330293 31251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.008s	sys 0.021s
I20260709 12:25:49.405234 26200 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 10.
W20260709 12:25:49.251675 30780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.030s	sys 0.024s
W20260709 12:25:49.311537 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6786345769996842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
I20260709 12:25:49.320927 26785 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:25:49.308357 31109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.017s	sys 0.022s
W20260709 12:25:49.491040 26094 leader_election.cc:336] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.627s (SENT)
I20260709 12:25:49.491480 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:49.266942 31159 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:49.402587 26194 rpcz_store.cc:276] 0709 12:25:47.699266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.837317 (+138051us) service_pool.cc:224] Handling call
0709 12:25:47.838074 (+   757us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:47.838652 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.838707 (+    55us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:47.863739 (+ 25032us) log.cc:844] Serialized 78 byte log entry
0709 12:25:48.369860 (+506121us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:49.310918 (+941058us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.311371 (+   453us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.314435 (+  3064us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.314693 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.314795 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.314850 (+    55us) raft_consensus.cc:1685] finished
0709 12:25:49.314868 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.315048 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":749674,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":75,"raft.queue_time_us":10288,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":38851,"spinlock_wait_cycles":84446720,"thread_start_us":1239650,"threads_started":2,"wal-append.queue_time_us":685507}
I20260709 12:25:49.500880 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:49.501691 26194 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:49.502697 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:49.503389 26194 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:49.320595 31104 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.505270 30857 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:49.417132 31168 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.507247 30857 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:49.320403 31241 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.277534 31243 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.438862 26047 rpcz_store.cc:276] 0709 12:25:47.883984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.138585 (+1254601us) service_pool.cc:224] Handling call
0709 12:25:49.173914 (+ 35329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.208670 (+ 34756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.344884 (+136214us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b5400228b50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.345048 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80270208}
I20260709 12:25:49.332056 31227 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.444474 26201 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:25:49.513649 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:49.514273 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3082) took 1209 ms. Trace:
I20260709 12:25:49.514840 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:49.515569 26201 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.341166 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 890) took 1326 ms. Trace:
I20260709 12:25:49.401497 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:49.516718 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115348us
W20260709 12:25:49.413136 31247 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.015s	sys 0.010s
I20260709 12:25:49.526376 31247 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:49.490530 30780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.030s	sys 0.025s
I20260709 12:25:49.448585 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:49.448175 26125 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:49.335399 31270 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:49.335523 31000 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:49.414322 31300 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:25:49.480181 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:49.507830 30857 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.482059 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.475445 26195 rpcz_store.cc:276] 0709 12:25:47.861821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.044842 (+183021us) service_pool.cc:224] Handling call
0709 12:25:49.342786 (+1297944us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.342939 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2427462400}
I20260709 12:25:49.489004 26791 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:49.385921 31122 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.508749 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:49.556988 26203 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.555903 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.557888 26128 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:49.509222 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:49.509680 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:49.508818 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:49.516373 26790 rpcz_store.cc:276] 0709 12:25:48.015090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.015384 (+   294us) service_pool.cc:224] Handling call
0709 12:25:49.340914 (+1325530us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54004ba854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.341093 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2376339840}
I20260709 12:25:49.559484 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.558732 26049 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:49.440006 31297 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:49.507978 31168 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:49.526782 31309 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:25:49.527918 31247 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:49.572103 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.542261 31317 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:25:49.402951 30791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.013s	sys 0.029s
W20260709 12:25:49.501232 31109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.018s	sys 0.022s
W20260709 12:25:49.528740 31272 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.390s	user 0.000s	sys 0.009s
I20260709 12:25:49.439833 31112 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:49.534211 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.599355 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.499926 31102 raft_consensus.cc:2764] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 12:25:49.474689 31195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.009s	sys 0.025s
I20260709 12:25:49.548175 31270 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:49.514777 26127 rpcz_store.cc:276] 0709 12:25:48.304660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.305180 (+   520us) service_pool.cc:224] Handling call
0709 12:25:48.305566 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.305816 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:48.681633 (+375817us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.513834 (+832201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.513903 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:49.513963 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.514052 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.514205 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68583,"prepare.queue_time_us":471814,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"thread_start_us":1138567,"threads_started":2}
I20260709 12:25:49.608739 31270 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.609405 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.609560 31243 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:49.610687 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.611486 26127 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:49.558434 26194 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:49.613340 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:49.614187 26194 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:49.615657 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.540733 31248 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:25:49.617719 30857 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:49.534142 31307 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:49.573258 31056 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:49.436345 31293 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:49.566232 26202 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.027s	sys 0.020s
W20260709 12:25:49.542603 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94114.6us
I20260709 12:25:49.626747 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:49.546391 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3084) took 1184 ms. Trace:
I20260709 12:25:49.555624 31308 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.572388 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.627521 26121 rpcz_store.cc:276] 0709 12:25:48.362298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.378746 (+ 16448us) service_pool.cc:224] Handling call
0709 12:25:48.379183 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.379444 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:49.155232 (+775788us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.545765 (+390533us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.545975 (+   210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.546024 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:49.546074 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.546155 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.546320 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9091,"prepare.queue_time_us":398492,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":9301,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151,"thread_start_us":1133452,"threads_started":2}
I20260709 12:25:49.629572 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.544811 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:49.447819 26198 rpcz_store.cc:269] 0709 12:25:47.697267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.697400 (+   133us) service_pool.cc:224] Handling call
0709 12:25:49.340384 (+1642984us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002b5f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:49.340560 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2517397248}
I20260709 12:25:49.573307 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:49.541590 26782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.003s	sys 0.019s
I20260709 12:25:49.547748 26201 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:49.554740 31266 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:49.590271 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:49.635663 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45514.6us
I20260709 12:25:49.637480 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:49.561257 31000 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:49.640477 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 95777.2us
I20260709 12:25:49.541829 31320 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:49.594041 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:49.602223 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.603292 26049 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:49.604754 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:49.625713 31331 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:25:49.606165 26125 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:49.564932 31318 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:49.618269 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:49.565886 31297 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:49.578241 30791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.015s	sys 0.029s
I20260709 12:25:49.561546 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 627) took 2472 ms. Trace:
I20260709 12:25:49.562515 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.669442 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2201) took 1555 ms. Trace:
I20260709 12:25:49.670045 26128 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:49.574226 31223 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.571566 31247 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:49.617756 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.577440 31317 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:49.632018 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:49.632385 26202 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.027s	sys 0.021s
W20260709 12:25:49.710753 26045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.044s	sys 0.047s
W20260709 12:25:49.711184 26045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.044s	sys 0.047s
W20260709 12:25:49.711498 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 79552.1us
I20260709 12:25:49.710891 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.712690 26128 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:49.636909 30780 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:49.712684 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.713366 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:49.714059 26050 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.564844 31122 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:49.634547 26782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.003s	sys 0.019s
I20260709 12:25:49.642261 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:49.604890 26203 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:49.654425 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1093) took 2336 ms. Trace:
I20260709 12:25:49.660856 31000 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.647342 31286 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.647591 30981 raft_consensus.cc:2764] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:25:49.666181 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48009.1us
I20260709 12:25:49.668455 26051 rpcz_store.cc:276] 0709 12:25:47.083351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.083623 (+   272us) service_pool.cc:224] Handling call
0709 12:25:47.083969 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.084329 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.084358 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.095013 (+ 10655us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.518213 (+423200us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b180018a0c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:48.229988 (+711775us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:48.230030 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.230262 (+   232us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.298569 (+1068307us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:49.554904 (+256335us) raft_consensus.cc:1685] finished
0709 12:25:49.554926 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.555085 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3533,"raft.queue_time_us":497,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":34617,"spinlock_wait_cycles":36276224,"thread_start_us":711567,"threads_started":1,"wal-append.queue_time_us":1343372}
W20260709 12:25:49.708447 31300 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:25:49.670086 26197 rpcz_store.cc:276] 0709 12:25:48.114411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.613005 (+498594us) service_pool.cc:224] Handling call
0709 12:25:48.630426 (+ 17421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.414007 (+783581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.669046 (+255039us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54002b5054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.669350 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1613741440}
I20260709 12:25:49.683531 26093 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:49.725361 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:49.725423 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.726104 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:49.685302 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.625716 31112 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:49.714939 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1785) took 1826 ms. Trace:
I20260709 12:25:49.715119 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:49.654762 30857 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:49.574961 31283 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.716686 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1161) took 2848 ms. Trace:
I20260709 12:25:49.718314 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:49.616163 31212 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.720780 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.720791 26787 rpcz_store.cc:276] 0709 12:25:47.317553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.320047 (+  2494us) service_pool.cc:224] Handling call
0709 12:25:47.320380 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.385644 (+1065264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.654108 (+1268464us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54005c2954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.654337 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2365070592}
W20260709 12:25:49.722945 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80801.3us
I20260709 12:25:49.744941 26788 rpcz_store.cc:276] 0709 12:25:46.868031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.869276 (+  1245us) service_pool.cc:224] Handling call
0709 12:25:46.886540 (+ 17264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.088420 (+201880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.088458 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.132584 (+ 44126us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.628747 (+496163us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:47.628789 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.278277 (+649488us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:48.278339 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.387643 (+1109304us) spinlock_profiling.cc:243] Waited 8.54 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:49.716489 (+328846us) raft_consensus.cc:1685] finished
0709 12:25:49.716510 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.716633 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":13013,"spinlock_wait_cycles":465774720,"thread_start_us":368403,"threads_started":1,"wal-append.queue_time_us":749286}
I20260709 12:25:49.654920 31195 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:25:49.654162 31288 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:25:49.637648 31168 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.685997 31320 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:49.665426 31225 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:49.597687 31159 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.668031 31266 raft_consensus.cc:2764] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:25:49.747694 30831 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:49.751387 31323 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:25:49.751046 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.751180 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:49.798317 26121 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 9.
I20260709 12:25:49.752338 30981 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:49.753155 31286 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:49.753880 31282 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.128s	user 0.010s	sys 0.005s
I20260709 12:25:49.802837 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:49.760676 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.762068 31238 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.015s	sys 0.038s
W20260709 12:25:49.762794 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 246458us
I20260709 12:25:49.766038 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.811818 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.813407 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.648108 31308 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.726395 26128 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:25:49.733115 31244 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:49.785209 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:49.742914 31307 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:25:49.710474 31223 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.729696 26202 rpcz_store.cc:276] 0709 12:25:47.888838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.267717 (+378879us) service_pool.cc:224] Handling call
0709 12:25:48.268097 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:48.274267 (+  6170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.274303 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:48.275179 (+   876us) log.cc:844] Serialized 52 byte log entry
0709 12:25:48.983519 (+708340us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:49.711156 (+727637us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.714557 (+  3401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.714642 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.714695 (+    53us) raft_consensus.cc:1685] finished
0709 12:25:49.714710 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.714875 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39757,"prepare.queue_time_us":133767,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":10254,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":51428,"thread_start_us":1300880,"threads_started":2,"wal-append.queue_time_us":912732}
I20260709 12:25:49.747396 30791 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.777024 31168 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.810007 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:49.817028 30831 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:49.748383 30857 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:49.883683 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52468.4us
W20260709 12:25:49.910629 31323 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:25:49.936039 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2623386989998835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
I20260709 12:25:49.891820 26787 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.894676 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 845) took 2147 ms. Trace:
I20260709 12:25:49.937096 26120 rpcz_store.cc:276] 0709 12:25:47.747514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.747654 (+   140us) service_pool.cc:224] Handling call
0709 12:25:47.747995 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.972149 (+1224154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.894170 (+922021us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003688d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.894434 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470580992}
I20260709 12:25:49.895795 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3081) took 1600 ms. Trace:
I20260709 12:25:49.939427 26122 rpcz_store.cc:276] 0709 12:25:48.295585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.303622 (+  8037us) service_pool.cc:224] Handling call
0709 12:25:48.304026 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.202410 (+898384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.895451 (+693041us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54002c9cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:49.895685 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2058897152}
I20260709 12:25:49.941591 26122 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:49.785970 31225 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:49.801379 31195 consensus_queue.cc:1243] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:25:49.905424 31321 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:25:49.906531 31333 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:25:49.806898 31312 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.958004 26787 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:25:49.961268 26787 pending_rounds.cc:85] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 9
I20260709 12:25:49.961761 26787 pending_rounds.cc:107] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:25:49.791465 31123 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
W20260709 12:25:49.838629 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.881453 31244 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.886757 26121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:49.948128 30857 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.955412 31274 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:49.723286 31229 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:49.954181 31237 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:49.954547 31314 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:50.001549 26122 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:49.957464 26786 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:49.963831 31338 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 12:25:49.964965 31339 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:25:49.965102 31129 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.050486 31129 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:50.051462 31129 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:49.989872 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.052205 31247 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.991123 26121 raft_consensus.cc:606] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:50.053928 26121 raft_consensus.cc:614] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:49.935431 31320 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.798398 31317 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:50.080178 31317 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:49.898455 31112 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:50.082196 31231 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:50.082650 31231 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:49.810824 31238 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.015s	sys 0.039s
I20260709 12:25:49.956616 31225 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:49.991267 31341 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:49.962345 26787 raft_consensus.cc:2889] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.992988 31244 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.041414 31237 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:49.980371 31312 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:50.042954 31314 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:50.044688 31342 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:25:50.046180 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2308) took 1646 ms. Trace:
I20260709 12:25:50.158759 26129 rpcz_store.cc:276] 0709 12:25:48.399328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.123822 (+724494us) service_pool.cc:224] Handling call
0709 12:25:49.124152 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.129122 (+  4970us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:49.129161 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.129616 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.569104 (+439488us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:49.569469 (+   365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.573501 (+  4032us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.045829 (+472328us) raft_consensus.cc:1685] finished
0709 12:25:50.045865 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.046094 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4394,"raft.queue_time_us":5481,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":438959,"threads_started":1,"wal-append.queue_time_us":632739}
W20260709 12:25:50.049882 31346 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:50.056756 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 887) took 2044 ms. Trace:
I20260709 12:25:50.165247 26789 rpcz_store.cc:276] 0709 12:25:48.012074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.012273 (+   199us) service_pool.cc:224] Handling call
0709 12:25:48.012719 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.210910 (+1198191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.056430 (+845520us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54006bf8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:50.056667 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687931264}
I20260709 12:25:50.085955 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1795) took 2077 ms. Trace:
I20260709 12:25:50.166965 26199 rpcz_store.cc:276] 0709 12:25:48.008685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.464070 (+455385us) service_pool.cc:224] Handling call
0709 12:25:49.343525 (+879455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.399131 (+ 55606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.085703 (+686572us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400336150. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:50.085878 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021759744}
I20260709 12:25:50.084890 31231 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:49.997107 31274 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.155432 30831 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:50.187454 31274 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:50.226543 31345 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
W20260709 12:25:50.231413 31337 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:25:50.232730 31343 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 12:25:50.249506 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1786) took 2360 ms. Trace:
I20260709 12:25:50.253093 31008 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:50.253582 31008 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:50.254608 31008 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:50.253165 26196 rpcz_store.cc:276] 0709 12:25:47.889212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.338259 (+449047us) service_pool.cc:224] Handling call
0709 12:25:49.341446 (+1003187us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:49.474616 (+133170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.249288 (+774672us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002b5f50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:50.249443 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305967488}
I20260709 12:25:50.257978 31198 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:50.261473 31198 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:50.262603 31198 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:50.255604 31008 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.266499 31328 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.267166 31328 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:50.268285 31198 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:50.269778 31335 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.321s	user 0.003s	sys 0.000s
I20260709 12:25:50.268095 31328 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:50.272001 31328 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:50.279663 31344 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:25:50.700984 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3020) took 3024 ms. Trace:
W20260709 12:25:50.701218 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 954426us
I20260709 12:25:50.705657 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:50.707556 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:50.710992 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:50.702349 26046 rpcz_store.cc:276] 0709 12:25:47.676218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.676397 (+   179us) service_pool.cc:224] Handling call
0709 12:25:47.676868 (+   471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:47.677295 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.677329 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:47.677963 (+   634us) log.cc:844] Serialized 52 byte log entry
0709 12:25:48.277183 (+599220us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:48.277232 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.277307 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.721786 (+1444479us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:50.699767 (+977981us) raft_consensus.cc:1685] finished
0709 12:25:50.699855 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.700543 (+   688us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1672,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"spinlock_wait_cycles":111110528,"thread_start_us":598593,"threads_started":1,"wal-append.queue_time_us":806023}
I20260709 12:25:50.714577 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3008) took 3073 ms. Trace:
I20260709 12:25:50.715179 26048 rpcz_store.cc:276] 0709 12:25:47.641217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.644352 (+  3135us) service_pool.cc:224] Handling call
0709 12:25:47.644706 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.645083 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:48.589400 (+944317us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.216004 (+626604us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.216282 (+   278us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.253475 (+ 37193us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.253632 (+   157us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:49.319267 (+ 65635us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.320109 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.320186 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.048909 (+728723us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b180043de00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:50.258751 (+209842us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b180043de00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:50.713925 (+455174us) raft_consensus.cc:1685] finished
0709 12:25:50.713973 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.714371 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95999,"prepare.queue_time_us":913517,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":15688,"raft.queue_time_us":16700,"raft.run_cpu_time_us":550,"raft.run_wall_time_us":34838,"spinlock_wait_cycles":46215168,"thread_start_us":1534705,"threads_started":2,"wal-append.queue_time_us":958709}
I20260709 12:25:50.720259 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 848) took 1536 ms. Trace:
I20260709 12:25:50.720816 26123 rpcz_store.cc:276] 0709 12:25:49.183231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.192369 (+  9138us) service_pool.cc:224] Handling call
0709 12:25:49.192777 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.193128 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.193162 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.193702 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.648277 (+454575us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:49.648523 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.648791 (+   268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.719522 (+1070731us) raft_consensus.cc:1685] finished
0709 12:25:50.719574 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.719939 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4870,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":453396,"threads_started":1,"wal-append.queue_time_us":798003}
W20260709 12:25:50.722293 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.724100 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3083) took 2418 ms. Trace:
W20260709 12:25:50.724834 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.725469 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.724738 26126 rpcz_store.cc:276] 0709 12:25:48.305037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.319512 (+ 14475us) service_pool.cc:224] Handling call
0709 12:25:48.328748 (+  9236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.208634 (+879886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.208667 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.209319 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.578860 (+369541us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:49.578923 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.579023 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.278861 (+699838us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:50.723223 (+444362us) raft_consensus.cc:1685] finished
0709 12:25:50.723273 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.723794 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29203,"raft.queue_time_us":265904,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":30877,"spinlock_wait_cycles":7267712,"thread_start_us":1248442,"threads_started":2,"wal-append.queue_time_us":770629}
W20260709 12:25:50.766321 26782 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.005s	sys 0.017s
W20260709 12:25:50.766973 26782 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.005s	sys 0.017s
W20260709 12:25:50.769272 31056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.004s	sys 0.018s
W20260709 12:25:50.769897 31056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.004s	sys 0.018s
W20260709 12:25:50.784647 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 771504us
I20260709 12:25:50.794234 31129 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.799458 30838 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.801152 31293 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:50.800931 26783 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:25:50.807334 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 1.08053e+06us
I20260709 12:25:50.807789 30838 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:50.821208 31122 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:50.824636 31353 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:50.813568 26047 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.144s	user 0.014s	sys 0.021s
W20260709 12:25:50.830137 26047 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.015s	sys 0.022s
I20260709 12:25:50.814030 26788 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.722898 30780 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:50.837924 30780 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:50.838943 30780 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:50.822250 31122 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:50.801051 31212 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:50.842442 31122 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:50.847177 26791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.179s	user 0.047s	sys 0.061s
I20260709 12:25:50.840211 31314 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:50.858187 26045 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.012s	sys 0.024s
I20260709 12:25:50.863080 31247 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:50.865559 26045 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.012s	sys 0.025s
I20260709 12:25:50.856681 31122 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:50.866446 26791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.047s	sys 0.061s
W20260709 12:25:50.866230 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.011s	sys 0.021s
W20260709 12:25:50.865976 26120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.005s	sys 0.012s
W20260709 12:25:50.858127 31112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.009s	sys 0.011s
W20260709 12:25:50.874573 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.012s	sys 0.021s
I20260709 12:25:50.852793 31353 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.877157 26122 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:50.878149 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3090) took 1461 ms. Trace:
I20260709 12:25:50.878535 26122 rpcz_store.cc:276] 0709 12:25:49.417006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.073602 (+656596us) service_pool.cc:224] Handling call
0709 12:25:50.832516 (+758914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.877720 (+ 45204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.878060 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.873137 31231 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:50.880928 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:50.881870 26783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1169) took 1309 ms (client timeout 1716 ms). Trace:
I20260709 12:25:50.884516 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1172) took 1302 ms. Trace:
I20260709 12:25:50.886786 26199 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:50.869148 31318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.011s	sys 0.015s
W20260709 12:25:50.885542 26120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.005s	sys 0.012s
I20260709 12:25:50.884938 26788 rpcz_store.cc:276] 0709 12:25:49.573597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.887389 (+313792us) service_pool.cc:224] Handling call
0709 12:25:49.887814 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.856407 (+968593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.875688 (+ 19281us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:50.871925 31123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.012s	sys 0.023s
W20260709 12:25:50.883834 26783 rpcz_store.cc:269] 0709 12:25:49.571982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.572135 (+   153us) service_pool.cc:224] Handling call
0709 12:25:50.881530 (+1309395us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:50.881765 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2657843968}
I20260709 12:25:50.893824 30918 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:50.882931 31112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.010s	sys 0.012s
W20260709 12:25:50.899181 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 1.1839e+06us
I20260709 12:25:50.900451 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2311) took 1712 ms. Trace:
W20260709 12:25:50.896651 31318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.011s	sys 0.016s
W20260709 12:25:50.909785 26128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.010s	sys 0.013s
W20260709 12:25:50.917577 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 122034us
W20260709 12:25:50.898243 31123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.013s	sys 0.024s
I20260709 12:25:50.904374 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1791) took 1580 ms. Trace:
W20260709 12:25:50.921833 26128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.010s	sys 0.013s
I20260709 12:25:50.919415 30838 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:50.924815 31168 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:50.941569 30918 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:50.943219 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:50.945605 26196 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:25:50.948886 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1115) took 1389 ms. Trace:
I20260709 12:25:50.949347 26791 rpcz_store.cc:276] 0709 12:25:49.559301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.566422 (+  7121us) service_pool.cc:224] Handling call
0709 12:25:49.568373 (+  1951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.572718 (+  4345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.572780 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.573394 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.916499 (+1343105us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:50.934994 (+ 18495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.948516 (+ 13522us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.948577 (+    61us) raft_consensus.cc:1685] finished
0709 12:25:50.948592 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.948773 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2705,"raft.queue_time_us":5652,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":16768,"thread_start_us":1248343,"threads_started":1,"wal-append.queue_time_us":1232337}
I20260709 12:25:50.952154 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:50.954185 31168 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:50.955231 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:50.968798 26198 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:50.969359 31346 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:25:50.969488 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 126708us
I20260709 12:25:50.970299 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:50.971125 26046 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.971668 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:50.956264 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:50.972811 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:50.973333 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3019) took 3305 ms. Trace:
I20260709 12:25:50.973737 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:50.975056 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:50.960101 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:50.961724 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.962530 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:50.912945 26120 rpcz_store.cc:276] 0709 12:25:49.187458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.938352 (+750894us) service_pool.cc:224] Handling call
0709 12:25:49.938788 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.939084 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:49.973567 (+ 34483us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.899515 (+925948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.899628 (+   113us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:50.899746 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.899878 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.900151 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34663,"prepare.run_cpu_time_us":1361,"prepare.run_wall_time_us":1793,"raft.queue_time_us":1122,"spinlock_wait_cycles":689408,"thread_start_us":916015,"threads_started":1,"wal-append.queue_time_us":851603}
I20260709 12:25:50.978103 26197 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:50.921361 26054 rpcz_store.cc:276] 0709 12:25:49.250377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.250583 (+   206us) service_pool.cc:224] Handling call
0709 12:25:49.251000 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.252151 (+  1151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.252196 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.263290 (+ 11094us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.742578 (+479288us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:50.719148 (+976570us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.785631 (+ 66483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.804054 (+ 18423us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.811865 (+  7811us) raft_consensus.cc:1685] finished
0709 12:25:50.811937 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.830793 (+ 18856us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1042492,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":4294,"raft.queue_time_us":142039,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":15611,"thread_start_us":467812,"threads_started":2,"wal-append.queue_time_us":611254}
I20260709 12:25:50.984158 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:50.990093 31356 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:50.989926 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:50.967439 30918 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:50.993793 26200 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:50.998819 31223 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:51.000880 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 1.28748e+06us
I20260709 12:25:51.001412 31312 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:50.973770 26045 rpcz_store.cc:276] 0709 12:25:47.667506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.682020 (+ 14514us) service_pool.cc:224] Handling call
0709 12:25:47.682321 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.682512 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:49.204669 (+1522157us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.894727 (+1690058us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.918498 (+ 23771us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:50.936977 (+ 18479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.973088 (+ 36111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.973274 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":941433,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":5570,"raft.queue_time_us":32995,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":3138825,"threads_started":3,"wal-append.queue_time_us":751343}
I20260709 12:25:51.005504 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.007270 31168 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.010439 26048 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:51.010315 26050 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:51.016961 31308 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.017925 31237 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:51.023774 31315 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.880s	user 0.025s	sys 0.011s
W20260709 12:25:51.024717 31198 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.756s	user 0.019s	sys 0.005s
I20260709 12:25:51.024447 31315 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.026512 30918 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:51.026921 31315 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.027334 30918 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:49.274905 26124 rpcz_store.cc:276] 0709 12:25:47.994586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.994841 (+   255us) service_pool.cc:224] Handling call
0709 12:25:47.995237 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.995523 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:48.551298 (+555775us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.253831 (+702533us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.253893 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.253937 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.254077 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.254193 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":781855,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":347,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":13136,"thread_start_us":1197027,"threads_started":2}
I20260709 12:25:51.031124 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:51.031975 26791 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:51.028023 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:51.017944 26049 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:25:51.040381 26121 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.003s	sys 0.019s
I20260709 12:25:51.040613 26199 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:51.040824 26121 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.003s	sys 0.019s
I20260709 12:25:51.041882 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3094) took 1520 ms. Trace:
I20260709 12:25:51.042634 31307 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:51.042270 26121 rpcz_store.cc:276] 0709 12:25:49.521261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.055089 (+533828us) service_pool.cc:224] Handling call
0709 12:25:50.055418 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.041442 (+986024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.041502 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.041558 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:51.041585 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.041675 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.041831 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":227890,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":985501,"threads_started":1}
I20260709 12:25:51.043367 31307 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:51.044413 26198 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:51.038290 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:51.047802 31095 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.636s	user 0.011s	sys 0.021s
W20260709 12:25:51.050132 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 122097us
I20260709 12:25:51.048770 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:51.049288 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1171) took 1475 ms (client timeout 1775 ms). Trace:
W20260709 12:25:51.051498 26790 rpcz_store.cc:269] 0709 12:25:49.573323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.574336 (+  1013us) service_pool.cc:224] Handling call
0709 12:25:51.048900 (+1474564us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540057a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:51.049081 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3073277312}
I20260709 12:25:51.033860 30918 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.036623 31312 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:51.060422 26197 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:25:51.072855 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:51.087563 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 376786us
W20260709 12:25:51.088387 30831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.003s	sys 0.013s
I20260709 12:25:51.064790 31223 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:51.066493 31168 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.094508 26046 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:51.097772 31351 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:51.098371 31351 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:51.099519 31165 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.898s	user 0.013s	sys 0.020s
W20260709 12:25:51.102321 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.101302 25952 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 10, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:51.102983 31270 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.494s	user 0.010s	sys 0.021s
W20260709 12:25:51.103432 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.104852 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.068733 31198 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 12:25:51.114252 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.103492 31270 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.077334 30918 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.099584 31351 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.124714 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.125691 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.129772 26051 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:51.133412 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.132618 31307 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.142355 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.144086 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.121567 25952 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 8 to 10, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 10 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:51.147009 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.147881 26048 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:51.148339 31270 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.150199 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:51.150612 31225 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.992s	user 0.009s	sys 0.014s
W20260709 12:25:51.151005 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 293987us
I20260709 12:25:51.151214 31225 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.152212 26050 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:51.152729 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.153446 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.157873 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:51.160573 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.158694 31165 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:51.162446 31274 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.945s	user 0.021s	sys 0.006s
I20260709 12:25:51.164599 31000 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:51.151547 26051 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:51.166219 31225 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.167279 26050 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:25:51.168371 26127 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.203s	user 0.029s	sys 0.031s
I20260709 12:25:51.115165 31312 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.177021 25952 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 11 to 13, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 13 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.178503 31353 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:51.179068 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:51.179644 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61234us
W20260709 12:25:51.158805 31008 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.894s	user 0.012s	sys 0.011s
W20260709 12:25:51.241866 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.246439 26199 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
W20260709 12:25:51.257054 30857 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.163s	user 0.008s	sys 0.019s
I20260709 12:25:51.257301 30831 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:51.257697 30857 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.258934 26203 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:51.260977 26195 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:25:51.261303 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 139217us
I20260709 12:25:51.261696 26790 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:25:51.289990 31363 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
W20260709 12:25:51.290153 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 129731us
I20260709 12:25:51.295329 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.203797 31165 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.294901 26127 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:51.245569 31000 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:51.249095 31274 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.319115 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.319640 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.300186 31238 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.262604 31198 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:51.314255 26050 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:51.315223 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 136401us
I20260709 12:25:51.317219 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.305788 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:51.309425 25952 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 9 to 11, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 11 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:51.298213 31129 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.004s
I20260709 12:25:51.333223 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:51.309146 31244 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.155s	user 0.020s	sys 0.011s
I20260709 12:25:51.308444 31307 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.320103 31165 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.323360 31000 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.322796 31237 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.348897 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.304617 31261 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:51.333175 31095 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.367040 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:51.354467 26046 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.304991 31283 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:25:51.367115 26051 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:51.368721 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.364454 26052 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:51.366611 25952 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 9 to 12, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 12 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:51.346020 31315 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:51.367506 31328 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.095s	user 0.011s	sys 0.013s
I20260709 12:25:51.332866 30918 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.364868 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.365113 31320 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.327589 30831 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:25:51.411413 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.384598 31008 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.344693 30857 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.390023 31247 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:51.406383 26045 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: [11.6-13.7]
W20260709 12:25:51.410128 31231 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.529s	user 0.018s	sys 0.003s
I20260709 12:25:51.389727 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:51.414209 31269 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:51.414680 26124 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:51.414803 31318 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:51.415612 26048 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:25:51.416133 31352 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:51.392824 30838 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.390287 31314 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:51.461149 31314 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.461596 31314 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:51.469005 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2310) took 2281 ms. Trace:
W20260709 12:25:51.469364 31122 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.001s
I20260709 12:25:51.470779 31312 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.390846 31315 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.471505 31315 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:51.469532 26128 rpcz_store.cc:276] 0709 12:25:49.187184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.934385 (+747201us) service_pool.cc:224] Handling call
0709 12:25:49.961096 (+ 26711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.991204 (+1030108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.002931 (+ 11727us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.046316 (+ 43385us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.341964 (+295648us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:51.342005 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.353495 (+ 11490us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.468773 (+115278us) raft_consensus.cc:1685] finished
0709 12:25:51.468794 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.468941 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118984,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":1202807,"threads_started":2,"wal-append.queue_time_us":347455}
I20260709 12:25:51.389357 26054 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:51.364717 31198 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.475435 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:51.475867 31198 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.427866 26125 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:25:51.476307 31008 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.476749 30838 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.478155 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:51.428339 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.388902 31237 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:51.419486 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 94025.9us
I20260709 12:25:51.479282 30838 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.480122 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:51.481066 31237 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.414270 31362 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:51.440455 30918 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.440501 31123 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:51.486219 31362 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.442464 31351 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:51.443883 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3089) took 2037 ms (client timeout 2156 ms). Trace:
W20260709 12:25:51.444113 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 472592us
I20260709 12:25:51.442924 31320 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.352059 31238 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.356068 31244 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:51.449940 31366 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.363s	user 0.000s	sys 0.003s
I20260709 12:25:51.430258 25952 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 9 to 12, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 12 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.457163 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.458971 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.459226 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.500744 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 884) took 4578 ms. Trace:
W20260709 12:25:51.460012 31317 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.379s	user 0.022s	sys 0.015s
I20260709 12:25:51.460552 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.481796 31274 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.484793 31198 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:25:51.483356 31008 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:51.483291 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.387828 31095 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.484157 31352 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:51.485839 31123 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.486724 31362 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.445304 31365 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:51.492887 31129 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.494361 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:51.559425 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 145470us
I20260709 12:25:51.421983 31247 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.447044 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1170) took 1873 ms. Trace:
I20260709 12:25:51.495659 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:51.489975 26127 rpcz_store.cc:269] 0709 12:25:49.406511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.617472 (+210961us) service_pool.cc:224] Handling call
0709 12:25:51.443622 (+1826150us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54001f77d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:51.443804 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460092800}
I20260709 12:25:51.506148 31370 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:51.506650 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1183) took 1829 ms. Trace:
I20260709 12:25:51.509096 31238 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.511313 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.513841 31244 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.513856 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 861) took 1970 ms. Trace:
I20260709 12:25:51.565140 26784 rpcz_store.cc:276] 0709 12:25:49.573015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.573123 (+   108us) service_pool.cc:224] Handling call
0709 12:25:51.446691 (+1873568us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54004f4554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:51.446870 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3863667072}
I20260709 12:25:51.515401 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.516389 31269 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.519248 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.482473 26046 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:51.543069 26125 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:25:51.542954 26050 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:51.495151 30831 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:51.576702 26122 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:25:51.557337 31212 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.581131 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.581241 26791 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:51.542531 26126 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:25:51.517791 31000 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.469861 31122 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:51.583266 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.503038 26782 rpcz_store.cc:276] 0709 12:25:46.877719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.878319 (+   600us) service_pool.cc:224] Handling call
0709 12:25:46.878579 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.878793 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:46.937093 (+ 58300us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.689982 (+1752889us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.269049 (+1579067us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:50.803149 (+534100us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.294444 (+491295us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.329587 (+ 35143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.355488 (+ 25901us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:51.389048 (+ 33560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.421065 (+ 32017us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.455689 (+ 34624us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23140,"prepare.queue_time_us":1965020,"prepare.run_cpu_time_us":7643,"prepare.run_wall_time_us":46282,"raft.queue_time_us":288370,"raft.run_cpu_time_us":835,"raft.run_wall_time_us":73368,"spinlock_wait_cycles":13808000,"thread_start_us":4222321,"threads_started":6,"wal-append.queue_time_us":1625494,"wal-append.run_cpu_time_us":1292,"wal-append.run_wall_time_us":1001625}
I20260709 12:25:51.494585 31283 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.573450 26121 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:51.574855 26051 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:25:51.496445 31056 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:25:51.578387 26124 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:25:51.576733 26052 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:51.565030 31365 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:51.567327 31247 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:51.574144 26129 rpcz_store.cc:276] 0709 12:25:49.543548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.162195 (+618647us) service_pool.cc:224] Handling call
0709 12:25:50.192445 (+ 30250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.192883 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.192922 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.865707 (+672785us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.385125 (+519418us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:51.385197 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.392317 (+  7120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.513476 (+121159us) raft_consensus.cc:1685] finished
0709 12:25:51.513498 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.513658 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91193,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":411747,"threads_started":1,"wal-append.queue_time_us":605795}
I20260709 12:25:51.579316 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:51.586899 26051 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 7
I20260709 12:25:51.581063 26128 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:51.584146 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.590972 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.591825 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.593844 26783 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.595582 31238 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:51.597124 31238 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:51.597736 31238 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:51.599422 31238 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.601866 31238 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.577903 31269 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.605655 26051 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:51.605824 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.567322 26786 rpcz_store.cc:276] 0709 12:25:49.597700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.071867 (+474167us) service_pool.cc:224] Handling call
0709 12:25:50.875544 (+803677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.900206 (+ 24662us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:50.905061 (+  4855us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.340441 (+435380us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.391225 (+ 50784us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.391522 (+   297us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:51.391946 (+   424us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.405549 (+ 13603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.405639 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.424555 (+ 18916us) raft_consensus.cc:1685] finished
0709 12:25:51.424575 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.426880 (+  2305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41495,"prepare.queue_time_us":1083,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":3117,"raft.queue_time_us":84802,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":17560,"thread_start_us":420898,"threads_started":2,"wal-append.queue_time_us":151538}
I20260709 12:25:51.610795 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.569667 31352 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:51.610505 26051 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.575953 30780 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:51.605934 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.594568 26049 raft_consensus.cc:2376] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15. Re-sending same reply.
I20260709 12:25:51.606992 26783 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:51.613343 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.583436 30831 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:51.614256 26126 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:51.604949 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1799) took 2333 ms. Trace:
W20260709 12:25:51.615899 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 121699us
W20260709 12:25:51.617925 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 69667.3us
I20260709 12:25:51.619269 26051 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:51.629987 26053 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:51.619683 31223 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:51.620977 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:51.621727 26050 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:51.622323 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.634569 26046 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.586386 31317 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:51.629432 25952 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 10, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.634555 26093 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:51.645169 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 161962us
I20260709 12:25:51.645327 26127 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:51.585906 31129 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:51.653681 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58480.6us
I20260709 12:25:51.631958 30831 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:51.664068 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.664578 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.631570 31328 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:25:51.588745 31212 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.675117 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.675797 26124 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.630374 26047 rpcz_store.cc:276] 0709 12:25:49.272065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.645469 (+373404us) service_pool.cc:224] Handling call
0709 12:25:49.646000 (+   531us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:50.892805 (+1246805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.896376 (+  3571us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:51.002292 (+105916us) log.cc:844] Serialized 52 byte log entry
0709 12:25:51.347799 (+345507us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:51.579746 (+231947us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.587281 (+  7535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.592545 (+  5264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.592621 (+    76us) raft_consensus.cc:1685] finished
0709 12:25:51.592632 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.604888 (+ 12256us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":260305,"raft.queue_time_us":25736,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":37462,"thread_start_us":1651590,"threads_started":3,"wal-append.queue_time_us":420905}
W20260709 12:25:51.673770 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 116551us
I20260709 12:25:51.676617 26129 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:25:51.679450 31375 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:51.681274 26049 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:51.674635 30780 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.682585 30780 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.628784 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:51.689003 26128 raft_consensus.cc:606] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.689498 26128 raft_consensus.cc:614] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:51.693466 31000 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.694733 31365 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.697052 31365 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:51.698532 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.700028 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:51.701053 31297 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.978s	user 0.017s	sys 0.016s
I20260709 12:25:51.701573 31297 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.701200 31352 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.703744 31297 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.704725 26051 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 10.
W20260709 12:25:51.705684 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45195.9us
I20260709 12:25:51.708436 31122 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.667790 31317 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:51.716950 31317 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:51.718405 31317 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.726027 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.727453 26052 raft_consensus.cc:2410] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:25:51.729622 31123 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.731775 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:51.737097 31353 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:51.739018 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:51.741418 31231 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:51.742110 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:51.743528 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:51.744655 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:51.745172 30780 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.627967 31283 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.720289 31165 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.735714 31314 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.797748 26127 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:51.763837 25952 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 11, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 11 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:25:51.802313 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 101669us
I20260709 12:25:51.742108 31231 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:51.812605 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.811208 26046 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:51.662127 31269 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.822485 31269 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.822898 31387 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.823380 26120 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:51.792846 31297 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.824378 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:51.825237 26047 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.816627 31237 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
W20260709 12:25:51.831202 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105982us
W20260709 12:25:51.847703 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94490.3us
I20260709 12:25:51.840103 31095 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.861941 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:51.865792 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:51.869248 26122 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:51.869915 31283 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:51.870783 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 124249us
I20260709 12:25:51.872992 31314 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.875309 31247 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:51.864050 25952 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:51.823505 31387 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.846972 31320 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:51.881779 26124 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
W20260709 12:25:51.885771 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141283us
I20260709 12:25:51.861691 31198 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:51.887604 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.893241 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:51.894281 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:51.896332 31393 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.896894 31393 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.898128 26757 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:25:51.898468 26757 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:51.863154 31237 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.886833 25952 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 8 to 12, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:25:51.895504 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:51.906829 31387 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:51.909479 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.908950 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:51.872988 31129 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:51.911245 26018 connection.cc:256] RPC call timeout handler was delayed by -2.0610281649996978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 12:25:51.917526 26791 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:51.917768 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:51.918655 26783 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:51.919584 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.919986 31376 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:51.920459 31376 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:51.921190 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.921440 31376 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.922377 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.924175 31320 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:51.924597 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:51.924747 31320 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.925243 26128 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:25:51.927912 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62213.7us
W20260709 12:25:51.942550 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.949455 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51537.3us
I20260709 12:25:51.912492 30857 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:51.899104 31393 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:51.957590 25952 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.951771 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:51.914487 31269 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:25:51.903007 31314 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:51.927994 31297 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:51.969197 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:51.969558 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:51.966920 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:52.009883 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 121404us
I20260709 12:25:51.946374 31387 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:52.039042 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.908685 31312 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:51.995687 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:51.966517 30838 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:51.910691 31315 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.103420 26122 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:51.967212 30857 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.046062 31247 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:52.047904 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80498.6us
I20260709 12:25:51.912889 31237 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:51.912636 30918 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:25:51.908918 31231 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:52.061858 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 95022.9us
I20260709 12:25:52.065486 25952 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 9 to 10, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:52.065527 31387 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.068636 31381 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:52.075114 26783 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:52.075853 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.050987 31269 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 12:25:52.081115 31398 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:25:52.107123 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.063843 31165 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:52.115690 30831 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.104189 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:52.130409 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 199058us
I20260709 12:25:52.104918 31315 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:52.009718 31362 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.101732 31394 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:52.158327 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:52.132265 30857 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.090045 31317 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:52.175817 25952 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:52.175904 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71781.5us
W20260709 12:25:52.185207 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 239758us
I20260709 12:25:52.190328 26125 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:52.161345 31381 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:52.176273 30780 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:52.175418 31318 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:52.219040 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 155200us
I20260709 12:25:52.219165 26123 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:52.184612 31312 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.183245 31394 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:52.174543 31261 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:25:52.221545 25952 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 10, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:52.222316 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52333.8us
I20260709 12:25:52.188467 31317 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:52.203863 31381 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.217088 31198 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.212718 30780 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:52.214941 31318 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:52.219671 31401 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:52.223630 31394 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.224114 30838 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:52.241554 30831 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:52.241891 31397 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 12:25:52.243247 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.234258 31365 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:52.244496 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:52.244047 25952 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 10, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 10 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:52.277505 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49815.5us
I20260709 12:25:52.228937 31317 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:52.241147 31404 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:25:52.248108 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 184740us
I20260709 12:25:52.248469 31318 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.250119 31198 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.236917 30780 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.250144 31401 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.223412 31312 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.280382 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:52.287097 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.288698 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 130463us
W20260709 12:25:52.279628 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 43690.6us
I20260709 12:25:52.284149 25952 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 12, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 12 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:52.294873 31198 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:52.295475 31405 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:52.295879 31237 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.297899 31401 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:52.302273 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.303268 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:52.305830 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:52.305872 31320 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.307842 30838 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:52.308848 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.295950 31405 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.324867 25952 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 9 to 12, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 12 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:52.314219 31312 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.316727 31318 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.318266 31362 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.318715 31408 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:25:52.278635 31365 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.322256 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:52.331198 26790 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:52.312155 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.326007 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 88870.3us
I20260709 12:25:52.334292 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:52.329799 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49570us
I20260709 12:25:52.331302 25952 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 11, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 11 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:52.328859 31269 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:52.339556 31376 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.340435 31320 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.341315 31297 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:52.341460 31212 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:52.344805 31387 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.345630 31223 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:52.346247 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:52.329603 31405 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.352886 26790 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:52.354264 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.359093 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.333885 31365 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.360788 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:52.362721 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.364863 31416 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:25:52.354436 31269 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.369953 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.373765 31225 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:52.361055 30831 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.376452 31225 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:52.377374 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.377487 31225 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.378139 26126 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:25:52.379050 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.378918 31225 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:52.366616 31297 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:52.373008 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:52.333128 31394 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.380206 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.403045 26126 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:52.381762 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52397.2us
I20260709 12:25:52.382372 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.386636 26129 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:52.352578 31401 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:25:52.381337 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.406940 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:52.400725 31353 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.420302 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:52.383395 31407 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:25:52.405054 26125 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
W20260709 12:25:52.381250 31417 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:25:52.380738 31247 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:52.425436 26783 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:52.383518 31391 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.422956 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:52.430786 26126 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:52.412175 31129 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:52.423869 31353 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.417740 31410 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:52.429858 31393 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:52.426648 31247 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:52.444746 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63513.1us
W20260709 12:25:52.436793 26169 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 2.155s (SENT) [suppressed 31 similar messages]
I20260709 12:25:52.429203 31223 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:52.381632 31297 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:52.452138 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 97975.5us
I20260709 12:25:52.449714 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:52.451107 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:52.437395 31312 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:52.447582 31393 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.460896 31393 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.463742 30831 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.464949 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:52.465993 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.466936 30857 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.469090 31387 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:52.469653 31353 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.450837 31247 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.443980 31401 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:25:52.470597 31369 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:52.471056 31406 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:52.471652 31406 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:52.470104 31387 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:52.471107 31369 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:52.441540 31129 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:52.466971 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:52.434497 31391 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.477550 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:52.481730 31369 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.483345 31369 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.474390 31387 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.484563 31387 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.485644 31399 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:52.486109 31391 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.487248 31320 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:52.487850 31320 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:52.488703 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:52.488979 31320 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.491031 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.456597 31223 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:52.493955 31421 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:25:52.493674 31223 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:52.495296 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.495186 31416 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:52.496798 31408 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:52.497042 31416 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.497406 31408 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:52.473941 31395 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:25:52.473423 31352 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:52.489581 31269 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.499889 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.504321 26125 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:52.507313 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93088.4us
I20260709 12:25:52.507397 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:52.509333 31420 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:52.509578 31394 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.486102 31399 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:52.510555 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3098) took 1101 ms. Trace:
I20260709 12:25:52.510967 26120 rpcz_store.cc:276] 0709 12:25:51.409012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.409175 (+   163us) service_pool.cc:224] Handling call
0709 12:25:52.510338 (+1101163us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54001c4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:52.510498 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931116288}
I20260709 12:25:52.510905 31399 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.512773 31399 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.513276 31231 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.514564 31381 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.480278 31415 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.522753 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:52.508150 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 151763us
I20260709 12:25:52.523875 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.525849 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:52.528331 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.509480 31412 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.205s	user 0.000s	sys 0.003s
W20260709 12:25:52.530419 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64549.3us
I20260709 12:25:52.530673 26128 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:52.504170 31352 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.537120 31388 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:25:52.550645 31352 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.527280 31238 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:52.551790 31238 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:52.552248 31238 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.537277 31406 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.535940 31274 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:52.557958 31274 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.558459 31274 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:52.561491 31414 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.266s	user 0.000s	sys 0.002s
I20260709 12:25:52.527101 31312 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:52.535712 31129 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:52.548797 31423 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:25:52.549103 30780 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.549476 31198 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:52.556599 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:25:52.531103 31415 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:52.498421 31408 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.540771 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.567379 31362 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:52.561561 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:52.616307 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59836.4us
W20260709 12:25:52.569399 31424 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 12:25:52.621037 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:52.625138 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63673.3us
I20260709 12:25:52.547638 31247 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.600718 31415 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.600685 31312 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:52.640226 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.640336 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:52.643385 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.568162 30831 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:25:52.625154 31391 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:52.639339 31312 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.648126 31387 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:52.649160 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.568107 31393 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.641546 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:52.656534 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.657677 26127 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.662036 31403 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.663172 31405 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.670876 31391 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:52.672631 26127 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
W20260709 12:25:52.675937 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.676630 26045 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:52.634927 31297 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:52.685921 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.644629 30918 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:52.601079 31365 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.705397 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.706367 26200 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:25:52.710346 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.713629 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.714877 26200 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:25:52.718165 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.719671 26200 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:52.721213 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.722020 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.724052 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:52.726271 26199 raft_consensus.cc:2368] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:25:52.729719 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:52.730783 26199 raft_consensus.cc:2368] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:25:52.645323 31399 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:25:52.690153 31056 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:52.737911 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44308.9us
I20260709 12:25:52.695596 31297 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:52.778730 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107779us
I20260709 12:25:52.782543 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:52.764360 30857 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.763181 31056 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:52.805804 31247 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.813102 30831 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.813436 31247 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:52.813740 30831 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.814218 31247 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:52.820076 31247 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.820206 31056 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:52.821517 31432 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:52.821828 31438 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.822359 31438 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.807091 31095 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:52.843330 26203 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:52.829597 31387 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.857497 31406 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.843720 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:52.865335 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48857.7us
I20260709 12:25:52.866231 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:52.867650 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:52.868454 31362 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:52.868932 31362 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:52.869385 31362 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:52.870010 31422 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.202s	user 0.000s	sys 0.003s
I20260709 12:25:52.871340 31314 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:52.872134 31314 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.872313 31438 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.874092 26094 leader_election.cc:400] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:25:52.874221 30831 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.874490 26094 leader_election.cc:403] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:25:52.875252 30831 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:25:52.875766 31425 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:52.809653 31274 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:52.819420 31297 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.876216 31425 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:52.904786 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.905581 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.877575 26094 leader_election.cc:400] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:25:52.880430 26791 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:52.908769 26094 leader_election.cc:403] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:25:52.909061 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.886152 26200 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:52.913740 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.884675 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:52.892263 26195 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:52.904033 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:52.927775 26198 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:52.927343 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.929467 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:52.930130 26123 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.930236 26046 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:52.931937 26050 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:52.932317 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.933583 30831 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:52.934542 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:52.936558 26202 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:52.937512 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:52.937791 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60319.7us
I20260709 12:25:52.933887 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:52.940048 31381 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:52.939777 31129 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:52.940734 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:52.894053 31434 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:25:52.860507 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 173927us
I20260709 12:25:52.940944 31308 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:52.944061 26197 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:52.953512 26047 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:52.949385 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:52.953552 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.958748 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:52.861086 31405 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.889590 31274 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:52.961344 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:52.997974 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:52.926646 31393 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:52.971902 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.003784 26052 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:25:53.001772 26050 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:53.032526 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.033190 26046 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.961341 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.041389 26045 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:52.961748 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.029431 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:52.961505 31317 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.000s
I20260709 12:25:52.898928 31418 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.057412 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.886401 31362 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.062618 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.063055 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.042603 26048 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.059360 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.907158 31425 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:52.864589 31352 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.000198 31365 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:52.941530 31238 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.077826 26127 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.078671 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:52.948444 31308 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:53.073134 26053 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:53.081852 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:53.082206 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.083772 26122 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:53.002120 31351 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:52.891155 31095 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.076565 26052 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:53.084080 26202 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:53.084764 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.095858 26203 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:53.073873 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.090018 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.090267 31415 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:53.091185 31431 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 12:25:53.079712 31198 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:53.093132 31425 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.003360 31381 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:53.093401 31405 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.093762 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.094393 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:53.103401 26195 raft_consensus.cc:2368] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:53.103698 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:53.104084 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.136540 26052 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:53.109104 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.094336 31231 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:53.059751 30918 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:53.082093 31274 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.092602 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:53.118995 26197 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.121150 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.127609 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.066010 31312 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:53.155048 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51443.6us
I20260709 12:25:53.100284 31351 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:53.076608 31317 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.139014 26196 raft_consensus.cc:2376] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13. Re-sending same reply.
I20260709 12:25:53.158843 26200 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:25:53.159216 26125 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:53.159358 26199 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:25:53.169175 26128 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:53.169552 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.171783 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:53.121949 31365 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:53.193504 31365 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.164575 31056 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:53.202180 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.205749 31274 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.179878 30857 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:53.209303 31423 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:53.210981 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.213558 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.214494 26047 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:53.215723 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.186755 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.188151 31351 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.190701 31317 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.191296 31238 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:53.192257 26121 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:53.191994 30918 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.254045 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.254868 26202 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.167820 31231 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.160048 31405 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.193961 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.217370 30780 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.218218 31315 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:53.227254 31308 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.152637 31381 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.083681 31418 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:53.186584 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.273231 26200 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.241009 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.212051 26194 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:53.247761 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.283387 26045 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:53.250125 31351 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.250748 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.284574 26196 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:25:53.286415 26201 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.291364 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.303869 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.304906 26790 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:53.313093 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.306782 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.170331 26046 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:25:53.314968 26194 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:53.316211 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:53.316529 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:53.317914 26194 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:53.319193 26194 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:53.320024 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.245508 26127 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:53.318034 26783 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.246757 31362 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.330744 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:53.332937 26203 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:53.337369 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:53.338616 26127 raft_consensus.cc:2410] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:25:53.346428 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.158969 31198 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:53.349174 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.353452 26790 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:53.354077 26791 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:53.358865 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:53.358958 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.359469 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.360255 26046 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:53.360913 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:53.361650 26047 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:53.278913 31315 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:53.365368 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:53.367909 26201 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:53.376868 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:53.313433 31401 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.379644 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.382608 26128 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:25:53.306882 31095 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:53.384378 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.411226 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.445758 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.316888 31274 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:53.443188 26202 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:53.468886 26047 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:25:53.432924 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.437067 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.441527 26052 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:53.386634 26200 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:53.443709 26123 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:25:53.482750 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:53.316610 31129 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 12:25:53.383838 31352 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.483232 30780 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.488277 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.490715 26120 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:53.491325 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 126071us
I20260709 12:25:53.492853 26047 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:53.495942 26125 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:53.513291 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.468832 31308 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.502211 26050 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:53.355062 31370 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 12:25:53.511616 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.545265 26195 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.510404 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:53.511615 26791 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:53.561256 26202 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.562965 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:53.563633 26123 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.565044 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.566080 26203 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.566891 26198 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:53.560012 26783 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:53.578789 26196 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:53.579962 26121 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:25:53.580513 26121 pending_rounds.cc:85] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
I20260709 12:25:53.580788 26121 pending_rounds.cc:107] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:25:53.581094 26121 raft_consensus.cc:2889] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:53.581889 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71572.3us
I20260709 12:25:53.493863 31391 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:25:53.506069 31274 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:53.582260 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.589078 26195 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.585809 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:53.590505 26200 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:53.593191 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:53.486609 31095 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
W20260709 12:25:53.607293 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.020s	sys 0.043s
I20260709 12:25:53.607486 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:53.607839 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.020s	sys 0.044s
I20260709 12:25:53.610064 26050 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:25:53.588225 31122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.020s	sys 0.034s
I20260709 12:25:53.487466 31401 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.606194 31450 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:25:53.620143 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.620702 26200 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:53.562080 31315 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:53.641908 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.652997 26196 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:25:53.606586 31274 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:53.600091 26123 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:53.571955 30780 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.617478 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.748713 26202 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:25:53.696158 26203 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:53.592823 31441 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:25:53.674889 31399 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:53.674069 31433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.015s	sys 0.026s
W20260709 12:25:53.738637 31446 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:25:53.728317 26050 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:53.637717 31352 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:25:53.746186 31381 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.747761 31401 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.749187 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:53.759426 26048 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.673s	user 0.005s	sys 0.006s
I20260709 12:25:53.725886 31122 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:25:53.767701 26048 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
W20260709 12:25:53.759140 31433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.016s	sys 0.028s
I20260709 12:25:53.775341 26046 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:25:53.827244 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.758188 31450 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:53.769413 31370 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.766520 31297 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.871623 31451 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 12:25:53.872592 31455 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:25:53.771365 31198 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.881681 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:53.884985 26167 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.675s (SENT)
I20260709 12:25:53.886834 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:53.892724 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:53.892900 26200 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:53.769629 31399 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:53.898083 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.891775 26201 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
W20260709 12:25:53.740389 30831 consensus_queue.cc:1175] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 13 status { last_received { term: 12 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:25:53.913647 31448 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 12:25:53.911962 31403 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.899046 31415 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:53.916002 31461 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.142s	user 0.002s	sys 0.000s
I20260709 12:25:53.908455 31274 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.914299 31418 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.880754 31370 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:53.934345 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 49414.3us
I20260709 12:25:53.885104 31297 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.923671 31441 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:53.925822 31312 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:53.940062 31403 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.975780 26195 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
W20260709 12:25:53.981362 31459 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:53.986147 31231 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:53.986423 30780 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:53.994905 31471 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.995750 31471 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:53.957854 31450 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:53.999136 31457 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:54.001962 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:54.003135 31457 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:54.003656 31457 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.003724 31365 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.004326 31457 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.011536 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:54.012890 31370 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:54.015177 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.015311 31308 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:54.016134 31308 consensus_queue.cc:1243] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:54.016739 31308 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:53.990849 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3060) took 1033 ms. Trace:
I20260709 12:25:54.017244 26051 rpcz_store.cc:276] 0709 12:25:52.957487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.957667 (+   180us) service_pool.cc:224] Handling call
0709 12:25:53.002337 (+ 44670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.002700 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.002732 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.062784 (+ 60052us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.488946 (+426162us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:53.488988 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.489066 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.990618 (+501552us) raft_consensus.cc:1685] finished
0709 12:25:53.990639 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.990786 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":375435,"threads_started":1,"wal-append.queue_time_us":601809}
I20260709 12:25:54.018870 31399 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:53.989420 26791 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:54.020437 31362 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:54.020982 31362 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:54.020558 31457 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.021903 31362 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.022949 31457 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.023223 31362 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.024641 31403 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.025520 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3061) took 1067 ms. Trace:
I20260709 12:25:54.016601 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.026080 26049 rpcz_store.cc:276] 0709 12:25:52.957848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.957984 (+   136us) service_pool.cc:224] Handling call
0709 12:25:53.028934 (+ 70950us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.082510 (+ 53576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.082542 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.117037 (+ 34495us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.672364 (+555327us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:53.672435 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.672571 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.025162 (+352591us) raft_consensus.cc:1685] finished
0709 12:25:54.025195 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.025423 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1444,"raft.queue_time_us":21607,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":19931,"thread_start_us":493767,"threads_started":1,"wal-append.queue_time_us":732578}
I20260709 12:25:54.028348 31095 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.029093 31095 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:54.029376 31297 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.030282 31095 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:54.030869 31095 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:54.030802 31320 raft_consensus.cc:670] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 12:25:54.036343 31314 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:54.032078 31095 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.037637 31391 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:54.038125 31391 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:54.038345 31095 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.039094 31391 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:54.040823 26756 connection.cc:256] RPC call timeout handler was delayed by -1.8807273470001746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 12:25:54.043717 26756 connection.cc:256] RPC call timeout handler was delayed by -1.8564273470001353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 12:25:54.044224 26756 connection.cc:256] RPC call timeout handler was delayed by -1.8069273470000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 12:25:54.044715 26756 connection.cc:256] RPC call timeout handler was delayed by -1.7763273470000058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 12:25:54.045125 26756 connection.cc:256] RPC call timeout handler was delayed by -1.7160273469999083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 12:25:54.045543 26756 connection.cc:256] RPC call timeout handler was delayed by -1.5819273470001463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
W20260709 12:25:54.045946 26756 connection.cc:256] RPC call timeout handler was delayed by -1.4001273470003071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 12:25:54.046350 26756 connection.cc:256] RPC call timeout handler was delayed by -1.380327347000275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 12:25:54.046801 26756 connection.cc:256] RPC call timeout handler was delayed by -1.3785273470002721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
W20260709 12:25:54.047216 26756 connection.cc:256] RPC call timeout handler was delayed by -1.2507273470000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 12:25:54.042846 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:54.051211 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:54.062937 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.056931 31457 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:54.065989 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.067297 31165 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.068004 31165 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:54.074491 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:54.057155 31453 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.162s	user 0.005s	sys 0.000s
I20260709 12:25:54.053498 31391 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.051928 31471 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.078670 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.081905 26047 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:54.080186 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.084781 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:54.084873 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.087366 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:54.088775 26200 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:54.088985 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.092410 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1837) took 1140 ms. Trace:
I20260709 12:25:54.092985 26198 rpcz_store.cc:276] 0709 12:25:52.939329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.939796 (+   467us) service_pool.cc:224] Handling call
0709 12:25:52.940100 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.567297 (+627197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.078696 (+511399us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54002b4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:54.078897 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440845696}
I20260709 12:25:54.095279 26051 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:54.095988 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.097056 26194 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:25:54.126178 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.131007 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:54.131664 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.128537 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.131518 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:54.131019 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.081403 31231 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.133797 31308 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:54.134476 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.134672 31308 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:54.135154 31308 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:54.135596 31308 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.135704 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:54.136714 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.140093 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3063) took 1169 ms. Trace:
I20260709 12:25:54.095279 31403 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:25:54.141199 31237 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:54.141628 31351 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.836s	user 0.014s	sys 0.000s
I20260709 12:25:54.141855 31095 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:54.145972 31362 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:54.133816 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.134182 31231 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.150321 26045 rpcz_store.cc:276] 0709 12:25:52.970672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.489372 (+518700us) service_pool.cc:224] Handling call
0709 12:25:53.546178 (+ 56806us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.570930 (+ 24752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.570961 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:53.585278 (+ 14317us) log.cc:844] Serialized 52 byte log entry
0709 12:25:53.927524 (+342246us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:53.931024 (+  3500us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.931285 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.931380 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.139793 (+208413us) raft_consensus.cc:1685] finished
0709 12:25:54.139821 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.140016 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9032,"prepare.queue_time_us":3274,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":28506,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":24949,"thread_start_us":310192,"threads_started":1,"wal-append.queue_time_us":490445}
I20260709 12:25:54.171306 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:54.173997 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.181113 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.186193 26048 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:25:54.130235 31471 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:54.188894 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.129251 31405 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:54.192564 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:54.191237 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136649us
I20260709 12:25:54.191596 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:54.195225 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.198417 26050 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:54.199909 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.173110 31237 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:54.203336 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:54.201836 26201 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:25:54.208937 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 267027us
I20260709 12:25:54.202948 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.210456 26048 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.190382 31369 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:54.212944 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84565.9us
I20260709 12:25:54.192339 30857 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.213843 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.191228 31471 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:54.195585 31362 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.195044 31472 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:54.201550 31095 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.196219 31457 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:54.212601 31365 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:54.231637 31231 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.208630 31237 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.205516 31405 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:54.219020 31471 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:54.245460 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57701.3us
I20260709 12:25:54.219297 31274 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:54.217342 31129 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:54.220281 31362 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.248036 26048 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:54.248869 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.249439 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.259881 26200 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:54.234504 31441 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:54.250245 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.237058 31095 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:54.261276 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72500.4us
I20260709 12:25:54.262080 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.262773 31274 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:54.263056 31095 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.239002 31472 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.263974 31274 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.265513 31472 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.266840 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.267942 31472 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:54.245437 31405 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.268201 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.268747 31472 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:54.268744 31405 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.268949 26199 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
W20260709 12:25:54.270391 31473 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:54.269918 31472 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.260838 31362 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.276110 31481 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:54.278113 31425 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:54.278589 31425 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:54.279706 31425 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.278092 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:54.281632 31481 raft_consensus.cc:2764] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:54.283013 31481 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:54.285320 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.286175 31481 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:54.286753 31481 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:54.287675 31481 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.293274 31480 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:54.295219 31480 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:54.295684 31480 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:54.296098 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:38246 (request call id 0) took 1092 ms. Trace:
I20260709 12:25:54.296756 31480 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:54.297899 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.298581 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.298929 31362 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.302050 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.302444 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:54.302809 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:54.303457 31362 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.298265 31095 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.304193 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.304567 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.304821 26050 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.305653 31365 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:54.306100 31095 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.306173 31365 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:54.302840 31405 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:54.307327 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59192.4us
I20260709 12:25:54.307299 31365 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.261977 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.232388 30857 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.296538 26053 rpcz_store.cc:276] 0709 12:25:53.204200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.724821 (+520621us) service_pool.cc:224] Handling call
0709 12:25:54.295739 (+570918us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b5400172354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743ab735 00007f5b743a39fa 00007f5b7936b4c9 00007f5b728e7c54 00007f5b728e7be2 00007f5b728e7b11 00007f5b728e7a8c 00007f5b728e6d42 00007f5b726e048c 00007f5b726df8c6
0709 12:25:54.295991 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65075584}
I20260709 12:25:54.315675 26050 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:54.320894 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.334975 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:54.337415 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.338184 26128 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:25:54.338625 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:54.298599 31480 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.340476 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.340561 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.350486 26125 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:54.351408 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.351935 26125 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:25:54.341521 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91308.1us
I20260709 12:25:54.342120 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.342929 26122 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:54.353801 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.354513 26122 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:54.353842 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.359715 26124 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:54.343638 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.246898 31231 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 12:25:54.358717 26167 leader_election.cc:336] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.716s (SENT)
I20260709 12:25:54.344169 31405 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.332140 31353 raft_consensus.cc:2764] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:25:54.374329 26019 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:25:54.379876 26019 leader_election.cc:403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:25:54.374830 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.377738 26122 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:54.353935 31403 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.327999 31381 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:25:54.355832 31312 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.389746 26122 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:25:54.390283 26122 pending_rounds.cc:85] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 11
I20260709 12:25:54.411051 26122 pending_rounds.cc:107] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:25:54.411463 26122 raft_consensus.cc:2889] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:54.419003 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47771us
I20260709 12:25:54.420051 26019 leader_election.cc:400] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:54.420403 26019 leader_election.cc:403] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:25:54.424325 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65662us
W20260709 12:25:54.425108 26167 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.760s (SENT)
I20260709 12:25:54.425496 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.414301 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:54.381387 31438 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:54.338320 31479 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:54.435725 26127 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
W20260709 12:25:54.436348 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.436630 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:54.436177 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:54.439139 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 559871us
W20260709 12:25:54.441407 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.441998 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.445948 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:54.450650 26128 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
W20260709 12:25:54.453686 26167 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.775s (SENT)
I20260709 12:25:54.433233 30780 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:25:54.438350 31438 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:54.445499 31403 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.449277 31435 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:54.497370 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43701.7us
I20260709 12:25:54.411197 31312 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.451857 31297 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:54.506310 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 165912us
I20260709 12:25:54.463536 31320 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.452533 31415 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.463156 31450 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 10. Current term is 11.
I20260709 12:25:54.390223 31457 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:54.513010 26167 leader_election.cc:336] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.866s (SENT)
W20260709 12:25:54.516281 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60113.2us
I20260709 12:25:54.463449 31370 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:25:54.501127 31403 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:54.524776 26018 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.716s (SENT)
I20260709 12:25:54.530822 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:54.491547 31462 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:25:54.492913 31274 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:25:54.515983 31369 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:54.515621 31297 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.520787 31320 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.524115 31415 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:54.541252 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 121390us
I20260709 12:25:54.540876 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.514907 31479 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:25:54.547947 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.533056 31403 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.552348 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.554662 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:54.556746 31485 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:54.561247 31312 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:54.556568 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.562242 26051 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:54.561204 31308 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:54.545442 31274 consensus_queue.cc:1243] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:54.548168 31415 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:54.571925 26167 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.920s (SENT)
I20260709 12:25:54.540875 31198 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.573309 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.586086 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.544297 31297 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.587277 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.588188 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:54.592527 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97643.5us
I20260709 12:25:54.545897 31320 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.619453 31231 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:25:54.664126 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.664904 26051 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.620112 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.666522 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:54.622395 26047 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:54.624612 31315 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.625244 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:54.667740 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 149389us
I20260709 12:25:54.541152 31450 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:54.568729 31458 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:54.598590 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.630667 26120 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:54.672169 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.672972 26048 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.676677 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:54.677425 26783 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.677744 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:54.638759 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:54.628420 31484 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:25:54.628521 31369 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:25:54.643158 31308 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.548692 31438 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.644362 31462 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:54.628325 31353 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:54.644654 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 90089.4us
I20260709 12:25:54.645291 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:54.645939 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74046.9us
W20260709 12:25:54.646855 31490 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:54.648147 31008 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:54.648471 31497 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:54.649724 31487 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 12:25:54.652599 31486 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:25:54.652917 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.652998 31297 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.739584 31462 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.746711 31497 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:54.538144 31471 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.757730 31297 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.757325 31497 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.758074 31462 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:54.690213 26018 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.915s (SENT)
I20260709 12:25:54.758761 31497 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:54.759013 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68826.2us
I20260709 12:25:54.759250 31471 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:54.760618 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:54.761088 26018 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.940s (SENT)
I20260709 12:25:54.652705 31416 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:54.701639 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.702385 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.703868 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:54.761488 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.712178 26783 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:54.733244 26051 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:54.733328 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:25:54.733882 26167 leader_election.cc:336] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.156s (SENT)
I20260709 12:25:54.733186 26052 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.671142 31458 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:54.701023 31238 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:54.685814 31231 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.693949 31450 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:54.775348 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:54.751586 31237 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.703394 31314 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:54.782243 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48992.6us
I20260709 12:25:54.783093 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:54.775782 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:54.761425 26050 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:54.761598 26047 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:25:54.765962 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64428.1us
I20260709 12:25:54.790369 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.769326 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:54.769511 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:54.791616 26124 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.772440 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:54.776042 26049 raft_consensus.cc:2376] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11. Re-sending same reply.
W20260709 12:25:54.769474 31495 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:54.805927 26045 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:25:54.812216 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.762478 31416 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:25:54.813697 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.817715 26053 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.816606 26123 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:54.768319 31458 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.776047 31312 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:54.821957 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52708us
I20260709 12:25:54.825344 26129 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:54.788520 31438 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.804659 31365 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:54.835642 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45411.2us
I20260709 12:25:54.842808 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.821118 31314 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.847257 31314 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:54.805099 31238 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:54.779449 31297 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.849087 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:54.849924 26782 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:54.849610 31416 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.851432 31416 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:54.830308 31491 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:54.821914 31008 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.831964 31481 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.776870 31403 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:54.855948 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:54.856843 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.882642 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 213941us
I20260709 12:25:54.870669 26052 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:54.885016 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:54.894834 26124 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:54.899933 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.900876 26195 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 11: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:54.899937 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:54.905093 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:54.905858 26195 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:54.908140 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 269457us
I20260709 12:25:54.886651 31480 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.873874 31441 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:54.899379 31405 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:54.898378 31457 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:54.928385 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:54.928901 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:25:54.927357 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82842.2us
I20260709 12:25:54.888473 31471 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:25:54.930401 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.875764 31238 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:54.930754 26201 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:54.940694 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:54.943895 26195 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.909299 31509 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:54.946332 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:54.919589 31480 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:54.955281 26167 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.793s (SENT)
I20260709 12:25:54.956640 26195 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:54.927419 31457 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:54.967332 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:54.995072 31512 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.995810 31512 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:54.996994 31504 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:25:55.007424 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87542.1us
I20260709 12:25:54.964624 31499 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.967365 31480 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:55.013985 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 124337us
I20260709 12:25:54.968578 31457 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:55.013487 31505 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:25:55.044756 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 283668us
I20260709 12:25:54.998781 31479 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.998360 31509 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:55.048405 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:55.049333 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:54.955245 30918 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:25:55.069506 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:55.069653 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 286625us
W20260709 12:25:55.056675 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.076597 26124 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:55.044668 31237 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:55.078119 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.072392 26053 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:55.075539 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:55.065013 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.079497 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.080104 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:55.082093 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:55.049175 31499 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:25:55.082808 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 127555us
I20260709 12:25:55.049680 31481 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:55.078750 31514 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:55.085156 31008 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:55.085989 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.088593 31381 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:55.087534 26167 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.875s (SENT)
I20260709 12:25:55.098490 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:55.074712 31450 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:55.093461 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50775.5us
I20260709 12:25:55.078274 31297 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:55.101804 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:55.068791 31405 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:25:55.059684 31479 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.105778 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:55.106396 26124 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.097545 31471 raft_consensus.cc:2764] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:25:55.108845 31521 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:55.095983 30857 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:55.133455 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47544.8us
I20260709 12:25:55.134421 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:55.109408 31510 raft_consensus.cc:2764] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:55.105189 31297 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:55.106499 31403 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:55.144402 26120 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:25:55.144840 26120 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 7
I20260709 12:25:55.145077 26120 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:55.145300 26120 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.146283 26122 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:55.147087 26128 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:55.148797 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:55.149873 26128 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:55.158385 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:55.170666 26123 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:25:55.179862 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.175906 26124 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:55.176024 26123 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
I20260709 12:25:55.182505 26123 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:25:55.182811 26123 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.152397 31381 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:55.192291 26756 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:25:55.198695 31519 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:55.199314 31519 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:55.200285 31519 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:55.205443 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.205117 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:55.159966 31518 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.137674 31509 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.173540 31499 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.217710 31512 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:55.189284 31528 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:55.221165 31528 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:55.221338 31510 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:55.221848 31231 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:55.221951 31510 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:55.222501 31510 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.224892 31529 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:55.161047 31521 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:25:55.191408 31403 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:55.208398 31515 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:25:55.239006 26199 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:25:55.248427 26199 pending_rounds.cc:85] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 10
I20260709 12:25:55.178382 31008 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:25:55.175297 31517 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:25:55.190682 31481 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.258522 31502 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:55.194847 31369 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:25:55.250260 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.266590 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.248996 26199 pending_rounds.cc:107] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:25:55.221598 31528 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:55.270710 31526 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.261765 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.267491 26782 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:25:55.249307 31320 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260709 12:25:55.269807 26199 raft_consensus.cc:2889] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.259035 31502 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:55.274574 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.249569 31497 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:55.284042 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:25:55.316653 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.389591 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.439909 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 283060us
I20260709 12:25:55.448711 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:55.449025 26783 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 11.
I20260709 12:25:55.236362 31297 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.273682 31521 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:55.455075 26786 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:55.456065 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:55.456558 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.466061 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.468600 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.271127 31528 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.276813 31320 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:55.472067 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 199557us
I20260709 12:25:55.474767 31315 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:55.475337 31315 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:55.475467 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.476286 26167 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.620s (SENT)
I20260709 12:25:55.476382 31315 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.476779 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:55.477882 31231 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:55.478835 31472 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:55.482064 31403 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:55.482723 31403 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:55.472359 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.450696 31479 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:55.489389 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.318399 31497 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.271533 31526 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.492012 26788 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:55.459022 31531 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.505823 31462 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:55.506474 31462 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:55.507432 31410 raft_consensus.cc:2764] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:25:55.508472 30780 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:55.511272 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.512809 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 436183us
I20260709 12:25:55.513589 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.514806 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:55.514856 26124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:25:55.515770 26124 raft_consensus.cc:606] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:55.516290 26124 raft_consensus.cc:614] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:55.517944 31498 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:55.519785 31520 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.532111 26126 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.522073 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3144) took 1181 ms. Trace:
I20260709 12:25:55.495434 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.532944 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:55.467264 31237 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:55.460934 31535 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.532637 26129 rpcz_store.cc:276] 0709 12:25:54.340828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.341252 (+   424us) service_pool.cc:224] Handling call
0709 12:25:54.341598 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.837464 (+495866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.521755 (+684291us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54001f95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:55.521996 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112107392}
I20260709 12:25:55.534783 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:55.521116 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:55.545382 26167 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.836s (SENT)
I20260709 12:25:55.559402 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:55.564824 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43816.3us
I20260709 12:25:55.570609 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:55.537189 31353 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.002s
I20260709 12:25:55.567714 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.465330 31521 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.537420 31237 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.521097 31352 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:55.520610 31523 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:55.575425 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59185.8us
I20260709 12:25:55.584285 26125 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:55.584736 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.584957 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.530182 31528 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:55.589264 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.520855 31450 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:55.583324 26788 raft_consensus.cc:2368] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16.
I20260709 12:25:55.483975 31403 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.461246 31381 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.521193 31399 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:25:55.590366 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.573235 31517 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:25:55.593161 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47839.2us
I20260709 12:25:55.598912 26126 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:55.573792 31369 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:55.584805 31237 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.584487 31353 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.629712 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.586597 31352 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:55.633644 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 159367us
W20260709 12:25:55.632838 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 158712us
W20260709 12:25:55.621308 31415 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.962s	user 0.011s	sys 0.002s
I20260709 12:25:55.595561 31198 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:55.595947 31365 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:55.644073 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:55.469820 31502 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.556885 31520 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.647498 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.649839 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.623570 31008 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.531071 31531 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.677908 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:55.685673 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:55.595810 30918 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:25:55.567981 31535 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.630053 31523 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.683234 26784 raft_consensus.cc:2368] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16.
I20260709 12:25:55.641232 31352 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.620625 31351 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:25:55.695776 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50870.8us
I20260709 12:25:55.690423 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:55.697773 26018 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.483s (SENT)
I20260709 12:25:55.691465 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:55.698282 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:55.692786 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.679554 30831 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:55.694152 31315 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:55.702570 26201 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:55.713544 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:55.720983 31425 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.738385 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:55.739152 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:25:55.744270 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:55.752579 26126 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.690028 31403 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.757339 26120 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:55.630669 31369 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:55.761646 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 161186us
I20260709 12:25:55.762970 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:55.763671 26053 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:55.719947 31514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.018s	sys 0.008s
I20260709 12:25:55.648928 31415 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.768034 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.772284 26791 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.772709 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:55.682407 31508 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.086s
W20260709 12:25:55.769369 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71627.2us
I20260709 12:25:55.682847 31365 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.690100 31008 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.748502 30831 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:55.764024 31523 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.793023 26053 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:55.763746 31315 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:25:55.796481 31507 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:25:55.796840 26197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:25:55.850479 26197 raft_consensus.cc:606] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:55.798812 31362 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.865813 26053 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:55.804137 31438 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.964s	user 0.011s	sys 0.000s
I20260709 12:25:55.800470 31450 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:55.800334 31462 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.801321 31525 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.805120 31534 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 12:25:55.771824 31514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.018s	sys 0.008s
I20260709 12:25:55.791368 26051 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:55.845686 26049 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:25:55.848687 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.786283 31471 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.889595 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:55.781684 31517 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:55.911175 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:55.917649 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.839893 31497 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:55.849365 31533 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.852917 31320 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:55.886160 31312 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.943019 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:55.947925 26051 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:55.910676 31514 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:55.940524 31462 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.932379 31095 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:55.967360 26049 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:55.979724 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.894901 31362 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:55.939455 31526 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:55.942327 31521 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:55.925614 31471 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.000882 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.998688 26789 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.952103 31308 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:55.996690 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.000521 31532 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
I20260709 12:25:55.952744 31481 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:55.919888 31525 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.028570 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.025480 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.025660 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.029479 26050 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:56.030086 26784 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:55.996378 31535 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:55.953755 31312 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:56.023737 31541 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.335s	user 0.001s	sys 0.000s
W20260709 12:25:56.033299 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 435261us
I20260709 12:25:56.042891 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:55.982137 31095 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:55.996590 31554 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.232s
I20260709 12:25:56.061414 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.068816 31553 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:25:56.042984 31537 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.083020 26047 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:56.070088 31525 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:56.072978 31510 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.850s	user 0.013s	sys 0.000s
I20260709 12:25:56.073176 31369 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:56.087761 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.073871 31514 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:56.074520 31416 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.223s	user 0.015s	sys 0.000s
I20260709 12:25:56.074751 31520 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.032039 31315 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
W20260709 12:25:56.076598 31458 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.238s	user 0.007s	sys 0.007s
I20260709 12:25:56.076506 31415 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:56.028386 31531 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:56.076853 31518 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.836s	user 0.009s	sys 0.002s
I20260709 12:25:56.007766 30831 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.062816 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.077451 31471 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.081269 26054 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:56.052271 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:56.073294 31362 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:56.132146 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 86854.6us
I20260709 12:25:56.133294 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:56.134462 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:56.135033 31520 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.135418 31554 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:56.135851 31554 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.135548 31520 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.136289 31554 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:56.137760 31520 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.138324 31520 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:56.130385 31399 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.139505 31399 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.140657 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.141614 31502 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:56.093617 31510 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:56.088565 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.089772 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 249623us
I20260709 12:25:56.086580 26791 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:25:56.035737 31481 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.102943 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.105590 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.038231 31317 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:56.012143 31517 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.108153 31518 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:56.104832 31458 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.187450 31458 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.113396 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.114342 26052 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.115594 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.190380 26788 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:25:56.117274 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:56.121172 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.120712 31558 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.002s
I20260709 12:25:56.132416 31576 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.163156 26785 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:56.151539 26126 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:25:56.153839 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 238471us
I20260709 12:25:56.195675 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.153549 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:56.169080 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.173477 31095 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.174113 31481 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.146067 31237 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.176038 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.188366 31546 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:25:56.178539 31352 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.128830 31008 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:56.194746 31362 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.184643 31528 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 12:25:56.251911 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.092828 31537 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.192634 26049 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:56.220808 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1833) took 1917 ms. Trace:
I20260709 12:25:56.255685 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:56.144182 31510 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.225307 26783 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:56.257298 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.256146 26046 rpcz_store.cc:276] 0709 12:25:54.303491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.308370 (+  4879us) service_pool.cc:224] Handling call
0709 12:25:54.308708 (+   338us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:54.322037 (+ 13329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.322070 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:54.333149 (+ 11079us) log.cc:844] Serialized 78 byte log entry
0709 12:25:54.613545 (+280396us) spinlock_profiling.cc:243] Waited 9.7 ms on lock 0x7b1800272860. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:54.785879 (+172334us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:54.820802 (+ 34923us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.505045 (+684243us) spinlock_profiling.cc:243] Waited 976 us on lock 0x7b1800272860. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:25:55.868979 (+363934us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.887192 (+ 18213us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.220408 (+333216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.220503 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.220543 (+    40us) raft_consensus.cc:1685] finished
0709 12:25:56.220555 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.220727 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39354,"prepare.queue_time_us":474635,"prepare.run_cpu_time_us":172,"prepare.run_wall_time_us":1724,"raft.queue_time_us":844007,"raft.run_cpu_time_us":522,"raft.run_wall_time_us":18602,"spinlock_wait_cycles":24547840,"thread_start_us":827625,"threads_started":4,"wal-append.queue_time_us":539072}
I20260709 12:25:56.162278 31425 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:56.188537 31480 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.114s	user 0.013s	sys 0.001s
I20260709 12:25:56.227362 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:56.268476 31480 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.269824 31352 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.270665 31480 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.271565 31480 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:56.271946 31480 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:56.230818 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.321017 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.322227 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.323272 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.250274 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.250886 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.324379 26126 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:56.324702 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.325852 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.326606 26791 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:56.214326 31471 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.253543 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.177892 31457 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.128s	user 0.008s	sys 0.004s
I20260709 12:25:56.327908 31471 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.328550 31457 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.330168 26785 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:56.330852 31457 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.331379 26785 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:56.331573 26128 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:56.248273 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.332446 26122 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:56.260697 26787 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:25:56.260923 31548 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:56.263103 31562 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:25:56.263749 31572 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.266486 31369 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:56.215565 31576 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:56.272261 26018 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.578s (SENT)
W20260709 12:25:56.227298 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71348.4us
W20260709 12:25:56.337182 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 64974.7us
I20260709 12:25:56.274507 31312 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:56.275856 26167 leader_election.cc:336] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 3.320s (SENT)
I20260709 12:25:56.276542 26127 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:56.342187 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.278772 31399 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.249827 31481 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:25:56.230607 31458 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:56.284981 26199 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:25:56.289352 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.290313 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:56.332371 31441 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.258387 31362 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.333379 31480 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:56.377306 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:56.375981 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.298331 31554 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.305569 31405 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:56.307772 31410 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:56.383019 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.384567 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:25:56.311008 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.307796 31567 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.362764 31403 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 12:25:56.333218 26784 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:56.332917 26788 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:56.281879 31425 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.372151 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:56.372985 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 97158.6us
W20260709 12:25:56.375043 26018 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.658s (SENT)
I20260709 12:25:56.410475 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.292528 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.347141 31577 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:25:56.386209 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.389496 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:56.347817 31552 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.451s	user 0.002s	sys 0.000s
I20260709 12:25:56.419332 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.391197 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3152) took 1247 ms. Trace:
I20260709 12:25:56.419122 26194 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:56.420491 26127 rpcz_store.cc:276] 0709 12:25:55.143938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.144055 (+   117us) service_pool.cc:224] Handling call
0709 12:25:55.144322 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.342298 (+1197976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.390972 (+ 48674us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:56.391127 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603171584}
I20260709 12:25:56.421414 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:56.422798 26198 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:25:56.388413 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.423543 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.376451 31458 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:56.410275 26167 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.773s (SENT)
I20260709 12:25:56.412449 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.378926 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:56.425045 26783 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:56.369851 31537 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:56.336086 31471 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.433151 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.395560 26122 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:56.392627 26201 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:56.381853 31405 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:56.415803 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:56.392365 31410 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:56.416518 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.370437 31369 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:56.360808 31237 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.230001 26786 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:25:56.364598 31576 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.426102 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.482511 26194 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:25:56.483454 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.484094 26194 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:56.484160 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.484918 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.392923 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.485351 26045 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:56.485615 26782 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:56.427790 26167 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 3.895s (SENT)
I20260709 12:25:56.382069 31438 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.489777 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.379719 31514 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.394785 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.490444 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:56.380550 31441 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.361135 31365 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.390278 31435 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:56.463372 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.463872 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 85037us
I20260709 12:25:56.464244 26202 raft_consensus.cc:2368] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16.
I20260709 12:25:56.457839 26049 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:56.406548 31518 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:56.444963 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:56.566715 26128 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:56.467018 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.410537 31425 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:56.470047 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.567585 26046 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:56.362957 31584 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:56.471197 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.569459 26783 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:56.472427 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.473917 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.337003 31548 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.476277 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.578737 26053 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:56.487222 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.579313 26051 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:56.583236 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.583901 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:56.488408 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.343513 31572 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.489900 26122 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:56.587342 26048 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:25:56.434428 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59404.2us
I20260709 12:25:56.406364 31008 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:25:56.428218 31458 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:56.432765 30780 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:56.598873 26127 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:56.429211 26790 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:25:56.490779 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.432577 30857 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:56.431735 31353 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:56.576149 31549 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.598713 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:56.527504 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.613843 26122 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:56.545960 31441 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:56.482097 31521 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:25:56.548755 31369 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.549636 31554 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.473668 31095 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:56.550010 31590 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:56.497987 31438 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.550304 31514 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.677935 31462 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:25:56.694335 31537 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:56.635377 31353 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.694737 31537 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:56.695565 31514 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.410244 31567 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.695639 31537 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.696672 31353 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.697078 31537 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:56.697355 31095 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:56.698264 31567 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.698406 31095 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.696090 31438 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.711266 26128 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:25:56.652592 31549 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.713505 26050 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:56.714519 31549 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.557467 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.734822 26199 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:56.736737 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.737339 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:56.738276 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:56.738804 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:56.467896 31415 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:56.576939 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 149170us
I20260709 12:25:56.584574 26052 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:56.743937 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.744609 26052 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:25:56.745420 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:56.745942 26052 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 16
W20260709 12:25:56.746541 26167 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.810s (SENT)
I20260709 12:25:56.585121 26045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:25:56.748811 26045 raft_consensus.cc:606] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.749179 26045 raft_consensus.cc:614] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:56.586395 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.347391 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:56.749949 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:56.606711 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:56.607004 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:56.607288 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.493846 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:56.610294 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.599684 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:56.612035 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.614323 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.615052 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:56.752102 26049 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:56.650508 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.753829 26199 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:56.618412 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:56.754343 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.585386 31365 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.553390 31533 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:25:56.593927 31576 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.613838 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.680431 26047 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:56.754647 26787 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:25:56.684794 31521 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:56.678143 31410 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.693089 30831 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:56.729063 26200 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:25:56.731917 26201 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:56.605425 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:56.756934 26120 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:56.758436 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 264672us
I20260709 12:25:56.759109 26121 raft_consensus.cc:2368] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16.
W20260709 12:25:56.760321 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 160767us
I20260709 12:25:56.762454 26198 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:56.770978 26052 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:56.771950 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.772643 26052 raft_consensus.cc:2376] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16. Re-sending same reply.
I20260709 12:25:56.562480 26196 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
W20260709 12:25:56.678939 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66051.7us
I20260709 12:25:56.764102 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.769381 26203 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: [11.10-18.15]
I20260709 12:25:56.789986 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.790977 26788 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:56.795011 26051 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:56.610194 31563 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.147s
I20260709 12:25:56.834520 26049 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
W20260709 12:25:56.827651 25938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 480498us
I20260709 12:25:56.769299 31438 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:56.829707 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:56.851408 26203 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.585927 31435 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.838927 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.763283 31521 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:56.769730 31519 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:56.845780 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.800722 31415 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.811863 31410 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.837939 31238 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:56.811051 30780 consensus_queue.cc:1175] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 9 index: 5 } }
I20260709 12:25:56.877432 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.879120 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.864940 31519 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:56.849375 31596 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:56.769459 31365 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.789306 31576 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:56.865166 31415 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:56.890197 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.860747 26045 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:56.884763 31519 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:56.916060 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.897307 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:56.906354 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:56.910519 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:56.917677 26203 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:25:56.918103 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.918817 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.918974 30831 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:56.919517 30831 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.919929 30831 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:56.921065 31597 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:25:56.928807 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:56.910694 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.911532 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.931854 31601 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:56.932647 31576 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:56.932510 31237 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:56.932911 31601 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:56.934867 31601 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:56.935810 31438 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:56.936306 31438 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:56.937194 31438 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.938975 31438 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:56.942106 26783 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:56.941391 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:56.948554 26049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:25:56.949558 26049 raft_consensus.cc:606] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:56.952428 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:56.953039 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:56.954066 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:56.958704 26782 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:56.961598 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:56.935846 31572 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:56.977026 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:56.977888 31458 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.979015 31514 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:56.886121 31480 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:56.995187 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66466.1us
I20260709 12:25:56.997347 26788 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:56.997749 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.000815 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.001796 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.002136 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:57.002966 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:57.009603 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.010424 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.011698 26019 leader_election.cc:400] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16.
I20260709 12:25:57.012012 26019 leader_election.cc:403] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:56.954677 31497 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:57.018512 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.017915 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.025620 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:57.025776 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.028077 26124 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:57.025391 31554 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:57.031610 31567 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:57.032313 31399 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.034408 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:57.036607 31416 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:57.037074 31416 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.037461 31416 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.038589 31317 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:57.039253 31317 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.039649 31317 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:57.046221 26197 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:57.005777 31514 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:57.056921 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 63311.8us
I20260709 12:25:57.057771 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:57.051635 26203 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 12:25:57.053509 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.059119 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.061415 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.018054 31441 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.889227 31521 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:56.960717 31578 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:25:57.017334 31314 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:57.017145 31353 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.023011 31523 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:57.055411 31567 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:57.098634 26783 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:57.100844 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.101507 26203 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.060972 31514 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:57.100169 26782 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:57.027387 31549 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:25:57.121353 31435 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.122265 31435 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:57.123512 31567 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:57.124801 31567 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.106962 26782 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:57.130148 31458 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.093312 31314 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.131347 31314 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.133024 31416 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.104645 31599 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:57.093675 31523 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.137193 31523 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.136998 31317 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.139201 31317 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.074221 31274 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:25:57.106402 31353 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:57.108680 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.109256 31508 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:57.109946 31365 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.113343 31480 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.044512 30857 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:25:57.163440 26785 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:25:57.074730 31584 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:57.141839 31238 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:57.169873 31508 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.145951 31416 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:57.180509 31508 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.095345 31498 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.145728 26197 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:25:57.159003 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.174420 26203 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:57.168169 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.175396 26782 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:57.184019 26782 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:25:57.152817 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.169107 31274 consensus_queue.cc:1243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:57.145576 31441 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:57.185864 26196 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.181285 31498 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.188611 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:25:57.190708 31498 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:57.193665 31425 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.000s
I20260709 12:25:57.190712 31497 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:25:57.194576 26196 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:57.019800 30831 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.196455 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.114979 31399 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16.
I20260709 12:25:57.241569 31508 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.270731 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:57.241693 31317 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:57.247975 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1197) took 1663 ms (client timeout 1866 ms). Trace:
W20260709 12:25:57.273489 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 254756us
W20260709 12:25:57.273864 26786 rpcz_store.cc:269] 0709 12:25:55.584357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.584711 (+   354us) service_pool.cc:224] Handling call
0709 12:25:57.247805 (+1663094us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b540056aed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:57.247918 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481803648}
I20260709 12:25:57.248817 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1208) took 1251 ms. Trace:
I20260709 12:25:57.254349 31584 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:25:57.140221 31548 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.257701 30831 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.257788 31314 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.235692 31518 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.269675 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:57.289332 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:25:57.290544 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:57.267884 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:57.291973 26051 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:57.257567 31362 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.271349 26048 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:25:57.266122 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:57.267108 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:57.274957 26790 rpcz_store.cc:276] 0709 12:25:55.997370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.000590 (+  3220us) service_pool.cc:224] Handling call
0709 12:25:57.248636 (+1248046us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54004501d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:57.248760 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979619712}
I20260709 12:25:57.275477 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:57.296615 26124 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:57.296972 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:57.298040 26048 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:25:57.264997 30918 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:57.309481 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.277272 26756 connection.cc:256] RPC call timeout handler was delayed by -2.5328384569997979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 12:25:57.312723 26756 connection.cc:256] RPC call timeout handler was delayed by -2.3429384569999456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 12:25:57.314580 26756 connection.cc:256] RPC call timeout handler was delayed by -2.2619384569998147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:25:57.315228 26756 connection.cc:256] RPC call timeout handler was delayed by -2.1404384569996182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260709 12:25:57.315752 26756 connection.cc:256] RPC call timeout handler was delayed by -1.9955384569998387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 12:25:57.316138 26051 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:25:57.316270 26756 connection.cc:256] RPC call timeout handler was delayed by -1.9442384569997557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 12:25:57.317297 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.286737 26050 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:25:57.278573 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 257207us
I20260709 12:25:57.278185 31462 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.322204 26120 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:57.303385 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:57.326519 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.331343 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:57.336738 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.337198 31548 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.338814 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:57.313913 31518 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.238255 31425 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.320083 30918 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.329756 31543 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:25:57.347689 31523 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.360143 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:57.362529 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59265.1us
I20260709 12:25:57.374998 26052 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:25:57.379383 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:57.364424 26051 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
I20260709 12:25:57.380345 26790 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.381770 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:57.367381 26049 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:57.270757 31508 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.362839 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:25:57.354890 31519 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.301448 31472 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:57.368636 31543 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.383610 26124 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:57.351761 26020 leader_election.cc:400] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16.
W20260709 12:25:57.404821 26784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 907) took 1274 ms (client timeout 1578 ms). Trace:
W20260709 12:25:57.389844 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72684.3us
W20260709 12:25:57.409319 26784 rpcz_store.cc:269] 0709 12:25:56.130286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.145542 (+ 15256us) service_pool.cc:224] Handling call
0709 12:25:57.404552 (+1259010us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400355a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:57.404726 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271123712}
I20260709 12:25:57.405561 26783 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.410835 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:57.412814 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61442.6us
I20260709 12:25:57.360137 31314 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.368777 31399 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.414711 31523 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.402613 31365 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:57.415356 31508 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.366961 31425 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.475199 31425 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:57.475843 31425 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.371265 31415 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.416381 31472 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:57.419245 31519 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.347886 31514 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.374151 31537 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:57.432135 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.449980 26790 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:57.366816 31518 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.371121 30918 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.313211 31481 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.089s
W20260709 12:25:57.459054 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 394203us
I20260709 12:25:57.468518 31458 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:57.469213 26783 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:25:57.473176 31605 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:25:57.371958 31584 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:57.481083 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70388us
I20260709 12:25:57.364089 31369 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 12:25:57.500183 31365 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:57.528517 31369 consensus_queue.cc:1243] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:57.529165 31369 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:57.529488 31369 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:57.529557 31365 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.530359 31369 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.531635 31369 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.480067 31399 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.532661 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.533963 31399 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.364074 31548 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.360781 31462 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.534628 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.536154 31548 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.536571 31462 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:57.484174 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:57.537290 26784 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.538944 31425 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.543630 26194 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:57.543777 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.479077 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:57.492923 26786 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEARNER]: Advancing to term 15
I20260709 12:25:57.507628 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.508656 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.545838 26199 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:25:57.509500 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.510308 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.512045 26196 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [13.12-15.13]
W20260709 12:25:57.513779 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 511687us
I20260709 12:25:57.549163 31578 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.549227 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.549749 31578 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.486714 31472 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:57.524827 26198 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:25:57.554737 31405 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.559484 26784 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:57.536392 31458 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:57.550282 26124 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:57.561847 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:57.561884 31458 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:57.521974 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.563840 31462 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:57.547545 26201 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:25:57.568181 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.570034 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:57.570618 26121 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.565796 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:57.573917 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 94669.7us
I20260709 12:25:57.568842 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:25:57.576210 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.580884 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:25:57.584911 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:57.593017 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.552057 31578 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:57.577150 31480 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:57.576695 31519 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:57.575594 31462 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:57.607676 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:57.619904 26195 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:57.604283 31544 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:57.639585 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:57.652997 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.654060 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:57.656692 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:57.613466 31512 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.027s
W20260709 12:25:57.679332 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.679857 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.680764 31544 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:57.657940 31362 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:25:57.681469 31544 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:57.682597 31544 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:57.669838 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.013s	sys 0.011s
W20260709 12:25:57.689173 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.013s	sys 0.012s
I20260709 12:25:57.690375 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.691040 26201 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:57.691005 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:25:57.620553 31536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.017s
I20260709 12:25:57.693235 26199 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:57.694700 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.695320 26199 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:57.624809 31537 raft_consensus.cc:2764] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:25:57.619457 31462 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:57.618336 31519 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:57.734025 31519 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:57.737967 26194 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:57.739423 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.660656 31543 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:25:57.746870 26194 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:25:57.695250 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:25:57.625061 31480 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:57.613723 31518 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:25:57.652442 31317 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:57.711840 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:57.716042 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.725706 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.726097 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.726819 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.727306 31537 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:57.678789 31512 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.008s	sys 0.028s
I20260709 12:25:57.678740 31608 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:57.730289 31410 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.855s	user 0.008s	sys 0.003s
I20260709 12:25:57.613685 31525 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:25:57.731513 31462 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.732268 31549 raft_consensus.cc:993] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: : Instructing follower a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:25:57.733330 31514 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:57.751607 26786 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:25:57.698149 31536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.008s	sys 0.017s
I20260709 12:25:57.700770 31095 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:57.784199 26195 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:57.784401 26203 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:25:57.784989 26198 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:57.758572 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58657.8us
I20260709 12:25:57.761240 26121 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:25:57.776718 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81534.9us
W20260709 12:25:57.791097 26045 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.021s
I20260709 12:25:57.805904 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:25:57.836357 26045 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.006s	sys 0.021s
I20260709 12:25:57.838798 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
W20260709 12:25:57.748492 31535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.000s	sys 0.038s
I20260709 12:25:57.806770 26197 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:25:57.716574 31551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.003s	sys 0.023s
W20260709 12:25:57.758358 31479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.017s	sys 0.012s
I20260709 12:25:57.810485 31369 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:57.809207 31608 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:57.862542 26201 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:25:57.832317 31481 raft_consensus.cc:2764] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:25:57.840920 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.870177 26203 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.806833 31317 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.843784 26121 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:25:57.844249 26054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.011s	sys 0.015s
I20260709 12:25:57.817942 31607 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.865324 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:25:57.881162 26122 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:25:57.841635 31535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.004s	sys 0.040s
I20260709 12:25:57.832458 31410 raft_consensus.cc:1081] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Signalling peer a981c9a7972c4e99a0903924c64fbb5c to start an election
W20260709 12:25:57.848637 31551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.007s	sys 0.024s
I20260709 12:25:57.869385 26127 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:57.850167 31479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.018s	sys 0.013s
W20260709 12:25:57.878777 26054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.015s
I20260709 12:25:57.846177 31525 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:57.868121 31608 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:57.900164 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 469356us
W20260709 12:25:57.885082 26018 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.013s	sys 0.007s
W20260709 12:25:57.909346 26018 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.013s	sys 0.008s
W20260709 12:25:57.909870 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 557441us
I20260709 12:25:57.837285 31095 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.839946 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.884258 31528 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:25:57.860828 31531 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.878862 31317 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.911505 26195 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:57.917032 26018 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.904s (SENT)
I20260709 12:25:57.856629 31610 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:25:57.953926 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1846) took 1492 ms. Trace:
I20260709 12:25:57.961833 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.962007 26047 rpcz_store.cc:276] 0709 12:25:56.461724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.461885 (+   161us) service_pool.cc:224] Handling call
0709 12:25:56.462264 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.757597 (+295333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.953643 (+1196046us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54001b49d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:57.953859 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434008448}
W20260709 12:25:57.877956 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71867.2us
I20260709 12:25:57.964977 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:57.922333 26197 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:57.872926 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:25:57.938683 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:57.972793 26203 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:25:57.915679 26198 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:58.006181 26195 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:25:58.071149 26052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.004s	sys 0.036s
W20260709 12:25:58.074139 26052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.004s	sys 0.036s
I20260709 12:25:57.879668 31480 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:58.075711 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108899us
W20260709 12:25:58.076740 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.072747 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:25:58.089015 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150429us
I20260709 12:25:57.896377 31525 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:58.090814 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.921371 31609 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.960165 31381 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:58.095914 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:58.099529 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:25:58.100610 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:58.070264 31472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.013s	sys 0.012s
I20260709 12:25:57.939815 31518 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:57.966645 31095 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:58.097390 26046 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:58.093786 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:25:58.107033 26047 raft_consensus.cc:606] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:25:58.107391 26047 raft_consensus.cc:614] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:58.116036 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.018s	sys 0.013s
I20260709 12:25:57.956952 31425 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.138s
I20260709 12:25:58.105712 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:58.098230 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:58.214620 26049 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:58.117882 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.105207 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 188204us
W20260709 12:25:58.142282 26200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.019s	sys 0.044s
W20260709 12:25:58.216543 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.216799 26200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.019s	sys 0.044s
W20260709 12:25:58.218339 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.388s (SENT) [suppressed 25 similar messages]
W20260709 12:25:58.190828 26126 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.025s	sys 0.018s
W20260709 12:25:58.219586 26126 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.026s	sys 0.019s
I20260709 12:25:57.893014 31607 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:58.142970 31518 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:58.075277 31531 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:58.093174 31535 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:58.151113 26202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.013s	sys 0.028s
I20260709 12:25:58.094658 31525 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:58.186268 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:25:58.149200 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.019s	sys 0.014s
I20260709 12:25:58.245481 26047 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:58.247344 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:58.249325 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:58.170776 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:58.252008 26046 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:25:58.206568 26128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.008s	sys 0.027s
I20260709 12:25:58.225577 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:58.229048 26121 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:25:58.098771 31613 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.123s
W20260709 12:25:58.166675 26788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.017s	sys 0.024s
W20260709 12:25:58.258534 26788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.019s	sys 0.026s
I20260709 12:25:58.259387 26049 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:58.260800 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:58.262415 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:58.108174 31381 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:58.241034 26202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.019s	sys 0.028s
W20260709 12:25:58.255760 26128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.008s	sys 0.027s
W20260709 12:25:58.249513 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:25:58.209513 31549 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:25:58.237558 26053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.005s	sys 0.017s
W20260709 12:25:58.333948 26053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.005s	sys 0.017s
I20260709 12:25:58.217607 30857 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:25:58.266281 26125 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.018s	sys 0.022s
W20260709 12:25:58.334806 26125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.018s	sys 0.022s
I20260709 12:25:58.104295 31609 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:58.230108 31548 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:25:58.332782 25938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 83315.4us
I20260709 12:25:58.273394 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:58.349128 26197 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:25:58.239279 31308 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:25:58.240731 31535 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.238763 31458 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:58.250274 26195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:48534
I20260709 12:25:58.373309 26195 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:58.373742 26195 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:58.289079 26018 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.554s (SENT)
I20260709 12:25:58.291162 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3107) took 1021 ms. Trace:
W20260709 12:25:58.377018 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 87978.2us
W20260709 12:25:58.377930 26018 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.615s (SENT)
I20260709 12:25:58.376634 26045 rpcz_store.cc:276] 0709 12:25:57.269842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.269957 (+   115us) service_pool.cc:224] Handling call
0709 12:25:57.271046 (+  1089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.271255 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:57.836902 (+565647us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.290748 (+453846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.290807 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:58.290862 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.290933 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.291097 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":951695,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":69,"thread_start_us":1016843,"threads_started":2}
W20260709 12:25:58.440588 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61608.3us
W20260709 12:25:58.462741 26048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.005s	sys 0.017s
W20260709 12:25:58.463430 26048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.006s	sys 0.018s
I20260709 12:25:58.464496 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3110) took 1163 ms. Trace:
I20260709 12:25:58.294912 26047 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:25:58.301494 26049 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:58.276708 31425 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:25:58.325343 31544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.015s	sys 0.013s
I20260709 12:25:58.339709 26046 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:25:58.473825 26199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.008s	sys 0.015s
W20260709 12:25:58.474067 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 375904us
W20260709 12:25:58.511629 26199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.008s	sys 0.015s
I20260709 12:25:58.342079 30857 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:58.491684 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.012s	sys 0.032s
W20260709 12:25:58.523425 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.012s	sys 0.032s
I20260709 12:25:58.524587 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:58.525393 26049 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:58.528924 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:25:58.372617 31535 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:58.474294 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 208377us
I20260709 12:25:58.513828 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:25:58.487771 26195 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:58.523934 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 685890us
W20260709 12:25:58.493886 26018 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.610s (SENT)
W20260709 12:25:58.569445 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75584.3us
W20260709 12:25:58.571175 26018 leader_election.cc:336] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.353s (SENT)
I20260709 12:25:58.571700 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:25:58.608341 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.465047 26048 rpcz_store.cc:276] 0709 12:25:57.301315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.302639 (+  1324us) service_pool.cc:224] Handling call
0709 12:25:57.303041 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.303299 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:57.848594 (+545295us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.463982 (+615388us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.464075 (+    93us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:58.464144 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.464255 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.464427 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":920199,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":58,"thread_start_us":1102906,"threads_started":2}
I20260709 12:25:58.342484 31549 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:58.566675 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:58.510504 31544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.015s	sys 0.014s
I20260709 12:25:58.490732 31584 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.374789 31308 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:58.664736 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 93592.8us
I20260709 12:25:58.387161 31458 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:58.494352 31425 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:25:58.673337 26129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.013s	sys 0.004s
W20260709 12:25:58.681028 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.042s
W20260709 12:25:58.682364 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115646us
W20260709 12:25:58.729087 26129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.014s	sys 0.004s
W20260709 12:25:58.653133 31622 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.478s	user 0.000s	sys 0.001s
I20260709 12:25:58.733706 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3181) took 1155 ms. Trace:
I20260709 12:25:58.737843 26129 rpcz_store.cc:276] 0709 12:25:57.578378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.578508 (+   130us) service_pool.cc:224] Handling call
0709 12:25:57.578818 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.579013 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:58.108550 (+529537us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.733273 (+624723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.733341 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:58.733389 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.733478 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.733631 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3454,"prepare.queue_time_us":745900,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"thread_start_us":1115786,"threads_started":2}
W20260709 12:25:58.696980 31629 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.368s	user 0.001s	sys 0.000s
I20260709 12:25:58.388015 31548 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.263s
W20260709 12:25:58.701988 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.729192 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.002s	sys 0.043s
W20260709 12:25:58.665401 26120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.007s	sys 0.019s
I20260709 12:25:58.730108 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:58.699471 26052 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.005s	sys 0.015s
W20260709 12:25:58.750247 26120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.008s	sys 0.019s
W20260709 12:25:58.750542 26052 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.005s	sys 0.015s
I20260709 12:25:58.751310 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3182) took 1172 ms. Trace:
I20260709 12:25:58.751451 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 932) took 2065 ms. Trace:
I20260709 12:25:58.752118 26783 rpcz_store.cc:276] 0709 12:25:56.686016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.168448 (+482432us) service_pool.cc:224] Handling call
0709 12:25:57.405391 (+236943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.514553 (+109162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.751142 (+1236589us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400355a50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:25:58.751373 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544021248}
I20260709 12:25:58.751792 26120 rpcz_store.cc:276] 0709 12:25:57.579165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.579376 (+   211us) service_pool.cc:224] Handling call
0709 12:25:57.579638 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.579819 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:58.101341 (+521522us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.750939 (+649598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.751000 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:58.751067 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.751132 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.751255 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33606,"prepare.queue_time_us":793982,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"thread_start_us":1136699,"threads_started":2}
I20260709 12:25:58.730971 31518 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:58.736439 31544 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:58.785768 26203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.020s	sys 0.010s
W20260709 12:25:58.809934 26203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.020s	sys 0.010s
I20260709 12:25:58.698451 31584 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:58.818231 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:58.824080 26047 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:25:58.808059 31536 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.014s	sys 0.014s
W20260709 12:25:58.846738 26126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.009s	sys 0.001s
I20260709 12:25:58.797601 31365 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:58.847400 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.003s	sys 0.032s
I20260709 12:25:58.795043 26195 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:25:58.851182 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.003s	sys 0.032s
W20260709 12:25:58.847219 26126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.009s	sys 0.001s
I20260709 12:25:58.820130 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3185) took 1222 ms. Trace:
I20260709 12:25:58.857244 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3184) took 1268 ms. Trace:
I20260709 12:25:58.663443 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:25:58.847175 31536 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.015s	sys 0.015s
W20260709 12:25:58.838639 31624 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.540s	user 0.000s	sys 0.001s
W20260709 12:25:58.842907 31628 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.539s	user 0.000s	sys 0.001s
W20260709 12:25:58.843453 31646 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:25:58.892066 26201 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:25:58.862656 26126 rpcz_store.cc:276] 0709 12:25:57.588637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.588800 (+   163us) service_pool.cc:224] Handling call
0709 12:25:57.589145 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.589356 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:58.220102 (+630746us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.856751 (+636649us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.856830 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:58.856899 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.856994 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.857161 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24139,"prepare.queue_time_us":881350,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"thread_start_us":1220153,"threads_started":2}
I20260709 12:25:58.879189 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1199) took 1499 ms. Trace:
W20260709 12:25:58.901741 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 333277us
I20260709 12:25:58.904973 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:58.905198 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:58.906045 26786 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 12: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:58.859133 26125 rpcz_store.cc:276] 0709 12:25:57.597622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.597888 (+   266us) service_pool.cc:224] Handling call
0709 12:25:57.598206 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.598416 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:58.335318 (+736902us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.819420 (+484102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.819498 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:58.819724 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.819829 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.820048 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38825,"prepare.queue_time_us":933387,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":2086,"thread_start_us":1133261,"threads_started":2}
I20260709 12:25:58.906425 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:58.907001 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:58.907397 26790 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:25:58.907724 26786 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:25:58.908843 26782 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:58.909353 26782 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:25:58.910471 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:58.901374 26785 rpcz_store.cc:276] 0709 12:25:57.379652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.379775 (+   123us) service_pool.cc:224] Handling call
0709 12:25:57.384295 (+  4520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.384678 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.384727 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.404889 (+ 20162us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.873412 (+468523us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:57.873458 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.873531 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.878850 (+1005319us) raft_consensus.cc:1685] finished
0709 12:25:58.878882 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.879095 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29074,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"thread_start_us":457359,"threads_started":1,"wal-append.queue_time_us":868382}
I20260709 12:25:58.819754 31518 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:25:58.857136 31664 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:58.845613 31544 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:58.929759 26049 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:58.945899 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:25:58.917423 31536 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:58.867030 31548 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
W20260709 12:25:58.917613 31635 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.567s	user 0.000s	sys 0.002s
I20260709 12:25:58.852011 31531 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.003043 26782 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:58.957404 31648 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:25:58.960844 31632 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:25:58.964164 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2256) took 1250 ms. Trace:
W20260709 12:25:58.964442 31625 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
I20260709 12:25:59.038048 26199 rpcz_store.cc:276] 0709 12:25:57.714115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.714296 (+   181us) service_pool.cc:224] Handling call
0709 12:25:57.714792 (+   496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:57.841053 (+126261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.841100 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:57.841692 (+   592us) log.cc:844] Serialized 52 byte log entry
0709 12:25:58.512773 (+671081us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:58.512835 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.512927 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.963832 (+450905us) raft_consensus.cc:1685] finished
0709 12:25:58.963860 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.964084 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145107,"raft.queue_time_us":129117,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":11571,"thread_start_us":634251,"threads_started":1,"wal-append.queue_time_us":1100825}
W20260709 12:25:58.964881 31643 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 12:25:58.976678 31614 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:59.058168 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1859) took 1708 ms. Trace:
I20260709 12:25:58.980262 31365 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:58.981676 26789 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:25:58.935500 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:58.900084 31381 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:58.919245 31435 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.931344 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:58.999239 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:25:58.983007 31637 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.512s	user 0.000s	sys 0.002s
I20260709 12:25:58.815578 31425 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:59.008579 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 494775us
I20260709 12:25:59.082935 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:25:59.012492 26783 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
W20260709 12:25:58.956400 31653 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:25:58.957494 31639 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:25:59.029438 26782 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:58.977978 31462 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.166s	user 0.013s	sys 0.001s
W20260709 12:25:58.918867 31638 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.553s	user 0.000s	sys 0.002s
I20260709 12:25:59.058737 26051 rpcz_store.cc:276] 0709 12:25:57.349802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.381093 (+ 31291us) service_pool.cc:224] Handling call
0709 12:25:57.381508 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.381958 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.382000 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.386237 (+  4237us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.840789 (+454552us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:57.840843 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.840928 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.057895 (+1216967us) raft_consensus.cc:1685] finished
0709 12:25:59.057918 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.058091 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":454174,"threads_started":1,"wal-append.queue_time_us":881223}
I20260709 12:25:58.948709 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:25:59.061371 26049 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:58.982528 31657 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:58.982237 31664 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:58.990487 31416 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.078975 26785 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:25:59.084170 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:25:59.088485 26053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.018s	sys 0.033s
W20260709 12:25:59.067387 31660 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:25:59.093416 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3106) took 1824 ms. Trace:
I20260709 12:25:59.114323 26054 rpcz_store.cc:276] 0709 12:25:57.269496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.269615 (+   119us) service_pool.cc:224] Handling call
0709 12:25:57.273752 (+  4137us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.274033 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.274074 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.274622 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.895637 (+621015us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:57.895680 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.895949 (+   269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.093122 (+1197173us) raft_consensus.cc:1685] finished
0709 12:25:59.093152 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.093335 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":593256,"threads_started":1,"wal-append.queue_time_us":956013}
I20260709 12:25:59.074921 31668 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:59.118860 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3186) took 1520 ms. Trace:
I20260709 12:25:59.088887 31365 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:59.119335 26128 rpcz_store.cc:276] 0709 12:25:57.598496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.598657 (+   161us) service_pool.cc:224] Handling call
0709 12:25:57.599058 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.605109 (+  6051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.605138 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.605595 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.326601 (+721006us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:58.326640 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.326705 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.118579 (+791874us) raft_consensus.cc:1685] finished
0709 12:25:59.118607 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.118784 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5930,"raft.queue_time_us":7674,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8482,"thread_start_us":652795,"threads_started":1,"wal-append.queue_time_us":1077838}
W20260709 12:25:59.135216 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51158.3us
I20260709 12:25:59.029357 31536 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.106362 31095 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.933s	user 0.009s	sys 0.005s
W20260709 12:25:59.096741 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.100461 26786 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:25:59.105829 26788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.008s	sys 0.016s
I20260709 12:25:59.066098 31633 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:59.144548 26049 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:59.145246 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1858) took 1796 ms. Trace:
I20260709 12:25:59.145640 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:59.147112 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:25:59.112776 26053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.019s	sys 0.033s
I20260709 12:25:59.084920 31544 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:59.151443 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.153287 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:59.155110 26782 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
W20260709 12:25:59.141383 26788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.009s	sys 0.016s
I20260709 12:25:59.105508 31657 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:59.102186 31549 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.106693 31308 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.089362 31435 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.158991 26787 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:59.117326 31664 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:59.170743 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:59.192421 26199 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:25:59.176470 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.161226 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 431151us
I20260709 12:25:59.199188 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:59.200465 26202 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.005s	sys 0.006s
I20260709 12:25:59.147725 26049 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:25:59.145686 26052 rpcz_store.cc:276] 0709 12:25:57.349471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.376159 (+ 26688us) service_pool.cc:224] Handling call
0709 12:25:57.376630 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.406079 (+ 29449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.406113 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.422023 (+ 15910us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.082077 (+660054us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:58.750939 (+668862us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.144970 (+394031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.145040 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.145092 (+    52us) raft_consensus.cc:1685] finished
0709 12:25:59.145104 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.145201 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35907,"prepare.queue_time_us":142377,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":64,"raft.queue_time_us":159503,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":82171,"thread_start_us":1580954,"threads_started":3,"wal-append.queue_time_us":920351}
I20260709 12:25:59.202145 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:59.167809 26782 pending_rounds.cc:85] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 10
I20260709 12:25:59.105784 31462 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.197180 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1885) took 1690 ms. Trace:
I20260709 12:25:59.204970 26782 pending_rounds.cc:107] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:25:59.128892 31458 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.207211 26782 raft_consensus.cc:2889] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.208588 26782 pending_rounds.cc:107] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:25:59.193958 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1183) took 2822 ms. Trace:
I20260709 12:25:59.208969 26782 raft_consensus.cc:2889] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.200852 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:25:59.200995 26202 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.006s	sys 0.007s
I20260709 12:25:59.218730 26201 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:59.156378 31528 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.204797 26200 rpcz_store.cc:276] 0709 12:25:57.506670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.506839 (+   169us) service_pool.cc:224] Handling call
0709 12:25:57.523117 (+ 16278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:57.549000 (+ 25883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.549046 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:57.549716 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 12:25:58.217331 (+667615us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:58.217389 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.217496 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.196913 (+979417us) raft_consensus.cc:1685] finished
0709 12:25:59.196943 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.197116 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19514,"raft.queue_time_us":26860,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":667141,"threads_started":1,"wal-append.queue_time_us":752152}
I20260709 12:25:59.202416 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:25:59.222208 26125 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:59.209203 26788 rpcz_store.cc:276] 0709 12:25:56.337790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.066995 (+729205us) service_pool.cc:224] Handling call
0709 12:25:57.096698 (+ 29703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.096936 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:57.573848 (+476912us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.158177 (+1584329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.158311 (+   134us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:59.158403 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.158578 (+   175us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.159192 (+   614us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":745056,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":80,"raft.queue_time_us":837513,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":16934,"thread_start_us":2048028,"threads_started":3,"wal-append.queue_time_us":937850}
W20260709 12:25:59.232332 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.160039 31518 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.124109 31416 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.156085 31497 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:25:59.191382 31531 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:59.157011 31661 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:25:59.216869 31652 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
I20260709 12:25:59.249292 26124 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
W20260709 12:25:59.249862 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.197845 31548 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.131364 31668 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:59.158627 31544 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:59.207402 31508 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:59.254699 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53897.6us
I20260709 12:25:59.277110 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1898) took 1439 ms. Trace:
W20260709 12:25:59.238715 31608 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.316s	user 0.005s	sys 0.010s
I20260709 12:25:59.143280 31656 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
W20260709 12:25:59.233477 31632 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:59.232637 26052 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:25:59.233335 31549 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:59.157107 31095 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.159575 31633 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:59.305102 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3119) took 1030 ms. Trace:
I20260709 12:25:59.305701 26052 rpcz_store.cc:276] 0709 12:25:58.274978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.303870 (+1028892us) service_pool.cc:224] Handling call
0709 12:25:59.304363 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.304594 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.304636 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.304699 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:59.304757 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.304833 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.305012 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.236013 31528 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.261096 31531 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:59.277559 26198 rpcz_store.cc:276] 0709 12:25:57.837982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.078245 (+240263us) service_pool.cc:224] Handling call
0709 12:25:58.078656 (+   411us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:58.099221 (+ 20565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.099266 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:58.107230 (+  7964us) log.cc:844] Serialized 78 byte log entry
0709 12:25:58.659747 (+552517us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:59.113590 (+453843us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.115036 (+  1446us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.118234 (+  3198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.118312 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.276855 (+158543us) raft_consensus.cc:1685] finished
0709 12:25:59.276874 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.277035 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28894,"raft.queue_time_us":37591,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":55767,"spinlock_wait_cycles":18786688,"thread_start_us":860931,"threads_started":2,"wal-append.queue_time_us":689347}
W20260709 12:25:59.286707 31353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.014s	sys 0.011s
W20260709 12:25:59.299834 31519 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.552s	user 0.015s	sys 0.001s
I20260709 12:25:59.313827 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3120) took 1036 ms. Trace:
W20260709 12:25:59.315346 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57577us
I20260709 12:25:59.398612 26052 rpcz_store.cc:276] 0709 12:25:58.277680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.309225 (+1031545us) service_pool.cc:224] Handling call
0709 12:25:59.309653 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.310045 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.310091 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.313532 (+  3441us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:59.313575 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.313639 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.313772 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.264568 31657 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.399176 31353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.014s	sys 0.011s
I20260709 12:25:59.398167 31519 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:59.403982 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:59.404559 31694 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:59.404735 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:25:59.406482 31672 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.190s
I20260709 12:25:59.407550 31458 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:59.407579 31671 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:25:59.255993 31497 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.263142 31548 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.303288 31308 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:25:59.251758 31518 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.506160 31660 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:25:59.432741 31662 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:25:59.433470 31095 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.409899 31544 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.435086 31676 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:25:59.431275 31657 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:59.308454 31549 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:59.300432 31661 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:25:59.299878 31608 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.431480 31666 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.194s	user 0.001s	sys 0.001s
I20260709 12:25:59.532810 26126 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:59.486491 31462 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:59.263991 31365 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.441937 31531 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.655311 26049 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:25:59.715442 31458 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.806293 31554 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:59.806903 31554 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:59.806350 31458 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:25:59.807828 31554 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.809144 31554 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:59.809329 31458 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.786155 31416 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.789736 31548 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.813973 31633 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.814198 31686 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.758188 31536 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.814962 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1196) took 2436 ms. Trace:
I20260709 12:25:59.815383 26791 rpcz_store.cc:276] 0709 12:25:57.378565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.378740 (+   175us) service_pool.cc:224] Handling call
0709 12:25:57.404289 (+ 25549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.541297 (+137008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.541336 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.541819 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.963971 (+422152us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:57.964025 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.814556 (+1850531us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:25:59.814601 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.814645 (+    44us) raft_consensus.cc:1685] finished
0709 12:25:59.814673 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.814896 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":354,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"spinlock_wait_cycles":314125568,"thread_start_us":421752,"threads_started":1,"wal-append.queue_time_us":1306120}
I20260709 12:25:59.791942 31608 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:25:59.792706 31549 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.793980 31670 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
W20260709 12:25:59.794377 31650 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:25:59.833946 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1862) took 2482 ms. Trace:
I20260709 12:25:59.525849 31671 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.837739 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.837935 31671 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.797622 31677 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:59.839942 31677 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:25:59.798730 31668 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:59.800341 31669 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:25:59.800828 31317 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.731s	user 0.013s	sys 0.001s
I20260709 12:25:59.854213 31317 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.756785 31660 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:59.801419 31661 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:59.802021 31519 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.570098 31308 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:59.863749 31308 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.775161 31656 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.864893 31656 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:59.865476 31525 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.597s	user 0.013s	sys 0.002s
I20260709 12:25:59.805444 31497 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.834425 26053 rpcz_store.cc:276] 0709 12:25:57.351062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.684130 (+333068us) service_pool.cc:224] Handling call
0709 12:25:57.684526 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.684912 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:58.336582 (+651670us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.149845 (+813263us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.164782 (+ 14937us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.164933 (+   151us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:59.164968 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.165043 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.833699 (+668656us) raft_consensus.cc:1685] finished
0709 12:25:59.833720 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.833879 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":539185,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":22025,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":14279,"thread_start_us":1287754,"threads_started":2,"wal-append.queue_time_us":842801}
I20260709 12:25:59.796126 31518 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:59.860321 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.856976 31665 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.874140 31317 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.872326 31525 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.893669 31664 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.000s
I20260709 12:25:59.901371 31656 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.901556 31664 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.908385 31662 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:25:59.828104 31691 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.912356 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.015s	sys 0.015s
I20260709 12:25:59.841173 31677 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:25:59.916282 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.015s	sys 0.015s
I20260709 12:25:59.768224 31353 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:25:59.814849 31686 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.911104 31435 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:25:59.919133 31521 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:59.869304 31608 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.944528 31697 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.945569 31535 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.295s	user 0.012s	sys 0.002s
I20260709 12:25:59.945194 31697 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.946003 31535 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:25:59.947929 31535 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:25:59.921028 31691 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.965163 31525 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:59.967960 26167 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.012s	sys 0.011s
W20260709 12:25:59.970772 26167 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.013s	sys 0.011s
W20260709 12:25:59.972642 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 710607us
I20260709 12:25:59.966922 31528 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:25:59.975934 26203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.073s	sys 0.091s
W20260709 12:25:59.978955 26203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.073s	sys 0.091s
I20260709 12:25:59.967060 31689 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.940089 31608 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:25:59.949747 31697 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:25:59.978178 26051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.020s	sys 0.010s
W20260709 12:25:59.999157 26051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.021s	sys 0.010s
I20260709 12:25:59.999940 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3114) took 1762 ms. Trace:
I20260709 12:25:59.913483 31665 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:25:59.979429 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.017s	sys 0.008s
W20260709 12:26:00.001354 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.017s	sys 0.008s
I20260709 12:26:00.000422 31689 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:00.003304 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.977172 26167 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.685s (SENT)
I20260709 12:26:00.004729 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:00.006827 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:00.008603 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.000873 26051 rpcz_store.cc:276] 0709 12:25:58.237277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.108108 (+870831us) service_pool.cc:224] Handling call
0709 12:25:59.108543 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.114730 (+  6187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.114784 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.124935 (+ 10151us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.302218 (+177283us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:59.311433 (+  9215us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.999559 (+688126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.999639 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.999685 (+    46us) raft_consensus.cc:1685] finished
0709 12:25:59.999702 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.999853 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5646,"prepare.queue_time_us":13894,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":356090,"raft.run_cpu_time_us":577,"raft.run_wall_time_us":22592,"thread_start_us":796103,"threads_started":2,"wal-append.queue_time_us":252016}
I20260709 12:26:00.012919 31608 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:00.014623 31619 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.838s	user 0.006s	sys 0.026s
W20260709 12:26:00.014832 26167 leader_election.cc:336] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.206s (SENT)
I20260709 12:26:00.015286 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:00.015635 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:00.015847 31435 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:00.016601 31435 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:00.016921 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:25:59.979925 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.019s	sys 0.049s
I20260709 12:26:00.014709 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1897) took 2235 ms. Trace:
W20260709 12:26:00.018301 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.019s	sys 0.049s
I20260709 12:26:00.022607 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.033666 31584 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:00.034514 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.036099 31691 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.036441 31664 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.037307 31686 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.037195 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.038158 26198 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:26:00.038556 31689 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.034659 31674 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:00.049805 31674 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.050619 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.052788 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.055334 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:00.056006 26198 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.059374 31405 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.052922 31674 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:00.061719 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.006s	sys 0.023s
W20260709 12:26:00.062191 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.023s
I20260709 12:26:00.048564 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.041070 31497 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:00.062529 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 662578us
I20260709 12:26:00.010799 26051 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:00.060840 31664 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.077201 26166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.014s	sys 0.018s
W20260709 12:26:00.078009 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.012s	sys 0.040s
W20260709 12:26:00.078382 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.012s	sys 0.040s
W20260709 12:26:00.077734 26166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.014s	sys 0.018s
W20260709 12:26:00.085948 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.084652 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.092876 26198 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
W20260709 12:26:00.101382 26052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.005s	sys 0.031s
W20260709 12:26:00.101956 26052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.005s	sys 0.031s
I20260709 12:26:00.101548 31543 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:00.105087 26095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.015s	sys 0.014s
I20260709 12:26:00.104056 31620 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:00.126322 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:00.129671 31699 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:00.122344 26095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.016s	sys 0.014s
I20260709 12:26:00.130580 31699 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.100693 31497 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.102371 31674 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.133267 31472 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.140300 31621 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:00.140494 31472 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.140987 31621 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.143003 31472 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.143720 31621 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.079084 31664 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:00.136739 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.135335 31525 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:00.119884 31691 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.137182 26128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.015s	sys 0.012s
I20260709 12:26:00.151521 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:00.162042 26128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.015s	sys 0.012s
I20260709 12:26:00.162498 26125 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:00.132763 31611 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:00.168048 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.176652 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:00.177326 26198 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:00.178015 31672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.015s	sys 0.017s
W20260709 12:26:00.180145 31672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.015s	sys 0.017s
I20260709 12:26:00.181712 31708 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:00.182262 31708 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:00.182657 31708 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:00.160425 31535 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:00.193552 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.194310 26194 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:26:00.195276 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.195619 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:00.195925 26200 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:00.196219 26199 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:00.147814 31497 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.135403 31620 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:00.201375 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.006s	sys 0.005s
W20260709 12:26:00.201858 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.005s
I20260709 12:26:00.163336 31317 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:26:00.150689 31674 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.208187 26199 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:00.168623 31525 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:00.132480 31694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.007s	sys 0.024s
I20260709 12:26:00.209060 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.219164 26125 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:00.172340 31687 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:00.210628 26198 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:00.211185 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:00.217664 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.198791 31665 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.169274 31691 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.133117 31543 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.234438 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.173374 31611 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.230335 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.237854 26198 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.238008 26782 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:26:00.239421 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:00.231797 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:00.240527 26783 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:00.240720 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.241057 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.241588 26786 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:00.242503 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.232190 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.207513 31698 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.219233 31694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.008s	sys 0.026s
I20260709 12:26:00.235349 26788 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:00.224449 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:00.251855 26195 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.236797 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.244041 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:00.244891 31679 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.253468 26199 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.245124 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.245678 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.243150 26194 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:00.249837 26790 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:00.257589 26783 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:26:00.259008 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:00.257875 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.261627 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.262262 26786 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:00.263314 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.264282 26784 raft_consensus.cc:2368] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14.
I20260709 12:26:00.275063 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:00.276541 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.276715 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:00.234483 31608 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:00.278981 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.284878 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.275401 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.287046 26198 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:00.242671 31699 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.289222 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.294080 26782 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:00.291654 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.288652 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:00.290526 31531 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:00.298825 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:00.301638 31497 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:00.303506 26125 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:00.256582 31698 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.277092 31576 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:00.257639 31694 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:00.299209 31686 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:00.261512 31661 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:00.237320 31697 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:26:00.334964 26196 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:26:00.335443 26196 pending_rounds.cc:85] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 10
I20260709 12:26:00.305155 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:00.299144 31630 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.981s	user 0.008s	sys 0.028s
W20260709 12:26:00.321511 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.298712 31531 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.290349 31665 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:26:00.298602 31692 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:00.297514 31317 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:00.340047 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.308341 31699 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:00.370882 31698 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.336742 26196 pending_rounds.cc:107] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.11
I20260709 12:26:00.373013 26196 raft_consensus.cc:2917] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Aborting config change with OpId 14.11: Aborted: Op aborted by new leader
I20260709 12:26:00.312531 31576 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:00.372741 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81965.4us
I20260709 12:26:00.336516 31664 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.296795 31608 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.333947 31661 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:00.292217 31679 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.365346 31671 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:00.366762 31531 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.416925 26196 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.389497 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:00.373416 31711 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:00.342821 31567 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:00.438143 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.389585 31692 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.390584 31673 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:00.364789 31687 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:26:00.456276 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2374) took 1253 ms. Trace:
W20260709 12:26:00.480647 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 281263us
I20260709 12:26:00.418901 31699 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:00.451090 31709 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.432653 31317 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:00.451349 31661 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:00.462134 31613 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.460943 31671 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.493875 26784 raft_consensus.cc:2376] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12. Re-sending same reply.
I20260709 12:26:00.500211 26128 rpcz_store.cc:276] 0709 12:25:59.203359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.203607 (+   248us) service_pool.cc:224] Handling call
0709 12:25:59.203980 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.205016 (+  1036us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:59.237732 (+ 32716us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.455555 (+1217823us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.455826 (+   271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.455881 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.455931 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.456050 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.456195 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62562,"prepare.queue_time_us":42374,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":48589,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":39085,"thread_start_us":1168732,"threads_started":2,"wal-append.queue_time_us":1019570}
I20260709 12:26:00.533574 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1861) took 3182 ms. Trace:
I20260709 12:26:00.528645 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.488493 31543 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.516111 31611 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.526081 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1235) took 1618 ms. Trace:
I20260709 12:26:00.502473 31620 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.534011 26050 rpcz_store.cc:276] 0709 12:25:57.350733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.385926 (+ 35193us) service_pool.cc:224] Handling call
0709 12:25:57.408382 (+ 22456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.416397 (+  8015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.416433 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.416967 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.783031 (+366064us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:57.783094 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.783196 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.187393 (+2404197us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:00.519752 (+332359us) raft_consensus.cc:1685] finished
0709 12:26:00.519785 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.533506 (+ 13721us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59734,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":9970048,"thread_start_us":365636,"threads_started":1,"wal-append.queue_time_us":674939}
I20260709 12:26:00.543582 31725 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:26:00.551117 26198 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:00.559121 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.559556 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.560498 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.511114 31673 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:00.517589 31399 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:00.536768 31671 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.494706 31567 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.486543 31718 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:26:00.581435 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:00.581864 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 905) took 3263 ms. Trace:
I20260709 12:26:00.582258 26127 rpcz_store.cc:276] 0709 12:25:57.318780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.318960 (+   180us) service_pool.cc:224] Handling call
0709 12:25:57.319346 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.893753 (+574407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.581596 (+2687843us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54001d24d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:00.581804 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257448704}
I20260709 12:26:00.461539 31694 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.550921 26789 rpcz_store.cc:276] 0709 12:25:58.907842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.071467 (+163625us) service_pool.cc:224] Handling call
0709 12:25:59.071903 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.265061 (+193158us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:59.713211 (+448150us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.205276 (+492065us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.205493 (+   217us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.208788 (+  3295us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.208907 (+   119us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:00.209092 (+   185us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.209322 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.209392 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.525724 (+316332us) raft_consensus.cc:1685] finished
0709 12:26:00.525759 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.525971 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33498,"prepare.queue_time_us":723753,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":6746,"raft.queue_time_us":707364,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":41357,"spinlock_wait_cycles":69868416,"thread_start_us":981753,"threads_started":3,"wal-append.queue_time_us":470017}
I20260709 12:26:00.582201 26050 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:00.563992 31691 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.515394 31576 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.609027 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:00.537114 31613 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:26:00.532343 31709 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:00.602487 31405 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.005s
I20260709 12:26:00.623126 31679 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.624105 31576 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.625540 31567 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.609619 31725 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:00.611244 31317 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.628813 31725 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.631718 31725 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.603415 31674 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.632508 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.605134 31697 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:26:00.634167 31608 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:00.635608 31716 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:26:00.636314 31415 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:00.639030 31709 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.639881 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:00.640784 26045 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:00.642045 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:00.643947 31715 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:26:00.645953 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 677) took 1446 ms. Trace:
I20260709 12:26:00.579612 31399 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.644223 31673 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:00.647953 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.634209 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:00.673305 26050 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:00.675074 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:00.652688 26045 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:00.677047 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:00.677830 26045 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:00.651165 26048 rpcz_store.cc:276] 0709 12:25:59.199654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.141835 (+942181us) service_pool.cc:224] Handling call
0709 12:26:00.142175 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.142582 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.142611 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.143230 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.319325 (+176095us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:00.319412 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.319585 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.645725 (+326140us) raft_consensus.cc:1685] finished
0709 12:26:00.645749 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.645890 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1153,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":175519,"threads_started":1,"wal-append.queue_time_us":345022}
I20260709 12:26:00.678859 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:00.660619 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.651705 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.679565 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:00.680461 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.662230 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.681680 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.682765 26120 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:00.689113 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:00.666731 31717 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.267s	user 0.000s	sys 0.002s
I20260709 12:26:00.690114 26129 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:00.651811 31612 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.706027 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:00.707237 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.711086 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.713284 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.713647 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:00.714732 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:00.715461 26048 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:00.715771 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.716377 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:00.719065 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:00.721179 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.721956 26123 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:00.723603 26048 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:00.725320 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:00.730506 26125 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:00.732857 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3122) took 2241 ms. Trace:
I20260709 12:26:00.733546 26052 rpcz_store.cc:276] 0709 12:25:58.492379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.400199 (+907820us) service_pool.cc:224] Handling call
0709 12:25:59.400981 (+   782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.102452 (+701471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.102488 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.115239 (+ 12751us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.381035 (+265796us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:00.381092 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.381189 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.732534 (+351345us) raft_consensus.cc:1685] finished
0709 12:26:00.732568 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.732768 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":395892,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":921328,"threads_started":2,"wal-append.queue_time_us":446635}
I20260709 12:26:00.736618 26051 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:00.739883 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:00.741156 26050 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:00.691905 31664 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:00.661962 31726 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:26:00.707144 31616 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:00.618635 31699 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.714682 31365 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:26:00.772327 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 383037us
I20260709 12:26:00.750581 26052 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:00.723238 31567 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:00.756620 26125 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:00.774721 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:00.777535 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.760298 31726 consensus_queue.cc:1243] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:00.723675 31612 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:00.823041 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48459.5us
I20260709 12:26:00.823974 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:00.828088 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.759402 31616 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.848673 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:00.858690 26201 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:00.865540 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.866667 26194 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:26:00.841037 31726 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:00.846071 31679 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:00.826423 31567 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.849726 31728 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:00.890412 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 174140us
I20260709 12:26:00.896136 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.906635 26045 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:26:00.852200 31608 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:26:00.887979 31726 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:00.914647 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.876754 31674 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:26:00.917222 26194 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:00.915688 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:00.895888 31728 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.922637 31317 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:00.923180 31317 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:00.923602 31317 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.924978 31726 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.923625 31616 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:00.931047 31567 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.931381 31525 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.932272 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.932835 31692 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.932950 26197 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:26:00.933991 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.934592 26197 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:00.935545 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.936187 26197 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:00.944779 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.943360 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:00.916303 31732 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 12:26:00.949432 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.949683 31425 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:00.949918 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.951174 26197 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:00.907642 31664 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:00.956809 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.960153 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.961879 26194 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:00.962949 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.957620 31369 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.965401 31508 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.965667 31369 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.966079 31508 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:00.967943 31369 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:00.968348 31508 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:00.968660 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.969281 26789 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:00.969776 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.970412 26788 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:00.950789 31739 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.972172 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.972632 31739 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:00.974792 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.974907 31739 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.976272 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.977090 26782 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:00.977335 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:00.951717 31708 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.769s	user 0.012s	sys 0.003s
I20260709 12:26:00.986219 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.978268 26788 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:00.992657 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:00.916481 31709 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:26:00.951781 31692 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:00.982167 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:00.955149 31731 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:26:00.937839 31728 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.995934 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.996513 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:00.996692 26791 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:00.943284 31399 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:00.954174 31738 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:01.001894 30857 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:01.002933 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1877) took 4575 ms. Trace:
I20260709 12:26:01.003374 26202 rpcz_store.cc:276] 0709 12:25:56.427134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.591725 (+164591us) service_pool.cc:224] Handling call
0709 12:25:56.635937 (+ 44212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.682024 (+ 46087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.682063 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.682574 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.488808 (+806234us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b180035f860. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:25:58.303009 (+814201us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:59.250289 (+947280us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.489187 (+238898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.002584 (+1513397us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b54002b5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:01.002642 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.002693 (+    51us) raft_consensus.cc:1685] finished
0709 12:26:01.002706 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.002850 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44038,"prepare.queue_time_us":981977,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":36455,"raft.queue_time_us":611019,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":14938,"spinlock_wait_cycles":13271296,"thread_start_us":1961047,"threads_started":3,"wal-append.queue_time_us":1603740,"wal-append.run_cpu_time_us":12913,"wal-append.run_wall_time_us":2128790}
I20260709 12:26:01.006333 26202 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:00.961328 31525 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:00.959414 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:00.987433 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:00.994441 26121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2373) took 2891 ms (client timeout 2098 ms). Trace:
W20260709 12:26:01.020063 26121 rpcz_store.cc:269] 0709 12:25:58.096462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.096665 (+   203us) service_pool.cc:224] Handling call
0709 12:26:00.987530 (+2890865us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:00.987642 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296145664}
I20260709 12:26:00.988197 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.997762 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.998231 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:00.998505 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.023718 26783 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:01.023955 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.024080 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.024677 26785 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:01.025364 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.025724 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.026055 31697 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.026372 31362 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:00.995055 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.026533 31697 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.995442 26790 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:01.026962 31362 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:01.019618 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:00.991739 31708 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.028167 26790 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:01.016587 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:01.026638 26785 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:26:01.029583 26194 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:01.022318 30857 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.022344 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:01.024609 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.030177 26791 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:00.994639 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.031117 26786 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:01.029953 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:01.027199 26782 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:01.032344 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44226.6us
I20260709 12:26:01.032960 26195 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:26:01.025667 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66364.3us
I20260709 12:26:00.994065 31694 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.050346 31671 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.044951 26782 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:00.955492 31317 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.033732 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:01.054504 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:00.997620 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:01.021807 31728 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.043673 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.026970 31656 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:01.048417 31674 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.046713 26783 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:01.030654 31415 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:01.035585 26790 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:26:01.035893 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:01.030686 30857 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.060614 26790 pending_rounds.cc:85] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 12
I20260709 12:26:01.034562 31697 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.027691 31362 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.061172 26790 pending_rounds.cc:107] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:26:01.061461 26790 raft_consensus.cc:2889] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:01.061864 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:01.050792 31694 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.056851 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.042773 26194 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:01.063464 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:01.052649 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:01.065573 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:01.067143 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:01.068280 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:01.069918 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:01.055608 26197 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:01.062682 30857 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.072605 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.073335 26783 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:01.054497 31687 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.017308 31692 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.074422 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:01.059501 31531 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.002s
I20260709 12:26:01.079663 26783 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:01.078224 26194 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:01.082265 26791 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:01.058454 31616 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.058277 31674 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.060595 31415 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:01.085690 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:01.072395 31508 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.085176 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:01.091542 26791 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:26:01.092010 26783 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.072803 31679 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:01.074951 31610 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:01.072659 31608 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:01.082974 31741 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:26:01.075752 31729 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:01.064845 31659 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:26:01.083487 31607 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.061604 31656 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.087246 31616 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:01.095357 31744 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:01.113314 31317 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.091490 31674 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.093581 31508 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.093984 31415 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.121693 31616 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.126041 31609 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:01.121610 31661 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.004s
I20260709 12:26:01.126837 31661 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.095207 31610 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.113133 31708 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.115167 31656 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.116855 31728 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.095446 31679 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.090600 31742 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:01.101053 31729 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:01.084020 31687 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.129359 31610 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:01.134733 31713 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.770s	user 0.002s	sys 0.004s
I20260709 12:26:01.135032 31661 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.088265 31664 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.138875 31679 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:01.139475 31729 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:01.149212 31712 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.812s	user 0.005s	sys 0.000s
I20260709 12:26:01.156849 31694 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.160429 31692 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.159534 31729 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.163564 31729 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.169898 31697 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.172940 31697 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:01.233517 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.245337 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:01.258626 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 195399us
I20260709 12:26:01.272047 26127 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:01.118547 30857 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.128485 31508 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.306552 30857 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.089977 31531 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.308641 31425 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:01.310158 31425 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.311503 31531 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:01.350993 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 77484.8us
I20260709 12:26:01.359763 31563 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:01.119585 31369 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.373602 31671 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.380445 31694 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:01.381124 31694 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.381703 31694 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:01.382216 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 950) took 1131 ms. Trace:
I20260709 12:26:01.382815 26784 rpcz_store.cc:276] 0709 12:26:00.277841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.278021 (+   180us) service_pool.cc:224] Handling call
0709 12:26:01.381486 (+1103465us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400650254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:01.381968 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446902528}
I20260709 12:26:01.315094 31480 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:26:01.390635 31425 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.403795 31687 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.403815 31472 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:01.404973 31472 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:01.405673 31472 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.414979 26782 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:01.415597 26782 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:01.432643 31753 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:01.433460 31753 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:01.421229 31697 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:01.440467 31697 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:01.463304 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 339480us
I20260709 12:26:01.435695 31753 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:01.480649 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 110320us
I20260709 12:26:01.481473 26194 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:01.482470 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:01.484503 26782 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.486281 26782 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.487790 31753 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:01.492336 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 389179us
I20260709 12:26:01.504822 31677 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.510928 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:01.515515 26128 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:01.517203 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:01.518657 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.521638 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1917) took 1145 ms. Trace:
I20260709 12:26:01.524545 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 674) took 2364 ms. Trace:
W20260709 12:26:01.526324 26169 connection.cc:256] RPC call timeout handler was delayed by -2.3117209639995053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
I20260709 12:26:01.527165 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2376) took 2323 ms. Trace:
I20260709 12:26:01.529515 30780 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:01.535542 26049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 676) took 2341 ms (client timeout 1901 ms). Trace:
I20260709 12:26:01.550928 26126 rpcz_store.cc:276] 0709 12:25:59.204043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.204356 (+   313us) service_pool.cc:224] Handling call
0709 12:25:59.204682 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.715273 (+510591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.526411 (+1811138us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400186fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:01.526927 (+   516us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754001792}
I20260709 12:26:01.563252 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:01.564230 26125 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:01.566509 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:01.586006 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75453.6us
W20260709 12:26:01.583664 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87814.7us
W20260709 12:26:01.591756 26169 connection.cc:256] RPC call timeout handler was delayed by -1.9040209639997556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:26:01.592429 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 110387us
I20260709 12:26:01.592684 31514 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:01.593315 31514 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:01.593701 31531 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:01.594475 31531 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:01.595036 31531 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.593818 31514 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:01.597965 31706 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:01.598954 31706 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:01.599074 26169 connection.cc:256] RPC call timeout handler was delayed by -1.740236898999683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 12:26:01.551541 26049 rpcz_store.cc:269] 0709 12:25:59.194056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.201825 (+  7769us) service_pool.cc:224] Handling call
0709 12:26:01.532366 (+2330541us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54001b49d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:01.535232 (+  2866us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040074624}
I20260709 12:26:01.550352 26051 rpcz_store.cc:276] 0709 12:25:59.160094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.076128 (+916034us) service_pool.cc:224] Handling call
0709 12:26:00.736350 (+660222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.740905 (+  4555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.523919 (+783014us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b5400153450. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:01.524261 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1517363968}
I20260709 12:26:01.624917 31706 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.644975 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.618896 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:01.641207 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.629133 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.655143 26124 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:01.655521 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.656293 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:01.657054 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.657483 26124 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:01.642565 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.658427 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.658615 26785 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:01.637198 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.659793 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:01.644778 31425 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:01.643258 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.657097 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:01.653811 26125 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:26:01.659734 26126 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 12:26:01.669117 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52747.1us
I20260709 12:26:01.641917 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.627493 31706 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:01.676766 26782 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:26:01.628386 31612 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:01.681556 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:01.682588 26120 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:01.684881 31425 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.549291 26199 rpcz_store.cc:276] 0709 12:26:00.376345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.377156 (+   811us) service_pool.cc:224] Handling call
0709 12:26:00.377540 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.440911 (+ 63371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.440956 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.441562 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.371306 (+929744us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b1800402340. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:26:01.402255 (+ 30949us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:01.402555 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.402677 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.520941 (+118264us) raft_consensus.cc:1685] finished
0709 12:26:01.520975 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.521378 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9457,"spinlock_wait_cycles":16895872,"thread_start_us":30415,"threads_started":1,"wal-append.queue_time_us":1038855}
W20260709 12:26:01.697856 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 95802.1us
W20260709 12:26:01.698472 31362 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.017s
I20260709 12:26:01.628870 31613 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:01.640547 31616 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:01.704401 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72019.2us
I20260709 12:26:01.690078 31697 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:01.691526 31753 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:01.677675 26782 pending_rounds.cc:85] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 12
I20260709 12:26:01.631223 31544 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.713331 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:01.717774 26782 pending_rounds.cc:107] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:26:01.728327 26782 raft_consensus.cc:2889] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:01.699530 31362 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.732013 26122 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:01.737458 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:01.744340 31613 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:01.744529 31425 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.745355 31613 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:01.746899 31613 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.748440 31526 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:01.749140 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 912) took 5395 ms. Trace:
I20260709 12:26:01.749536 26787 rpcz_store.cc:276] 0709 12:25:56.353378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.097739 (+744361us) service_pool.cc:224] Handling call
0709 12:25:59.158765 (+2061026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.198701 (+ 39936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.748675 (+2549974us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b540063abd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:01.748926 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4672012160}
W20260709 12:26:01.754863 26018 connection.cc:256] RPC call timeout handler was delayed by -2.6204977930001405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 12:26:01.755559 26018 connection.cc:256] RPC call timeout handler was delayed by -2.5718977930000619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 12:26:01.761173 31415 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.006s
I20260709 12:26:01.761448 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:01.761972 31415 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.762446 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 10.
I20260709 12:26:01.738440 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:01.782507 31362 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.786055 31415 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.724948 30857 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:01.794183 26785 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:01.794934 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.769157 31616 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.800089 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:01.801277 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.802814 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.803592 31425 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:01.804348 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.805390 26127 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:01.806903 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.802505 31508 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.808856 31609 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:01.809208 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:01.809796 31609 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.811453 31508 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.803529 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:01.824559 30857 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:01.832621 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 104471us
I20260709 12:26:01.833698 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:01.834708 30857 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.835598 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:01.837105 31761 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 12:26:01.842808 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:01.838127 31674 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.706s	user 0.011s	sys 0.014s
I20260709 12:26:01.845680 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:01.839145 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58425.3us
I20260709 12:26:01.838805 31514 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.837155 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:01.839875 31362 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
W20260709 12:26:01.838626 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71092.6us
W20260709 12:26:01.849864 31680 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:26:01.813544 31610 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:01.790948 31472 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.857692 31415 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:01.867379 26200 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:01.877403 26202 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:01.878974 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43505us
I20260709 12:26:01.881021 26791 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:01.881150 31725 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:01.881820 31725 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:01.883172 31725 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:01.884269 26783 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.757s	user 0.009s	sys 0.010s
I20260709 12:26:01.884927 26783 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:26:01.885826 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2267) took 2571 ms (client timeout 1566 ms). Trace:
W20260709 12:26:01.886375 26201 rpcz_store.cc:269] 0709 12:25:59.314449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.315065 (+   616us) service_pool.cc:224] Handling call
0709 12:26:01.885297 (+2570232us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:01.885522 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3155858304}
I20260709 12:26:01.873952 26198 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:01.894694 26202 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: [10.8-13.10]
I20260709 12:26:01.888355 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:01.805415 31425 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.886420 26123 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 9.
I20260709 12:26:01.857985 31694 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:01.854815 31531 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:01.922891 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 222939us
W20260709 12:26:01.931566 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51560.3us
I20260709 12:26:01.856353 31674 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:01.963500 31679 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.003s
I20260709 12:26:01.965068 31674 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:01.939774 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:01.969336 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44126.4us
I20260709 12:26:01.970194 31694 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.956387 31631 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.960188 31610 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.960749 26120 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:01.961388 31472 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:01.962615 31369 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.008s
I20260709 12:26:01.953663 31425 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:01.974159 26200 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:01.938812 31609 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:01.991714 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47627.1us
I20260709 12:26:01.979799 31415 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:01.995045 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:01.964015 31679 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:02.005232 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 297936us
I20260709 12:26:01.993233 31369 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:02.004760 31415 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.989914 31674 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:26:02.022652 31415 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:02.030382 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 274133us
I20260709 12:26:02.031020 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:01.984910 31631 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.044858 31770 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:02.050473 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1204) took 1036 ms. Trace:
I20260709 12:26:02.050940 26789 rpcz_store.cc:276] 0709 12:26:01.014342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.026319 (+ 11977us) service_pool.cc:224] Handling call
0709 12:26:01.032186 (+  5867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.032657 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.032682 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.032727 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:01.032755 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.032810 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.050059 (+1017249us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b54001225d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:02.050264 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12789632}
I20260709 12:26:02.042622 26786 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:26:02.053069 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 82487us
I20260709 12:26:02.054800 26093 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.055797 31472 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:02.056288 31472 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.056686 31472 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.056809 31531 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:02.057525 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66715us
W20260709 12:26:02.043514 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.064121 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.036381 31725 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.034495 31095 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:02.058902 31514 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.040549 31609 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:02.061182 31631 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:02.070808 26018 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.705s (SENT)
I20260709 12:26:02.076968 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:02.078604 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 125300us
I20260709 12:26:02.085610 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.125732 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1892) took 4434 ms. Trace:
W20260709 12:26:02.113854 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 95313us
W20260709 12:26:02.132452 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.133049 26196 rpcz_store.cc:276] 0709 12:25:57.691637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.691823 (+   186us) service_pool.cc:224] Handling call
0709 12:25:57.725930 (+ 34107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.461950 (+2736020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.125059 (+1663109us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:02.125480 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5987128192}
W20260709 12:26:02.137591 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:26:02.138618 25938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58449.4us
I20260709 12:26:02.085836 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.074586 31095 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:02.143512 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.141460 26051 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:02.076817 31770 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.156654 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.157979 26046 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:26:02.159881 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.160313 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.161233 26048 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:02.147467 31362 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:26:02.162537 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.138420 31535 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:26:02.164568 26052 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:02.164853 31753 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:02.165361 31753 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:02.166986 31753 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.138576 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.169066 26093 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.170590 31544 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:02.171258 31544 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:02.172137 31616 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:26:02.169068 26047 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:26:02.172791 31616 raft_consensus.cc:1068] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Unable to promote non-voter 24e61c63097e4d3dbb149050e6fcc997: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:02.174016 31616 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:02.174425 31616 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:02.174662 31609 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.176108 26784 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:02.176023 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:02.176911 26789 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.177459 31609 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.178047 31531 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:02.164350 31362 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:26:02.152254 31095 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.180876 31753 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:02.183974 31563 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.184402 31362 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.184540 31664 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:02.185014 31362 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:02.185622 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.184414 26049 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:02.190299 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75540.8us
W20260709 12:26:02.195184 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 128318us
I20260709 12:26:02.194702 31732 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:26:02.215149 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:02.201076 31679 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.225873 26791 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:02.230264 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 199797us
I20260709 12:26:02.184679 31563 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.217779 30857 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:02.245004 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.245535 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.247841 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:02.249171 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:02.250075 26051 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:02.229957 31770 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:02.258917 26789 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:02.194742 31544 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.260679 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:02.260892 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:02.253182 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:02.269037 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.271114 26787 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:02.274397 26047 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:02.241261 31362 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.286582 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.259804 31770 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:02.287689 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1877) took 3147 ms. Trace:
I20260709 12:26:02.290944 30831 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:02.246649 31609 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.287932 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:02.339030 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 143346us
I20260709 12:26:02.345324 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:02.347299 26052 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:26:02.343186 26018 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.980s (SENT)
I20260709 12:26:02.345278 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:02.342922 31772 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:26:02.333177 31770 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:02.329728 26053 rpcz_store.cc:276] 0709 12:25:59.139646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.866686 (+727040us) service_pool.cc:224] Handling call
0709 12:26:01.532730 (+1666044us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:01.533648 (+   918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.533693 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:01.579898 (+ 46205us) log.cc:844] Serialized 52 byte log entry
0709 12:26:01.821809 (+241911us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:02.078390 (+256581us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.079590 (+  1200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.101431 (+ 21841us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.101517 (+    86us) raft_consensus.cc:1685] finished
0709 12:26:02.101536 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.287252 (+185716us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54001b49d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:02.287441 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":430681,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":7242,"raft.run_cpu_time_us":536,"raft.run_wall_time_us":2944,"spinlock_wait_cycles":3830160256,"thread_start_us":424331,"threads_started":2,"wal-append.queue_time_us":333873}
I20260709 12:26:02.366231 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 862) took 1458 ms. Trace:
I20260709 12:26:02.367594 26197 rpcz_store.cc:276] 0709 12:26:00.907645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.071046 (+163401us) service_pool.cc:224] Handling call
0709 12:26:01.071383 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.071684 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.071709 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.076474 (+  4765us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.446976 (+370502us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:01.721448 (+274472us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.721757 (+   309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.730470 (+  8713us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.364819 (+634349us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b540031bda0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431adbe 00007f5b74325930 00007f5b743258d8 00007f5b74325858 00007f5b74325813 00007f5b74324a9c 00007f5b7a2102f6 00007f5b7a20aaa8 00007f5b743617cb 00007f5b743642c0
0709 12:26:02.365198 (+   379us) raft_consensus.cc:1685] finished
0709 12:26:02.365367 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.366001 (+   634us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":341942,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":21931,"raft.run_cpu_time_us":579,"raft.run_wall_time_us":4975,"spinlock_wait_cycles":143837568,"thread_start_us":282018,"threads_started":2,"wal-append.queue_time_us":286462}
I20260709 12:26:02.371810 31381 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:02.339030 30831 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:02.348732 31609 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:02.376753 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 135670us
I20260709 12:26:02.382467 26047 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:02.331348 31415 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.383765 31609 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.381383 30831 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.392693 26791 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:02.376117 31543 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:02.402098 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49560.3us
I20260709 12:26:02.402505 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3117) took 4098 ms. Trace:
I20260709 12:26:02.403065 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:00.018824 26203 rpcz_store.cc:276] 0709 12:25:57.779767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.077162 (+297395us) service_pool.cc:224] Handling call
0709 12:25:58.077682 (+   520us) raft_consensus.cc:1471] Updating replica for 7 ops
0709 12:25:58.098606 (+ 20924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.098644 (+    38us) raft_consensus.cc:1539] Triggering prepare for 7 ops
0709 12:25:58.174794 (+ 76150us) log.cc:844] Serialized 182 byte log entry
0709 12:25:58.818537 (+643743us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:59.194095 (+375558us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.194613 (+   518us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.194928 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.309561 (+114633us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.827463 (+517902us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b180045fee0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:25:59.980713 (+153250us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.999139 (+ 18426us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.013958 (+ 14819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.014301 (+   343us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.014364 (+    63us) raft_consensus.cc:1685] finished
0709 12:26:00.014383 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.014625 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12652,"prepare.queue_time_us":1216980,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":133,"raft.queue_time_us":2291184,"raft.run_cpu_time_us":693,"raft.run_wall_time_us":50268,"spinlock_wait_cycles":150805248,"thread_start_us":1680627,"threads_started":4,"wal-append.queue_time_us":857773}
I20260709 12:26:02.394183 31381 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.380406 31549 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:02.403015 26054 rpcz_store.cc:276] 0709 12:25:58.264216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.125469 (+861253us) service_pool.cc:224] Handling call
0709 12:25:59.143186 (+ 17717us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:59.143625 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:59.198221 (+ 54596us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.758959 (+2560738us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180034ce40. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:26:02.060684 (+301725us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:02.143375 (+ 82691us) log.cc:844] Serialized 78 byte log entry
0709 12:26:02.152053 (+  8678us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:02.331770 (+179717us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.336769 (+  4999us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.338899 (+  2130us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.345062 (+  6163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.357017 (+ 11955us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.362023 (+  5006us) raft_consensus.cc:1685] finished
0709 12:26:02.362045 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.362193 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6050,"prepare.queue_time_us":160688,"prepare.run_cpu_time_us":247,"prepare.run_wall_time_us":10497,"raft.queue_time_us":13390,"raft.run_cpu_time_us":627,"raft.run_wall_time_us":24468,"spinlock_wait_cycles":9425536,"thread_start_us":425361,"threads_started":2,"wal-append.queue_time_us":2891013}
I20260709 12:26:02.406996 31535 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:26:02.400872 31543 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:02.426170 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 137720us
W20260709 12:26:02.427065 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.426728 31472 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.428156 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.430325 31563 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.431238 31563 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:02.431561 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.431596 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:02.404099 31526 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.433111 31616 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.436548 31549 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.452690 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.453920 26054 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.455132 31526 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.463856 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.468699 31753 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.417030 31782 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.469841 31753 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:02.470628 31782 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.473357 31753 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:02.469465 31544 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:26:02.474952 31753 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.479074 26054 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:02.494666 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:02.497416 31542 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:02.499289 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.499770 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.500471 26052 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.500742 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:02.501785 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:02.504609 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:02.505384 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:02.499621 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.431951 31563 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:26:02.438455 31674 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:02.513763 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:02.480949 31732 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:02.522163 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:02.488835 31526 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:02.523542 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:02.529003 31612 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:02.522883 26121 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:26:02.532711 31616 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.533654 26121 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:02.539058 31415 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.489619 31381 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.540512 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:02.527513 31775 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:26:02.526516 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.537930 31472 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.547091 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:02.544018 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:02.543316 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.541410 26123 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.549486 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:02.518102 31732 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:02.550209 26782 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.515993 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.542006 26124 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:02.546309 26127 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.554836 26053 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:02.531759 31674 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.554044 26054 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.525545 26052 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:02.552822 26047 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:26:02.557502 26125 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:02.552410 31732 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.567380 26782 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:02.569262 31381 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.569839 26783 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:26:02.570005 26196 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:02.570233 26783 pending_rounds.cc:85] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 8
I20260709 12:26:02.570531 26783 pending_rounds.cc:107] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:26:02.570787 26783 raft_consensus.cc:2889] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.571187 26783 pending_rounds.cc:107] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:26:02.571413 26783 raft_consensus.cc:2889] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.572361 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:02.574753 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:02.575780 26196 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:02.577945 31726 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:02.579167 31608 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:02.579609 31608 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:02.580703 31608 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.582126 31608 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.582756 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:02.583436 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:02.579164 31674 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:02.584663 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:02.584446 31543 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.569285 26126 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:02.587056 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.605741 26197 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:02.567778 31770 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.608881 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:02.609583 26196 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.611632 26195 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:02.544657 31095 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.605000 31543 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.626595 31095 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.627133 31549 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.627774 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:02.628984 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:02.631072 31786 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.178s	user 0.003s	sys 0.000s
W20260709 12:26:02.631494 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 296281us
I20260709 12:26:02.651958 26196 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:02.616348 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:02.641959 26018 connection.cc:256] RPC call timeout handler was delayed by -2.2766977930000394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2194s for a response
W20260709 12:26:02.653709 26018 connection.cc:256] RPC call timeout handler was delayed by -2.2271977929999593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 12:26:02.654064 26018 connection.cc:256] RPC call timeout handler was delayed by -1.8995977929998844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26130000000000003s for a response
W20260709 12:26:02.654359 26018 connection.cc:256] RPC call timeout handler was delayed by -1.8623497079997833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 12:26:02.642387 26054 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
W20260709 12:26:02.643646 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.585649 31732 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:02.656721 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 161298us
I20260709 12:26:02.665136 26791 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:02.653838 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 138508us
I20260709 12:26:02.669657 26128 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:02.673383 31095 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.673511 31609 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:02.674208 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:02.674084 31095 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:02.676368 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.676570 31769 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.678011 26125 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:26:02.679981 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53704.1us
I20260709 12:26:02.678972 31782 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.678393 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:02.677984 26169 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:02.688736 26018 connection.cc:256] RPC call timeout handler was delayed by -1.5116977930001667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3044s for a response
W20260709 12:26:02.689190 26018 connection.cc:256] RPC call timeout handler was delayed by -1.445649708000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24540000000000003s for a response
W20260709 12:26:02.689510 26018 connection.cc:256] RPC call timeout handler was delayed by -1.3401165250002123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 12:26:02.689806 26018 connection.cc:256] RPC call timeout handler was delayed by -1.2771086880002258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 12:26:02.690085 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1090497079999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2828s for a response
W20260709 12:26:02.690361 26018 connection.cc:256] RPC call timeout handler was delayed by -1.0329637109998657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 12:26:02.693342 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:02.674104 31609 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:02.697582 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45723.3us
I20260709 12:26:02.685303 30780 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.699527 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.710126 26054 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.704578 31095 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.705125 31726 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:02.705824 26791 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:02.717069 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:02.723618 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:02.653322 31725 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 12:26:02.728950 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.727849 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.730432 26784 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:02.698773 31782 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:02.732623 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 233490us
I20260709 12:26:02.716771 31609 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.734313 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.732564 31472 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:02.720855 31362 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.740540 31607 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.742452 31791 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.735544 31701 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:26:02.744529 31369 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:02.753398 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:02.735196 31769 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:02.734954 31726 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:02.745891 31770 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:02.747362 31732 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.755507 26054 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:02.761868 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.758622 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.760577 26052 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:02.763522 26052 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:02.770112 30780 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.771526 26053 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:02.771781 31472 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:02.772675 31631 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:02.773141 31472 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.750080 31725 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:26:02.755155 31362 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.778115 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:02.778792 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:02.779130 26196 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:02.779611 26195 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:02.779417 31607 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.780673 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:02.781541 26197 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:02.782378 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:02.782635 31607 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:02.752291 31677 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:02.784037 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:02.764376 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:02.798347 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.799249 26046 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:02.799785 26051 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:02.800962 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.800115 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:02.803962 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49419us
I20260709 12:26:02.767552 31237 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:02.805501 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.806234 26051 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:02.807250 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.802345 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.809679 26046 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:02.808084 26051 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:02.814152 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.811326 26054 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:02.818213 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.820802 26200 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.760300 31782 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:02.831928 31782 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:02.833981 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.835130 26045 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:02.836664 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:02.813158 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71382.5us
I20260709 12:26:02.801505 31677 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:02.826701 31726 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:02.817042 31701 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.828836 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.829833 31362 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:02.830274 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71730.8us
I20260709 12:26:02.834738 31549 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:02.841557 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.840306 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.839327 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.839800 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:02.853336 26045 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:02.846386 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.824222 31769 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.858624 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.854578 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:02.818092 31607 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:02.847321 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 89826.6us
I20260709 12:26:02.861513 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.851318 31549 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.844048 31677 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.818719 31706 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:26:02.874060 26200 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:02.848426 31701 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:02.877187 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:02.883479 26791 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:02.882025 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.885106 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:02.874104 31549 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.887928 26046 raft_consensus.cc:2376] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16. Re-sending same reply.
I20260709 12:26:02.919265 26050 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:26:02.889328 26169 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:02.924808 26050 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 12:26:02.924928 31706 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:02.925457 31706 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:02.926357 31706 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:02.931210 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 88615.3us
I20260709 12:26:02.939985 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:03.002767 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:03.004010 26121 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:26:03.005798 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:02.931308 31770 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.009298 26127 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:02.886308 31791 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:26:03.012038 26129 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:02.943946 31810 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:03.042479 31810 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:03.042975 31810 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.045583 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:03.046707 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:03.054003 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 124636us
I20260709 12:26:02.951478 31612 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.057201 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1222) took 1401 ms. Trace:
W20260709 12:26:03.059033 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 196730us
I20260709 12:26:03.011397 31770 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.060091 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.060611 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:03.059536 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.933418 31514 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:26:03.062844 26786 rpcz_store.cc:276] 0709 12:26:01.655274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.657209 (+  1935us) service_pool.cc:224] Handling call
0709 12:26:01.657548 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.052502 (+394954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.056837 (+1004335us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400660154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:03.057077 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846721792}
I20260709 12:26:03.068289 26196 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:03.070770 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:03.071553 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.080336 31514 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:03.080853 31514 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:03.081736 31514 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.083006 31514 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:03.083599 31608 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.003s
I20260709 12:26:03.086387 26120 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:03.093662 26168 leader_election.cc:336] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.152s (SENT)
I20260709 12:26:03.067325 31770 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.112366 31807 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 12:26:03.116431 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48723.2us
I20260709 12:26:03.118120 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.068929 31612 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.122802 26126 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:03.146246 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:03.149887 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.104856 31808 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:03.127950 31807 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:03.160022 26782 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.162209 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:03.201226 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.194526 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.159061 31807 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:03.204878 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.210189 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 140796us
I20260709 12:26:03.203095 31808 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.231712 31808 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:03.271363 31808 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:03.228593 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76517.1us
I20260709 12:26:03.280865 26199 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:03.229895 31806 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:03.281916 31806 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.282341 31806 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:03.208133 31807 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.284372 31674 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:03.286430 31674 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:03.286806 31674 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:03.244335 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:03.287637 31674 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.248303 31812 raft_consensus.cc:1064] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:26:03.289669 31674 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.231374 31369 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:03.221558 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.292171 31812 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.297122 31549 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.299213 31753 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.299695 31549 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.300199 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:03.302071 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98854.2us
W20260709 12:26:03.302767 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.303314 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.304412 31726 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:03.305584 26123 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.299798 31753 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.307096 31369 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:03.307925 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.308991 31753 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.309218 31369 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:03.310252 31369 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:03.311975 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:03.295797 25946 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 12, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:03.311347 31369 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.270249 31612 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:03.315078 26127 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.315228 31369 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.314219 31608 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:03.316756 31608 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:03.317742 26127 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:26:03.289404 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 953) took 1047 ms. Trace:
W20260709 12:26:03.318349 26168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.572s (SENT) [suppressed 25 similar messages]
I20260709 12:26:03.317736 31608 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.306517 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:03.322286 31822 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:03.321214 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 114527us
W20260709 12:26:03.324020 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:03.325377 26120 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.318509 26788 rpcz_store.cc:276] 0709 12:26:02.242022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.126419 (+884397us) service_pool.cc:224] Handling call
0709 12:26:03.289316 (+162897us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:03.268741 31609 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.003s
I20260709 12:26:03.305657 31726 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:26:03.351166 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51034.9us
I20260709 12:26:03.365605 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.323308 31812 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:03.370301 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.371237 26197 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.345276 31831 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:26:03.385004 31831 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:03.385219 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57968.7us
I20260709 12:26:03.386080 31836 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:03.389387 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:03.348592 31616 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.336366 31726 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:03.351332 31828 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.362497 31549 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.325410 31612 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:26:03.380324 31812 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.404487 31812 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.406065 26202 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:03.357582 31609 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.408965 31609 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.409629 31609 raft_consensus.cc:2764] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:03.410892 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:03.411793 26784 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:03.411677 31803 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:26:03.413386 31810 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.415606 31810 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.425285 31726 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.426966 31726 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.399461 31616 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.430954 31616 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:03.432660 31612 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:03.432588 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:03.435117 26788 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:03.433251 31612 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:03.431660 31549 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:03.480966 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 124056us
I20260709 12:26:03.397369 31770 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:03.501055 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.516671 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.520505 31549 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.526582 26120 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:03.527621 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 162136us
I20260709 12:26:03.528579 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:03.528795 31803 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:03.530021 31803 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.530535 31803 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.534659 26197 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
W20260709 12:26:03.534924 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.535235 31803 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.534695 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:03.537468 31701 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.538890 31701 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:03.539633 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.544193 26784 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:03.532065 31680 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:03.549806 31680 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:03.551090 31680 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:03.557364 31677 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.000s
I20260709 12:26:03.557931 31677 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.561415 31826 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:03.561823 31807 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.563043 31680 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.566031 31770 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:03.567049 31837 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:03.568125 31808 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.569849 31752 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:03.570955 31808 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.572650 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.559806 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.597574 26202 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [13.12-15.13]
I20260709 12:26:03.601495 26199 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.602353 26789 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:26:03.576843 31769 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.010s
I20260709 12:26:03.604130 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:03.605849 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.606251 26122 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:03.607220 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 246076us
I20260709 12:26:03.601307 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.615258 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.616914 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.619863 31679 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:03.620357 31679 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:03.620153 31812 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.621372 31679 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.622784 31679 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.624595 26789 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEARNER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:03.627928 31806 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.630234 31806 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.630667 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:03.633196 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:03.634244 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.634923 26203 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:03.635573 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1930) took 1198 ms. Trace:
I20260709 12:26:03.635885 26203 rpcz_store.cc:276] 0709 12:26:02.437015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.577543 (+1140528us) service_pool.cc:224] Handling call
0709 12:26:03.597348 (+ 19805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.635328 (+ 37980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.635522 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.637218 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.644776 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1240) took 1145 ms. Trace:
I20260709 12:26:03.645234 26783 rpcz_store.cc:276] 0709 12:26:02.499145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.499280 (+   135us) service_pool.cc:224] Handling call
0709 12:26:02.499637 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.596865 (+ 97228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.644436 (+1047571us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:03.644677 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134445312}
I20260709 12:26:03.648962 26052 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.585204 31808 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:03.598976 31807 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:03.655000 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 95282.9us
I20260709 12:26:03.656563 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.665612 26194 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 10: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:03.665622 26052 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:03.668599 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.668826 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:03.669346 26787 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.609908 31612 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.671779 31612 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.675647 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3213) took 1181 ms. Trace:
I20260709 12:26:03.656454 31808 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:03.677052 31808 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.678171 26787 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:03.616129 31769 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:03.651607 31415 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:03.681161 31415 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:03.682183 31415 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.683424 31751 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:03.684319 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76227.1us
I20260709 12:26:03.681223 26200 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.685148 31609 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:03.651371 31677 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.655246 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.676100 26128 rpcz_store.cc:276] 0709 12:26:02.494068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.513833 (+ 19765us) service_pool.cc:224] Handling call
0709 12:26:02.547519 (+ 33686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.671487 (+123968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.675358 (+1003871us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:03.675562 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276906880}
I20260709 12:26:03.709052 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 931) took 1111 ms. Trace:
I20260709 12:26:03.689674 31679 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:03.685734 31526 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:03.695621 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:03.725907 31526 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:03.725772 31679 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.726290 26789 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:26:03.686779 31833 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:03.727237 31526 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.713034 31752 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:26:03.703445 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.688323 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.739347 31679 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.689709 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:03.705514 26197 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:26:03.685681 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.696712 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.713743 31095 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:26:03.693959 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.700773 25946 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:03.724954 26126 rpcz_store.cc:276] 0709 12:26:02.598277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.606865 (+  8588us) service_pool.cc:224] Handling call
0709 12:26:03.708729 (+1101864us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54001d1fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:03.708944 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078064384}
I20260709 12:26:03.734860 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:03.754477 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 282544us
I20260709 12:26:03.698630 31805 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.713392 31751 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:26:03.741783 31610 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.762018 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.714727 31812 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:03.749579 26199 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:03.766872 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.744125 31828 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:03.751374 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:03.729163 31835 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.344s	user 0.000s	sys 0.004s
I20260709 12:26:03.754613 26200 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:03.760672 26123 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:03.761487 26052 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:26:03.693429 26198 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:03.749724 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:03.772400 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:03.768613 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.774313 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3124) took 1047 ms. Trace:
W20260709 12:26:03.778277 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 92676.8us
I20260709 12:26:03.778810 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:03.782254 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.788414 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.791354 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.774785 26052 rpcz_store.cc:276] 0709 12:26:02.727047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.727533 (+   486us) service_pool.cc:224] Handling call
0709 12:26:03.648571 (+921038us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b54000d91a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7432306f 00007f5b7439bf60 00007f5b7439d7de 00007f5b743a927c 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2
0709 12:26:03.774251 (+125680us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49176448}
W20260709 12:26:03.800938 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 111359us
I20260709 12:26:03.802172 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.737917 31514 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:03.802222 26194 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:03.808517 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:03.813452 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.798421 31631 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:26:03.815444 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:03.818213 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.775300 31806 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:03.821754 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.824568 26195 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:03.826517 26197 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:03.776336 31610 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.828663 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.797742 31677 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:03.831437 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.799078 31732 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.835505 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.827796 31317 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:03.842971 31317 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:03.837838 25946 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 10, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 10 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:03.845252 25946 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:03.848402 31843 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:03.848620 25952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 191) took 33849 ms (client timeout 15000 ms). Trace:
I20260709 12:26:03.849056 31843 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.849539 31843 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.821614 31526 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.852988 31677 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:03.854025 31677 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:03.838228 31732 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.857287 31732 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.840255 26203 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.842027 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:03.862529 26200 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:03.861189 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:03.866640 26201 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:03.775183 31805 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.871920 26018 leader_election.cc:400] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14.
I20260709 12:26:03.875861 26018 leader_election.cc:403] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:03.875895 26201 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:03.877264 26199 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:03.836330 31514 consensus_queue.cc:1243] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:26:03.875023 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.860476 31664 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:03.882490 26200 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:03.868705 31381 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.884929 30780 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:03.886067 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 691) took 1545 ms. Trace:
I20260709 12:26:03.868973 31674 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:03.889371 26199 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.908424 31806 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14.
I20260709 12:26:03.910147 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.911585 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:03.914844 31095 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:03.843892 31317 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:03.888046 26048 rpcz_store.cc:276] 0709 12:26:02.341076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.341247 (+   171us) service_pool.cc:224] Handling call
0709 12:26:02.361672 (+ 20425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.393803 (+ 32131us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:02.409889 (+ 16086us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.646065 (+1236176us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b1800178b40. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:26:03.884650 (+238585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.884716 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:03.884763 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.885866 (+  1103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.886004 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15103,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":15653,"spinlock_wait_cycles":14567168,"thread_start_us":225752,"threads_started":1}
I20260709 12:26:03.920081 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:03.925112 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:03.927469 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:03.929421 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:03.863313 25946 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:03.973119 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.973824 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.974246 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.945479 26203 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:03.947834 31769 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:26:03.849115 25952 rpcz_store.cc:269] 0709 12:25:30.006514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.014679 (+  8165us) service_pool.cc:224] Handling call
0709 12:26:03.848304 (+33833625us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:52.813155 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:52.813882 (+   727us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:52.813933 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:25:52.813968 (+    35us) tablet.cc:662] Decoding operations
0709 12:25:52.866019 (+ 52051us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:52.866159 (+   140us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:52.866196 (+    37us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:25:52.868066 (+  1870us) tablet.cc:701] Row locks acquired
0709 12:25:52.868128 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:25:52.942225 (+ 74097us) write_op.cc:270] Start()
0709 12:25:53.028902 (+ 86677us) write_op.cc:276] Timestamp: P: 1783599952925874 usec, L: 0
0709 12:25:53.065204 (+ 36302us) op_driver.cc:348] REPLICATION: starting
0709 12:25:53.092685 (+ 27481us) log.cc:844] Serialized 7170 byte log entry
0709 12:25:54.038412 (+945727us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:25:54.348680 (+310268us) op_driver.cc:464] REPLICATION: finished
0709 12:25:54.560421 (+211741us) write_op.cc:301] APPLY: starting
0709 12:25:54.560529 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:55.253797 (+693268us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:55.253842 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:56.313410 (+1059568us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:56.313826 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:25:56.313899 (+    73us) write_op.cc:312] APPLY: finished
0709 12:25:56.314424 (+   525us) log.cc:844] Serialized 164 byte log entry
0709 12:25:56.467144 (+152720us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:56.467354 (+   210us) write_op.cc:454] Released schema lock
0709 12:25:56.467427 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:56.587284 (+119857us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:56.591620 (+  4336us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2232,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":188742,"apply.run_cpu_time_us":44039,"apply.run_wall_time_us":1930486,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79475,"num_ops":18,"prepare.queue_time_us":150987,"prepare.run_cpu_time_us":51342,"prepare.run_wall_time_us":1056504,"raft.queue_time_us":257795,"raft.run_cpu_time_us":15866,"raft.run_wall_time_us":169751,"replication_time_us":1283300,"spinlock_wait_cycles":11026048,"thread_start_us":1162091,"threads_started":5,"wal-append.queue_time_us":1028543,"wal-append.run_cpu_time_us":47745,"wal-append.run_wall_time_us":2483739}]]}
I20260709 12:26:03.955022 31567 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:03.920528 31472 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:26:03.955914 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.935820 31362 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:03.957325 31610 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:03.890996 31381 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.007227 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:04.008661 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:03.964382 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.935034 26049 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.975637 26199 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:04.012813 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:03.977811 31095 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:03.989377 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:03.996656 30780 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:03.922010 31752 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:03.979846 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.916003 31828 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:03.980712 26203 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:04.020411 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:03.931816 31674 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.069342 26196 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:26:04.023388 30780 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.983376 31769 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:04.019557 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:03.936292 31751 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 12:26:04.004437 31844 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:04.002678 31362 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:04.069339 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 89579.1us
I20260709 12:26:04.003779 31631 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:04.004200 31813 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.090629 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76133.8us
W20260709 12:26:04.047510 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.090853 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:04.091573 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:03.977602 31808 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.090260 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:04.106724 26199 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:04.106915 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.107690 26052 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:04.108745 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.022716 31095 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:04.114547 26049 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:04.109046 30780 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.103696 31631 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.005190 31680 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.093s
W20260709 12:26:04.112219 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 123015us
I20260709 12:26:04.089898 31818 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:04.116513 26122 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:04.100390 31607 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:04.093593 31793 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:04.129164 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.119114 26203 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.134626 30780 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:04.137827 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.122802 31631 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:04.143662 31813 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.123544 31514 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:04.145931 26201 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:04.146785 31813 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:04.146994 25946 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:04.148726 25946 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 10 to 11, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 11 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:04.153676 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:04.130733 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:04.160636 26128 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:04.159886 26052 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:04.164822 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.165932 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.169843 26049 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:04.171952 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 333078us
I20260709 12:26:04.171406 26202 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:04.120134 31095 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:04.180258 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.183818 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:04.185738 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:04.131491 31793 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.182677 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:04.192323 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97324.4us
I20260709 12:26:04.173225 31813 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.228094 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.228893 26052 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:04.230939 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:04.227664 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.232991 26045 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:04.234324 31725 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:04.234912 31725 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:04.239097 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88216us
I20260709 12:26:04.240885 31472 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:04.241324 31472 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:04.240657 25946 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 8 to 12, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 12 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:04.242190 31472 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.243561 31472 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:04.248660 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:04.249146 26045 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:04.251361 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.252498 31677 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:04.195003 31362 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.257669 31362 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:04.167006 31415 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:26:04.206243 31793 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.189265 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:04.217782 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.187518 31514 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:04.222443 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.289783 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.223028 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.177237 31607 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:04.188371 31751 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.204974 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.265193 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.194344 31782 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:04.167326 31526 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.220438 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.273082 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.312559 26049 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:04.276607 31381 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:04.281364 31848 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:26:04.159366 31818 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.281983 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.293047 26200 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:04.294095 26197 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:04.297034 26047 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11.
W20260709 12:26:04.297149 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107983us
I20260709 12:26:04.305302 31680 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:04.308823 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.243846 25946 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 7 to 9, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.312841 26045 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:04.213722 31732 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.182294 31769 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:04.301780 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.312837 26195 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:04.335449 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.312714 31415 consensus_queue.cc:1243] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:04.236074 31725 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.344422 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.307379 31751 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:04.332790 31849 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
W20260709 12:26:04.347255 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.366389 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:04.367455 31609 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:04.332384 31610 raft_consensus.cc:670] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } [suppressed 2 similar messages]
I20260709 12:26:04.368103 31609 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:04.368427 31674 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:04.369257 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:04.369477 31609 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.304729 31607 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:04.389056 31846 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.292s	user 0.003s	sys 0.000s
I20260709 12:26:04.398799 31769 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:04.334826 31838 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:04.351253 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.353724 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:04.414871 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.355782 31834 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:04.356510 31808 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:04.356838 31751 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:04.357625 31843 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
W20260709 12:26:04.358518 31854 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:26:04.406024 26123 raft_consensus.cc:1217] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Deduplicated request from leader. Original: 8.5->[9.6-11.7]   Dedup: 9.6->[11.7-11.7]
I20260709 12:26:04.406018 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.415287 26049 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
W20260709 12:26:04.414482 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46653.8us
I20260709 12:26:04.423620 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:04.423570 26126 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:04.432564 26123 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 12:26:04.466923 26128 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:04.473793 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 191884us
I20260709 12:26:04.405964 31818 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:04.473394 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45880.7us
I20260709 12:26:04.477738 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:04.481395 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:04.482483 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:04.512578 31838 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.547240 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64887.4us
I20260709 12:26:04.548612 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:04.556782 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.556970 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.557936 30780 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.559638 26785 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:04.404171 31805 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:04.350219 31514 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:26:04.576035 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.578586 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:04.447871 31725 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:04.467989 31808 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:04.584388 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.588824 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:04.589919 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.412977 31631 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:04.593742 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.593930 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.561098 31701 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:04.425952 31843 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.647718 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:04.668937 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78287.1us
I20260709 12:26:04.426020 31834 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:04.679066 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.581458 31608 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:04.685828 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:04.640722 31818 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:04.686650 26194 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
W20260709 12:26:04.687296 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 98547.4us
I20260709 12:26:04.688484 26047 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:04.688589 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.643889 31677 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.679704 31813 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:04.677737 31680 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.709301 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:04.709980 26782 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
W20260709 12:26:04.699399 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.141s (SENT)
I20260709 12:26:04.711273 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:04.706552 31803 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.683810 31770 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.708014 26791 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:04.590643 31805 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.105s
W20260709 12:26:04.716977 31863 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:26:04.587569 31860 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 12:26:04.687894 31608 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:04.576965 31838 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.689394 31867 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.720042 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.727448 26052 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:26:04.732759 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.741884 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.704830 31609 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:26:04.722170 31803 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.643144 25946 catalog_manager.cc:5697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 7 to 13, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 13 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:04.747735 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54042.3us
I20260709 12:26:04.721204 31608 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:04.751571 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.754004 30780 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:04.755664 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.754408 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.746774 31609 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:04.719218 31680 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:04.760605 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 67740.1us
W20260709 12:26:04.756127 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.763094 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:04.732755 31869 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:04.724965 31770 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:04.788360 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.788671 31770 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:04.790316 31845 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.916s	user 0.015s	sys 0.017s
I20260709 12:26:04.797695 31838 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:04.798411 31680 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:04.801877 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.802665 26048 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:04.804286 26054 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:04.804443 31834 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:04.804755 26054 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:04.737773 31867 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.805534 26054 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.806797 26054 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.807636 31867 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:04.772902 31803 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:04.768891 31631 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:04.813130 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.813704 26052 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:26:04.815685 31095 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.005s
I20260709 12:26:04.761380 31609 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:04.817044 31609 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:04.778224 31753 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:04.818173 31609 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:04.779608 31843 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:04.756989 31732 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:04.768677 31847 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:26:04.801960 26053 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:04.774821 25946 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 9 to 11, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 11 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:04.808982 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43579.1us
I20260709 12:26:04.818367 31753 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:04.821976 25946 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 5 to 8, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 8 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:04.825114 31869 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:26:04.828150 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.829116 25946 catalog_manager.cc:5697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 1 to 9, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 9 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.834378 25946 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:04.836858 31869 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:04.837535 31793 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 12:26:04.836680 25946 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 9 to 10, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 10 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.838092 31793 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.839365 25946 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.835636 31701 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.841969 25946 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 7 to 10, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 10 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:04.785498 31472 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.003s
I20260709 12:26:04.869580 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:04.877267 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.877698 26194 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
W20260709 12:26:04.844218 31870 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 12:26:04.883328 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:04.869940 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:04.870803 26785 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:04.872710 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:04.881009 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.869549 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.885150 26123 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:04.823599 31753 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:04.888016 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:04.890095 26791 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:04.886483 26786 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:04.888514 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:04.905047 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.926672 26787 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:26:04.876180 31362 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.005s
I20260709 12:26:04.867689 31808 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:04.894541 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:04.882464 31806 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.931872 26054 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:04.898089 26128 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:26:04.938683 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.945297 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:04.930140 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.948623 26045 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:04.853284 31095 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:04.947975 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53556.9us
I20260709 12:26:04.954732 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.957000 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.958171 26203 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:04.966552 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.957340 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.968206 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:04.950245 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.970417 26127 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:04.953245 26120 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:04.962849 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1961) took 1226 ms. Trace:
I20260709 12:26:04.971556 26197 rpcz_store.cc:276] 0709 12:26:03.746707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.917725 (+171018us) service_pool.cc:224] Handling call
0709 12:26:04.293649 (+375924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.321379 (+ 27730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.962625 (+641246us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400397450. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:04.962789 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832130304}
I20260709 12:26:04.963315 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.964095 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:04.961313 26168 leader_election.cc:336] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.506s (SENT)
I20260709 12:26:04.965695 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:04.964856 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:04.957000 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:04.973937 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 240935us
I20260709 12:26:05.010084 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:05.011760 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47139.6us
I20260709 12:26:05.012916 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.974694 26196 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:26:05.017791 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.021147 26050 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:26:05.023777 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51260.3us
I20260709 12:26:04.946432 31828 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.026435 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:04.935470 31793 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:04.968297 31664 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:26:04.957602 31812 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:05.035327 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:04.875866 31608 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.031617 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.037931 30780 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:04.891958 31862 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.128s
I20260709 12:26:04.957314 31875 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.036470 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.037245 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:05.052754 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.037405 26123 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:04.893612 31732 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.123s
I20260709 12:26:04.973937 31701 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.040364 26045 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:26:05.039450 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78195.2us
W20260709 12:26:05.032975 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68968us
I20260709 12:26:05.043406 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.043820 26789 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:26:05.044302 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.010087 31237 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:05.030297 31836 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:05.068945 26168 leader_election.cc:336] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.567s (SENT)
I20260709 12:26:05.086728 26046 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:05.086630 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.043787 31631 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:05.075114 30780 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:05.088654 30780 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:05.075591 31608 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.040493 31828 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.076927 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.077206 31725 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.066532 31803 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:05.077752 31095 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:05.077636 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.078375 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.039862 31770 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:05.079125 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.079869 31818 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.039692 31679 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:26:05.017031 31362 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:05.067906 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1962) took 1313 ms. Trace:
I20260709 12:26:05.120805 26203 rpcz_store.cc:276] 0709 12:26:03.754436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.926329 (+171893us) service_pool.cc:224] Handling call
0709 12:26:03.926774 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.945765 (+1018991us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b580045ecd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:26:05.067529 (+121764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.067572 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:05.067614 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:05.067644 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.067727 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.067852 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":742,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":9740,"spinlock_wait_cycles":50401024}
I20260709 12:26:05.123847 26051 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:05.090760 31631 consensus_queue.cc:1243] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:26:05.126037 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57162.3us
W20260709 12:26:05.127164 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.969748 31806 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.091187 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:05.131505 26195 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.091864 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:05.095299 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56126.3us
I20260709 12:26:05.095943 26790 raft_consensus.cc:1217] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 10.10->[12.11-14.12]   Dedup: 14.12->[]
W20260709 12:26:05.099102 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.102994 26053 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:05.061575 31812 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:05.105934 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:05.065634 31813 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:05.134755 26790 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 12:26:05.129629 30780 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:05.123178 31769 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.724s	user 0.016s	sys 0.000s
I20260709 12:26:05.063185 31875 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.113675 26047 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:05.114106 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:05.143177 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.143476 26046 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.114681 26045 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:05.082278 26200 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:05.144502 30780 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:05.077697 31867 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.147068 26195 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:05.044396 31751 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:26:05.080010 31237 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:05.085353 31836 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:05.119364 26756 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.901s (SENT)
I20260709 12:26:05.130592 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:05.171705 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.133924 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:05.107506 26054 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:05.132098 31880 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.173799 26046 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:05.116255 31770 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:05.113852 31725 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.184223 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.185046 26054 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
W20260709 12:26:05.106544 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 94368.8us
I20260709 12:26:05.148795 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.089373 31701 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.152376 26201 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:26:05.186638 26201 pending_rounds.cc:85] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 6
I20260709 12:26:05.186905 26201 pending_rounds.cc:107] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:26:05.187129 26201 raft_consensus.cc:2889] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.153577 26050 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.125641 31818 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.160408 31237 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.161111 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.167157 31867 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.118276 31860 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.173194 26052 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:05.178144 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44290us
I20260709 12:26:05.198938 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:05.199216 31867 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.200861 31810 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:05.201298 31818 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:05.116345 31803 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.205735 31803 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.146332 26049 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:05.137421 31813 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.186292 26053 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.186271 31770 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:05.194486 31828 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:05.179021 31836 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.196635 31725 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.184945 31610 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.100124 31808 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.207683 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:05.222875 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103560us
I20260709 12:26:05.226361 31237 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:05.181224 31609 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:05.228847 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.234819 26049 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:05.238538 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:05.239262 26789 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
W20260709 12:26:05.240263 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.237883 26782 pending_rounds.cc:85] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 8
I20260709 12:26:05.243620 26782 pending_rounds.cc:107] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:26:05.244007 26782 raft_consensus.cc:2889] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.218562 31813 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:05.246846 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:05.251767 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:05.249997 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:05.254570 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.256771 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:05.257733 26789 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:05.261809 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.262564 26049 raft_consensus.cc:2376] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17. Re-sending same reply.
I20260709 12:26:05.190855 31782 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:26:05.264652 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:05.217803 31472 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:26:05.182782 31880 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.245002 31362 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:05.284284 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:05.285606 30780 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:05.286116 30780 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.286505 30780 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.284372 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:05.267244 31818 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:05.302037 30780 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:05.327610 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.333034 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.337585 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.340656 26050 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:05.249430 31751 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:26:05.348199 30780 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:05.336364 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.349112 26051 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.274494 31862 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:05.296222 31875 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:05.356781 31882 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:26:05.290869 31362 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.333184 31526 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:05.334899 31612 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.375638 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:05.376715 26790 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.379415 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 882) took 1367 ms. Trace:
I20260709 12:26:05.385847 26199 rpcz_store.cc:276] 0709 12:26:04.011694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.291390 (+279696us) service_pool.cc:224] Handling call
0709 12:26:04.310146 (+ 18756us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:04.313892 (+  3746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.313922 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:04.718107 (+404185us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:26:04.721194 (+  3087us) log.cc:844] Serialized 78 byte log entry
0709 12:26:04.963891 (+242697us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:04.963933 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.963994 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.358670 (+394676us) raft_consensus.cc:1685] finished
0709 12:26:05.358696 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.379311 (+ 20615us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1231,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":1999,"spinlock_wait_cycles":12050816,"thread_start_us":216883,"threads_started":1,"wal-append.queue_time_us":366809}
W20260709 12:26:05.395325 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.365226 31362 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:05.397918 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.398033 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 151271us
W20260709 12:26:05.395149 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 141807us
I20260709 12:26:05.401698 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.402661 26050 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:05.369789 31526 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.406433 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:05.366585 31862 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:05.407341 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.407356 26199 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
W20260709 12:26:05.410413 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.281912 31812 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:05.413533 26094 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.646s (SENT)
I20260709 12:26:05.417137 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:05.426924 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:05.427886 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:05.429721 31095 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:05.440444 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.410272 31526 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:05.449028 31890 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:26:05.451092 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:05.451889 31567 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:05.452410 31567 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:05.382009 31612 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.455685 31612 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:05.456367 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 1966) took 1622 ms. Trace:
I20260709 12:26:05.456939 26201 rpcz_store.cc:276] 0709 12:26:03.833792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.231936 (+398144us) service_pool.cc:224] Handling call
0709 12:26:04.232211 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.191088 (+958877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.456000 (+264912us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400302854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:05.456286 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031778688}
I20260709 12:26:05.463269 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.464157 26050 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11.
W20260709 12:26:05.467422 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 123484us
I20260709 12:26:05.472187 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:05.422664 31806 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:05.479204 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.482800 26050 raft_consensus.cc:2376] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17. Re-sending same reply.
I20260709 12:26:05.480021 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:05.477133 31888 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.291s	user 0.000s	sys 0.002s
I20260709 12:26:05.484107 26199 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:05.496950 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.499796 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:05.500758 26789 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.499919 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.515412 26050 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 12: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:05.515484 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.518824 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.519695 26053 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.520036 26051 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:05.544502 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47671.4us
I20260709 12:26:05.545732 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.555912 26053 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:05.561053 26786 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:05.563052 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.563876 26050 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:05.563851 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.565165 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.558549 31701 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.542235 31769 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.571421 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:05.572767 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:05.574252 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:05.577320 26784 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:05.578636 26789 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:05.574745 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:05.622413 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49725.7us
I20260709 12:26:05.623317 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.630257 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.631121 26053 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:05.632398 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47977.5us
I20260709 12:26:05.569049 31612 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.651567 26053 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:26:05.634120 31889 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:05.624024 31664 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.503981 31862 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.647795 26050 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:05.730268 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.651093 31894 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:26:05.680408 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:05.668495 31664 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:05.747259 31889 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:26:05.760946 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80650.7us
I20260709 12:26:05.691994 31680 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 12:26:05.765750 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:05.771241 26045 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:05.772362 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 149115us
I20260709 12:26:05.773195 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.791348 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:05.769629 31610 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:05.818466 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.819398 26050 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:26:05.821172 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.821887 26050 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:05.822182 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.823221 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.825485 26050 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:05.827797 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.829411 31664 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.829411 26050 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:05.831333 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:05.832213 31898 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:26:05.832186 26050 raft_consensus.cc:2376] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17. Re-sending same reply.
I20260709 12:26:05.836264 31893 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.836813 31893 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.840044 31893 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.842084 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.843576 31415 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.843196 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:05.845449 31769 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.845955 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.846403 31769 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:05.846724 26053 raft_consensus.cc:2368] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14.
I20260709 12:26:05.847008 31769 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.848482 31902 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:05.849082 31902 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:26:05.853310 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:05.853814 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80702.2us
I20260709 12:26:05.853946 26053 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14. Re-sending same reply.
I20260709 12:26:05.854530 31880 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:05.850773 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.855253 31514 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.856060 31514 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.856348 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:05.856937 31818 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.003s
I20260709 12:26:05.857355 31818 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:05.857523 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.857978 31818 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:05.858201 31514 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:05.858806 31514 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:05.859483 31514 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:05.859685 31904 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:26:05.860375 31514 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:05.860467 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:05.863130 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:05.801431 31610 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.864068 31610 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:05.801718 31869 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:05.865150 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:05.865242 31869 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:05.865952 26199 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:05.867585 31869 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:05.865957 31567 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.869450 31567 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.872929 26050 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 12:26:05.874359 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.874187 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:05.874779 31906 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:26:05.874966 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:05.877595 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.878422 26053 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:05.858462 31818 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.884577 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.884789 30780 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:05.885982 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.886250 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.844393 31415 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.886929 26051 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14. Re-sending same reply.
I20260709 12:26:05.887717 26050 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
W20260709 12:26:05.899626 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 76286.4us
I20260709 12:26:05.888744 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.900406 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:05.900802 26049 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:05.900117 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:05.900120 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.901830 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:05.842031 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:05.791595 31701 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:26:05.901878 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.915940 26051 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:05.891058 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:05.902915 25938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60947.3us
I20260709 12:26:05.912585 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.913971 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.918262 26049 raft_consensus.cc:2376] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14. Re-sending same reply.
W20260709 12:26:05.919836 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.921869 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:05.923062 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.923419 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63063.2us
I20260709 12:26:05.924544 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.925395 26048 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:05.926523 30780 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:05.928555 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:05.920922 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.930686 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.922717 26050 raft_consensus.cc:2376] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15. Re-sending same reply.
I20260709 12:26:05.931238 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.931519 26049 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:05.935398 30780 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.937965 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:05.851557 31902 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:05.904671 31836 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.671s	user 0.008s	sys 0.003s
I20260709 12:26:05.898872 31680 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.948179 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46404.4us
I20260709 12:26:05.948969 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:05.953410 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:05.954043 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.914970 31677 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:05.926338 31806 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:05.938460 31905 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.272s	user 0.000s	sys 0.004s
I20260709 12:26:05.960391 26048 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:05.930004 31782 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.941953 31631 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.944501 31514 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.932529 31664 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:05.944856 31362 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.001s
W20260709 12:26:05.978361 26094 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.728s (SENT)
I20260709 12:26:05.985680 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:05.957573 31894 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.980573 26201 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:05.930967 31751 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:05.950145 31869 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.965926 31902 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.933284 31793 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:26:05.958271 31909 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.959594 31680 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.960987 31415 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.948392 31836 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:05.979450 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.000550 31631 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.018515 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.012033 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.016968 26203 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:26:06.017184 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:05.978446 31806 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.023012 31793 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:06.023602 31793 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:06.024583 31793 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.025856 31793 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.019891 26128 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
W20260709 12:26:06.040957 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 178494us
W20260709 12:26:06.044061 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.046267 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.049043 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.050457 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:05.966413 31677 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.064978 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.066994 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:05.999051 31901 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:06.069380 31769 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:05.998494 31362 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.084065 26194 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
W20260709 12:26:06.071197 31526 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.629s	user 0.003s	sys 0.008s
W20260709 12:26:06.077931 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 119693us
W20260709 12:26:06.079579 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101249us
W20260709 12:26:06.099807 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.751s (SENT)
I20260709 12:26:06.100394 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.061679 31751 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:06.081588 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.085299 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80648.2us
I20260709 12:26:06.035184 31806 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:06.070911 31631 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:06.070935 31237 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.004s
I20260709 12:26:05.992161 31899 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:26:06.109673 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.030232 31514 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:26:06.026744 31415 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.030537 31909 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.010630 31664 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.107556 31362 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.108374 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.005753 31869 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.165647 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:06.166503 26782 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:06.167171 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.167250 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.167914 26124 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:06.168444 26121 raft_consensus.cc:2410] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:26:06.162773 31237 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:06.177626 31770 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.947s	user 0.010s	sys 0.001s
I20260709 12:26:06.178148 31770 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.091315 31753 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:06.112655 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:06.117965 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3164) took 1175 ms. Trace:
I20260709 12:26:06.186667 31725 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:06.186383 31751 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.187103 31725 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.187521 31725 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.097105 31916 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.189769 31664 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:06.190219 31916 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.101749 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.193171 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:06.193470 31813 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.928s	user 0.009s	sys 0.002s
I20260709 12:26:06.193990 31813 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.194352 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.195000 26120 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:06.195254 26197 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:06.147904 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.197098 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.197166 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1969) took 1125 ms. Trace:
I20260709 12:26:06.156939 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.197860 31836 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:06.197638 26197 rpcz_store.cc:276] 0709 12:26:05.071717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.071882 (+   165us) service_pool.cc:224] Handling call
0709 12:26:06.196824 (+1124942us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400399254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:06.196971 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444136320}
I20260709 12:26:06.160602 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.198770 31818 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.179102 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.181015 31677 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:06.102316 31526 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.202764 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.181883 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.182525 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.185060 31812 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:06.205606 26126 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:06.205221 26045 rpcz_store.cc:276] 0709 12:26:04.942256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.147855 (+205599us) service_pool.cc:224] Handling call
0709 12:26:05.152036 (+  4181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.780954 (+628918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.117602 (+336648us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54000f8f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:06.117881 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423571840}
I20260709 12:26:06.206916 26782 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:06.198470 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:06.157977 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44111us
W20260709 12:26:06.204754 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44778us
W20260709 12:26:06.186398 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73859.8us
I20260709 12:26:06.185397 31680 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:06.195869 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.211217 31912 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.210398 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:06.212450 26120 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:26:06.215523 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.281939 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 73316us
I20260709 12:26:06.283036 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:06.285338 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.174352 31610 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.287781 26129 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:06.207517 31415 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:06.211859 31514 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.292210 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:06.298262 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.289954 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 79640.3us
W20260709 12:26:06.314384 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.100903 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 142817us
I20260709 12:26:06.325827 26121 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:06.316725 31916 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:06.332418 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.333088 31610 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:06.325286 31514 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:06.337244 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.339067 31875 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:06.339527 31875 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:06.342295 31782 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:06.343269 31892 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.921s	user 0.000s	sys 0.006s
I20260709 12:26:06.348609 31514 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:06.362504 31770 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.363587 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1930) took 1351 ms. Trace:
I20260709 12:26:06.365242 26122 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:06.364058 26047 rpcz_store.cc:276] 0709 12:26:05.011928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.174507 (+162579us) service_pool.cc:224] Handling call
0709 12:26:05.184671 (+ 10164us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:05.185101 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.185133 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:05.188937 (+  3804us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.288183 (+ 99246us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:05.499750 (+211567us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.500028 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.500099 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.164150 (+664051us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:06.363258 (+199108us) raft_consensus.cc:1685] finished
0709 12:26:06.363363 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.363527 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":274778,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":3239,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":18160,"spinlock_wait_cycles":6186368,"thread_start_us":281221,"threads_started":2,"wal-append.queue_time_us":136328}
I20260709 12:26:06.368782 31875 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.369819 31909 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.373030 31875 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:06.380326 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47738.5us
I20260709 12:26:06.316009 31362 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:06.384699 31362 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:06.385033 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.385639 31362 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.386525 31680 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.291671 31567 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 12:26:06.389292 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:06.323027 31914 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:26:06.394186 31902 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:06.403121 26756 leader_election.cc:400] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:06.406780 26756 leader_election.cc:403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:26:06.398131 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104519us
I20260709 12:26:06.396257 31921 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.410964 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.435813 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.436721 26121 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:06.383908 31725 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.406189 31610 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:06.457927 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47054.6us
I20260709 12:26:06.463289 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:06.465092 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 140596us
I20260709 12:26:06.471168 26120 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:26:06.436383 31836 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
W20260709 12:26:06.473629 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70744.6us
I20260709 12:26:06.502830 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.504542 31886 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.435716 31922 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:26:06.505895 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:06.416522 31567 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:26:06.416821 31913 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:26:06.507251 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.543596 26200 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:06.508607 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.509377 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:06.510862 26197 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:06.513206 26199 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:06.514426 26198 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:06.515923 26195 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:06.416750 31812 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:06.458508 31808 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.520714 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.522051 31912 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.523972 26052 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:06.525486 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.525733 31725 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:06.548961 26047 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:06.528447 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.530015 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.531893 26046 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:06.553368 26049 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:06.549315 26053 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 12.
I20260709 12:26:06.436187 31753 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:06.502199 26094 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11.
I20260709 12:26:06.535714 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.556808 26094 leader_election.cc:403] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:06.557044 26051 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:06.537629 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.505102 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:06.541812 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.542080 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.544111 26194 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:06.559234 26790 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:06.547269 26756 leader_election.cc:400] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14.
I20260709 12:26:06.458319 31921 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.473796 31237 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:26:06.526966 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.564805 26045 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:06.554833 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.565425 26202 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:06.555765 26121 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:06.555941 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.559805 26756 leader_election.cc:403] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:26:06.560993 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.543155 31680 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.562372 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.565287 26129 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:06.548846 31381 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.551985 31514 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.566627 26198 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:06.565814 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:06.566797 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.566320 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.566819 26120 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:26:06.571374 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.549476 31886 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:06.549096 31770 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:06.574288 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72240us
I20260709 12:26:06.574086 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:06.576035 26123 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:26:06.576507 26200 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:06.556668 31753 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:06.572227 31095 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 10. Current term is 11.
I20260709 12:26:06.575876 31677 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:06.604403 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:06.605361 31921 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.605500 31886 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:06.606616 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59523.7us
I20260709 12:26:06.607638 31526 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.608672 31770 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.578581 31514 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.609449 31769 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:06.564827 31631 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:26:06.614634 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:06.615568 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:06.616062 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:06.619886 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
W20260709 12:26:06.612344 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48471.5us
I20260709 12:26:06.560752 31806 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:26:06.555704 31808 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.678612 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:06.653393 26050 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.564532 31915 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:06.630923 26198 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:06.701349 26202 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:26:06.707957 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.701774 26129 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:06.708873 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:06.576205 31381 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.657603 31664 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:06.718343 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:06.815917 26121 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:26:06.622164 31753 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:06.819319 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 110564us
I20260709 12:26:06.820325 26121 pending_rounds.cc:85] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 10
I20260709 12:26:06.820779 26201 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:06.828348 26121 pending_rounds.cc:107] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 16.11
W20260709 12:26:06.816605 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 200617us
I20260709 12:26:06.831068 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:06.826115 26047 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:06.817101 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:06.816725 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 202291us
I20260709 12:26:06.827847 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:06.828671 26121 raft_consensus.cc:2889] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.832324 26782 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:06.828918 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.572907 31725 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14.
I20260709 12:26:06.623874 31920 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:06.748777 31928 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:26:06.746032 31770 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.700871 31612 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.720731 31813 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.773135 31909 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:06.841789 31793 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:06.842548 31095 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.843150 31875 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:06.815865 31915 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:06.843665 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 228531us
I20260709 12:26:06.864027 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.772794 31526 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.772518 31902 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.674396 31924 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.172s
I20260709 12:26:06.851058 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:06.782102 31677 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.742465 31535 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:06.749886 31869 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.751355 31514 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.815178 31769 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.856428 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:06.818549 31916 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:06.894359 31806 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.858208 31095 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:06.815526 31664 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.909467 31916 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:06.849766 31920 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.914600 31916 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:06.859477 31770 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:06.861349 31909 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.859685 31875 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.915937 31664 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:06.624553 31782 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.223s
I20260709 12:26:06.894055 31912 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:06.857738 31612 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:06.911747 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50921.9us
I20260709 12:26:06.899276 31237 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.897578 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.900159 31535 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:06.905786 31924 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:26:06.926836 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75886.2us
I20260709 12:26:06.907023 31769 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.916484 31770 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.916307 31920 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.931293 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:06.932603 26788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:53642
I20260709 12:26:06.932961 31095 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.937644 31909 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.943188 31753 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.944063 31915 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:06.916700 31806 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.919682 31875 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.945679 30780 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:06.893893 31526 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.947567 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.913884 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
W20260709 12:26:06.952992 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.991329 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.992437 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:06.993270 26126 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.946345 31381 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:06.901859 31631 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.975381 26788 raft_consensus.cc:606] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.903271 31902 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.925719 31664 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:06.946741 31913 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:06.977284 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49066.2us
I20260709 12:26:06.901361 31677 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:06.926913 31924 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:06.948601 31793 raft_consensus.cc:2764] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 12:26:07.018853 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173165us
I20260709 12:26:06.993129 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:07.019657 26121 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.995105 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:06.964560 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 133544us
I20260709 12:26:06.987238 31922 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:07.001451 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53981.1us
I20260709 12:26:06.976085 31770 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:06.975112 31869 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.030467 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.006305 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:06.930897 31237 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.016276 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.017431 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.034580 26123 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:06.987423 30780 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:07.008821 31913 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.023322 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.001497 26788 raft_consensus.cc:614] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:07.026278 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.039350 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.042479 31909 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:07.043150 31909 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.043720 31909 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.047586 31806 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.035993 31869 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.048955 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.050132 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.050139 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.009943 31942 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:07.056334 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 130755us
I20260709 12:26:07.061722 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.050410 30780 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.002848 31567 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 12:26:07.036957 31938 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.110s	user 0.000s	sys 0.003s
I20260709 12:26:07.064875 26120 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:07.025624 31922 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.066075 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.071555 26122 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:07.079217 31942 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:07.079741 31948 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.079988 31942 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:07.052914 31913 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.047653 31944 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.014542 31664 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.008643 31631 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.116160 26120 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:07.105314 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.085852 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.060590 31921 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:07.055533 31782 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.107227 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.118065 31782 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.119314 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.120420 26127 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:07.120929 31782 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:07.108178 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.121555 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.122119 26129 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:07.123304 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.056885 31894 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:07.054436 31808 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.066877 31701 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.060362 31793 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:07.055655 31806 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:07.081751 31922 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.108688 26121 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:07.123378 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.124872 26127 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:07.010288 31535 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.140843 26129 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:07.126207 31725 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:07.146216 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:07.146603 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:07.158751 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72989.5us
I20260709 12:26:07.159847 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:07.130160 31869 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.163199 31782 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:07.166760 26127 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 11: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:07.179536 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.181306 26788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:53642
W20260709 12:26:07.215854 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44507.3us
I20260709 12:26:07.160735 31701 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:07.223223 26788 raft_consensus.cc:606] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.223670 26788 raft_consensus.cc:614] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:07.220829 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:07.221356 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61583.4us
I20260709 12:26:07.225184 26203 raft_consensus.cc:2376] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:26:07.223485 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.227332 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.230221 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:07.231058 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:07.232939 26197 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:07.235292 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 195986us
I20260709 12:26:07.238632 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.172151 31535 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.177556 31725 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:07.240597 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.241914 26052 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:07.215023 31808 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.226866 31701 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.219738 31921 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.245690 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:07.257867 26791 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:07.249075 31362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.002s	sys 0.030s
I20260709 12:26:07.252842 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:07.259238 31362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.002s	sys 0.030s
I20260709 12:26:07.259853 31362 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.246364 26783 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:07.214479 31806 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:26:07.245222 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 139993us
I20260709 12:26:07.234289 31612 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.272064 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:07.276098 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 219130us
I20260709 12:26:07.280092 26197 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:07.281706 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.130471 31944 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.113479 31948 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.253818 31701 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:07.288853 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65440.5us
I20260709 12:26:07.254571 31753 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:07.292071 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.265909 31920 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:07.377089 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 85082.1us
W20260709 12:26:07.373210 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.300686 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.081297 31924 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.248885 31869 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:26:07.256346 31808 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:07.444833 31955 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:07.455668 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.456869 31806 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.457576 31631 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.458366 31631 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:07.447996 31769 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.461134 31664 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.461815 31808 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.461903 31664 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:07.462641 31808 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:07.463490 31808 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:07.463908 31808 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.466596 31612 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.467321 31612 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.440805 31948 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:07.444833 31943 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
W20260709 12:26:07.445636 31954 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:26:07.446291 31947 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:26:07.448570 31961 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:07.449328 31960 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:07.263152 31921 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.267769 31362 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.479163 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.398581 31920 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.486544 26126 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
W20260709 12:26:07.491986 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 191323us
W20260709 12:26:07.498597 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 212190us
I20260709 12:26:07.497829 31526 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.460065 31631 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:26:07.457527 31806 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.533010 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:07.540349 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.553565 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.554253 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.541921 31362 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:07.530392 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:07.498164 31808 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.540489 31677 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:07.553148 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.418746 31725 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.559316 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:07.564170 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3183) took 1021 ms. Trace:
I20260709 12:26:07.511852 31920 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.555871 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.586282 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.508540 31961 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:07.590006 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 203821us
I20260709 12:26:07.587821 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:07.592382 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 320383us
I20260709 12:26:07.592521 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.589905 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:07.593201 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.570945 26045 rpcz_store.cc:276] 0709 12:26:06.542648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.722049 (+179401us) service_pool.cc:224] Handling call
0709 12:26:06.751860 (+ 29811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.782282 (+ 30422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.782329 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.782910 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.950382 (+167472us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:06.950437 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.950660 (+   223us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.563789 (+613129us) raft_consensus.cc:1685] finished
0709 12:26:07.563822 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.564065 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29393,"raft.queue_time_us":29783,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":166933,"threads_started":1,"wal-append.queue_time_us":359627}
I20260709 12:26:07.584532 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.564745 31812 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:07.571091 31909 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:26:07.563462 31526 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:07.515275 31952 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:26:07.596415 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.503400 31875 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.603785 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.589293 31514 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:07.591347 31960 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:07.570117 31968 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:07.623600 31526 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.623453 31362 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:07.593197 31948 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:07.595309 31782 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:07.596871 31769 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:07.621747 31895 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:07.625388 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:07.643218 31770 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.000s
W20260709 12:26:07.625947 31916 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.711s	user 0.013s	sys 0.000s
W20260709 12:26:07.646813 31913 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.005s
W20260709 12:26:07.654716 31886 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.009s	sys 0.017s
W20260709 12:26:07.685766 31886 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.010s	sys 0.017s
I20260709 12:26:07.639389 31922 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.685952 31974 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:07.636169 31806 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:07.687641 31961 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.691864 26782 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:07.606509 31944 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:07.692939 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67640.4us
I20260709 12:26:07.698586 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:07.637564 31921 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.645607 31612 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:07.655596 31948 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.658004 31782 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:07.725407 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 125204us
I20260709 12:26:07.655345 31526 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:07.757140 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 160806us
W20260709 12:26:07.737041 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140703us
I20260709 12:26:07.758984 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.656877 31362 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.706427 31732 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:07.626434 31909 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.649477 31701 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.706449 31770 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.706511 31916 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.706823 31913 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:07.793704 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.725553 31948 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.823199 31732 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.792800 31922 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:07.767863 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 164420us
I20260709 12:26:07.737586 31972 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:07.756558 31567 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.762650 31971 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:07.828722 31982 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.830232 31362 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:07.830312 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:07.830961 31979 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:07.852890 26194 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:07.852075 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:07.854652 26197 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:07.831341 31875 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:07.831447 31916 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.844420 31732 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.875988 31812 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.832029 31770 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:07.705862 31813 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:07.736644 31921 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.786830 31973 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:07.832737 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:07.833120 26128 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:07.836166 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:07.887856 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 43752.6us
I20260709 12:26:07.844518 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:07.863730 31920 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:07.805253 31959 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:26:07.845948 31972 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.896579 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.857824 31909 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.858361 31915 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:07.863205 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:07.845721 31974 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:07.864828 31977 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:07.865394 31963 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:26:07.867812 30780 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.864740 31944 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.901120 31972 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.910903 31944 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.913172 31944 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:07.917248 31902 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:07.869860 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 110984us
I20260709 12:26:07.920171 26197 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:07.920574 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.728668 31782 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:07.873831 31913 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:07.886932 26788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:53642
I20260709 12:26:07.889992 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:07.769896 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 488270us
I20260709 12:26:07.924046 26788 raft_consensus.cc:606] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.924657 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.894381 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.893886 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.925669 26194 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:07.926671 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:07.896334 26124 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:07.855751 31567 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:07.897254 26195 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:07.785513 31968 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:07.930370 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.896154 31960 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:07.934782 26199 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:07.941077 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.894994 31921 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.947834 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:07.951869 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:07.945477 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.895803 31526 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.959026 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:07.960013 26049 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:07.960443 26195 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:07.960912 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:07.959511 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.944254 31818 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:07.934437 31806 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:07.964028 31988 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:07.963580 26196 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:07.963773 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:07.921448 31981 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:07.966511 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.968490 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:07.920866 31808 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.973392 26054 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.968807 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:07.967880 26049 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.974299 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:07.969058 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:07.975390 26045 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:07.969539 26052 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:07.976001 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.970495 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:07.977818 26051 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:07.894820 31973 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:07.971748 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:07.979491 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:07.974318 26048 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.968202 26196 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:07.950508 31902 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.986434 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.987162 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.957401 31631 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:07.990240 26788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:53642
I20260709 12:26:07.968211 31664 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.991753 26788 raft_consensus.cc:606] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:07.993047 26788 raft_consensus.cc:614] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:07.993932 26195 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:07.994920 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:07.995752 26783 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:07.953016 31909 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:26:07.965540 31818 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:07.997637 26049 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:08.000252 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:07.919761 31977 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.001878 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:07.968307 31957 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:07.964859 31970 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:08.012475 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:08.013134 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:07.985280 31875 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.007947 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:07.992571 31982 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:07.965811 31677 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:26:07.996939 31979 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:08.016410 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:26:08.030809 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:08.026759 26788 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:08.027558 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.033512 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:08.039268 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.039111 26054 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:08.005259 31631 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:07.989626 31902 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:08.043886 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43750.4us
I20260709 12:26:08.047466 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:08.049211 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:08.049355 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.050428 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:08.056975 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.028802 31979 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:08.062001 26788 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:08.066026 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:07.986181 31981 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.041049 31970 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:08.046077 31909 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.972203 31974 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.098824 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.099629 26790 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:08.003232 31968 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.101019 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.071722 31979 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:08.008462 31960 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.060868 31732 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.122031 31977 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:08.078691 31806 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 12:26:08.130956 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81684us
I20260709 12:26:08.131765 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.132396 26048 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:08.135013 31664 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:08.141031 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.087678 31909 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.060961 31818 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.135394 31977 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.142995 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.154994 26047 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:08.132578 31987 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:08.174759 26201 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:08.150686 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.175175 32003 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.116187 31631 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.178961 30780 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.183736 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.175887 31526 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.142763 31921 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.184908 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.195685 26201 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:08.185747 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.168608 31991 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.165273 32000 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:08.164006 31808 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.154003 31567 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.176445 31818 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.176954 31987 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.223044 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.208153 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.206671 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.208369 30780 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:08.223723 26783 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:08.223513 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:08.221652 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 319583us
I20260709 12:26:08.168731 31977 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.224807 26782 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.171056 31677 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:26:08.225873 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.227705 31567 raft_consensus.cc:2764] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:26:08.232823 26788 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:08.213284 31991 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.237970 26789 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.238062 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.238844 26199 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.239261 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.240324 26789 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.240497 26201 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:08.242794 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:08.242131 30780 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:08.169447 31970 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.246817 26785 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:08.197697 32003 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.248019 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.248759 32000 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:08.249411 32003 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.249419 32000 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:08.249893 31732 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.241473 31988 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.250429 32000 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.250787 31988 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.253441 31988 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.248821 31902 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:08.245843 30780 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.231896 31875 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:08.256142 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.258780 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.244370 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:08.256129 30780 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.237201 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.259660 26788 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:08.259966 26198 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:08.256559 26785 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:08.261296 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.262603 31751 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:08.262897 32002 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.263793 31732 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:26:08.263302 31751 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.266654 31751 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:08.268551 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.269882 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.271152 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.271872 26787 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:08.272588 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.223232 31973 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.273653 31567 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:08.220600 31381 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:08.275271 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.295586 26784 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:26:08.292923 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2337) took 1097 ms. Trace:
I20260709 12:26:08.296402 26198 rpcz_store.cc:276] 0709 12:26:07.195507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.219092 (+ 23585us) service_pool.cc:224] Handling call
0709 12:26:08.243972 (+1024880us) spinlock_profiling.cc:243] Waited 914 us on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:26:08.292846 (+ 48874us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103168}
I20260709 12:26:08.234624 31987 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:08.307969 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 165095us
I20260709 12:26:08.309149 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.309711 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.311512 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.318326 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.319186 30780 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.298450 31971 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.322751 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.323624 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.324440 26784 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.324713 26783 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:08.326550 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.214004 31972 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.329552 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.349205 26783 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:08.272032 31948 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.356652 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.360970 26194 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:08.321537 32002 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.353228 30780 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.353539 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:08.350593 26789 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.292254 31362 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:26:08.358723 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.329804 31990 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.345229 31808 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:08.323156 31812 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:26:08.307932 31894 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:08.371011 26788 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.327764 31677 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:26:08.308256 31381 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.432516 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.350196 31981 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.384728 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.433004 31237 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.350358 31862 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:08.385063 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.433698 31862 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:08.348176 32003 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:26:08.387871 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.434688 31862 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.435609 26201 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.386224 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.436101 31862 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:08.436563 26047 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:08.436882 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3261) took 1441 ms (client timeout 1796 ms). Trace:
W20260709 12:26:08.437309 26128 rpcz_store.cc:269] 0709 12:26:06.994983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.030187 (+ 35204us) service_pool.cc:224] Handling call
0709 12:26:08.436659 (+1406472us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:08.436809 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843867520}
I20260709 12:26:08.388938 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.440557 26049 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:08.440693 31915 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.002s
I20260709 12:26:08.390326 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.444548 26054 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:08.444934 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:08.449564 31886 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:08.450032 31886 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:08.450899 31886 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.451864 31924 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.456270 26054 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:08.388392 30780 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.457455 30780 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.368655 31921 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.358180 31514 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:08.387271 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.392727 26050 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:08.463666 26789 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.293908 31875 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.381691 26200 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:08.370682 31813 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:08.473397 26054 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:08.398191 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.402761 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:08.477864 26790 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.402894 31701 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:08.393281 31962 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:08.417937 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.796s (SENT) [suppressed 43 similar messages]
I20260709 12:26:08.482255 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:08.486735 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68834.1us
I20260709 12:26:08.487432 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.418064 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.485019 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.489948 26052 raft_consensus.cc:2376] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14. Re-sending same reply.
I20260709 12:26:08.419457 26784 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:08.422734 26785 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:08.492055 26199 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.424417 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.425222 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:08.493808 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.495276 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.427121 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.428539 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.496613 26129 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:08.429942 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:08.430223 31994 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
W20260709 12:26:08.431641 32001 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 12:26:08.433235 26121 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.460131 26051 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:08.542932 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:08.472072 32008 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:08.460626 26047 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:08.543035 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:08.546635 26202 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:08.543820 26790 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:08.547425 26054 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.390359 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.483279 31381 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:08.552098 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.469847 26201 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:08.396931 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:08.479053 30780 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:08.452142 31886 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.395052 31909 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:08.480485 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 82413.8us
I20260709 12:26:08.482512 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.571408 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.482847 26050 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.483742 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.500947 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:08.493227 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:08.572715 26128 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:08.450368 31987 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.496986 26786 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:08.576689 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.473915 31751 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.525799 26126 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:08.472389 31514 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.393672 31970 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 12:26:08.542446 26789 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:08.353974 31971 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.549762 26783 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:08.466993 26049 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:08.566805 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.490839 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:08.603233 26047 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:08.543759 31912 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.600579 26784 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:08.548619 31237 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:08.608171 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.528585 31979 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.548767 31894 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:26:08.545437 31812 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:26:08.634658 26054 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 12. Current term is 13. Ops: [9.6-12.7]
I20260709 12:26:08.617983 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.612176 26049 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:26:08.679975 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.630757 26791 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.594167 31952 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.605327 32012 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:08.620410 31869 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:08.675443 26050 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:08.690004 26050 pending_rounds.cc:85] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 7
I20260709 12:26:08.690358 26050 pending_rounds.cc:107] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:08.676786 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.620814 31237 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.605914 31962 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:08.630857 32002 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:08.658320 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.609817 31912 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.646082 31972 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:08.679394 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.621657 26127 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:26:08.613910 31981 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:08.617606 31567 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:08.679020 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.686702 26194 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:08.676059 26782 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:08.601471 31909 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.690564 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.689255 31769 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:08.690706 26050 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:08.734120 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55207.2us
I20260709 12:26:08.732194 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.737066 26054 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.694140 31961 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:08.732558 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:08.732918 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53646.3us
I20260709 12:26:08.688823 31869 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.693760 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:08.742983 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49311.1us
I20260709 12:26:08.743649 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.745271 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.731638 31567 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.735332 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.731370 31962 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:08.749272 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:08.730140 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 123533us
W20260709 12:26:08.712396 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 221624us
I20260709 12:26:08.739209 26194 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:08.739466 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.741822 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.691771 31952 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.738440 31806 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:08.742213 31631 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:08.752861 31920 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:08.753077 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.772739 26049 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:08.755686 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.773149 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.756916 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.734463 31957 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.737300 32015 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:08.729955 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.736564 31769 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:08.746556 31869 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.748275 31961 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:08.779699 26054 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:08.748863 31970 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:08.756350 31567 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:08.748202 31972 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:08.787201 31782 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.864s	user 0.008s	sys 0.005s
I20260709 12:26:08.780273 31381 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:08.788853 32024 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:08.759388 31612 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:08.733304 31981 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.795096 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:08.778367 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.795801 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:08.821030 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52687.6us
W20260709 12:26:08.830945 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69989.4us
I20260709 12:26:08.823546 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:08.832351 26054 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:08.809383 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:08.835378 32030 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:08.640218 31895 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 3 to 12, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:08.771340 31875 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.796803 31769 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:08.821331 32026 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:26:08.842013 31979 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.767902 31942 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:08.783399 31987 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.741348 31909 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.783694 31973 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:08.783897 31957 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.864850 31751 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.844444 31782 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:08.895925 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100248us
I20260709 12:26:08.889110 31894 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:08.906785 26045 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:08.876912 31915 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:26:08.913286 26203 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:08.913456 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48667.7us
I20260709 12:26:08.914633 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.884131 31793 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:26:08.776719 31631 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.905577 31970 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:08.903537 31862 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.919672 31912 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:08.923969 32000 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.003s
I20260709 12:26:08.911758 31894 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:08.895855 32030 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:26:08.909511 31812 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:08.923941 31782 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:08.945971 32023 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.203s	user 0.003s	sys 0.000s
I20260709 12:26:08.950079 31972 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:08.952029 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 173729us
I20260709 12:26:08.952821 31956 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:09.004854 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:09.004928 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.953425 31563 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:08.953544 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 158554us
I20260709 12:26:09.013206 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:09.013672 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:08.953807 32013 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:08.957418 26054 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:08.960876 31612 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:08.964669 32021 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
W20260709 12:26:08.965153 32020 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:26:08.965333 32035 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:08.966432 31981 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:08.966612 32031 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 12:26:08.967553 32018 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:26:08.968011 32032 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:08.968675 32029 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:26:08.969347 31952 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.902768 31961 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.934008 31862 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:08.935040 31095 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:08.881320 31962 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:08.971233 31957 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:08.937815 31912 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:26:08.973218 31920 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:26:08.973311 31812 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:08.974725 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47100.3us
I20260709 12:26:09.065165 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:09.066553 31912 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:09.067342 31912 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:26:08.965071 32022 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:26:09.061945 32032 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:08.977033 32030 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:08.935516 32000 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.069695 32032 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.072642 32000 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:08.980397 32016 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:26:09.085274 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:09.065104 31920 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:09.086866 31920 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:09.088039 31920 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.090090 26046 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.993249 31894 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.093166 31894 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.094398 31948 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:09.056208 31095 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:08.993459 31935 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:08.996728 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:08.997651 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.015394 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:08.988998 32026 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:09.022927 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:08.982108 31924 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 12:26:09.016872 31782 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:09.043044 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:08.978531 31909 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:09.045737 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 129372us
I20260709 12:26:09.050882 31972 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.051726 31514 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.020810 32015 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.017980 32013 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.012485 31563 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:09.132967 31563 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.134338 31563 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.134778 26046 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:09.020794 31381 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:09.058692 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:09.025125 31973 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:09.103461 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88166.5us
I20260709 12:26:09.096894 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2345) took 1097 ms. Trace:
I20260709 12:26:09.089747 31920 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:09.100468 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:09.110023 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 87174.5us
I20260709 12:26:09.179343 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:09.180591 31924 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:09.181094 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:09.182363 31924 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:09.182884 31924 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:09.040967 31612 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:09.113216 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70257.2us
I20260709 12:26:09.183873 31924 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.184916 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:09.186292 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:09.115121 26053 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:26:09.187616 31960 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:09.188346 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.048879 31987 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:26:09.115989 31818 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:09.120383 26048 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:26:09.164490 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.208761 26196 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:09.189194 26053 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:26:09.210695 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.153287 26197 rpcz_store.cc:276] 0709 12:26:07.998983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.147568 (+148585us) service_pool.cc:224] Handling call
0709 12:26:08.148202 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.293386 (+145184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.293415 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.313668 (+ 20253us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.502694 (+189026us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:08.502732 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.754886 (+252154us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:08.754959 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.096551 (+341592us) raft_consensus.cc:1685] finished
0709 12:26:09.096575 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.096827 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":448,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":10734,"spinlock_wait_cycles":332816256,"thread_start_us":139127,"threads_started":1,"wal-append.queue_time_us":255915}
I20260709 12:26:09.188508 31960 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:09.204824 26049 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:09.097004 31095 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.169509 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.114435 31770 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.114428 31979 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.996774 31970 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.112270 31948 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:09.112820 31782 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:09.251087 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150717us
I20260709 12:26:09.148581 31862 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.208537 31818 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:09.265152 32052 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:09.266382 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:09.148284 32013 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.189432 32040 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:09.177943 32046 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.157s
I20260709 12:26:09.272358 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:09.214004 31960 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.273340 31782 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:09.260717 31793 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.189201 31912 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 12:26:09.316570 32052 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:09.280313 31818 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:09.317255 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45003.5us
W20260709 12:26:09.317414 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.327829 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.315203 32040 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.313803 31960 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.268271 31770 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.371527 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.374256 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.375147 26198 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:09.380633 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.387640 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.398591 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.399945 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:09.420565 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 234309us
I20260709 12:26:09.438228 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.439039 26045 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:09.396406 31909 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.346414 31952 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 12:26:09.354296 31971 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.347092 31793 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.481854 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:09.482072 31886 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.905s	user 0.007s	sys 0.007s
I20260709 12:26:09.499661 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:09.408277 32032 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:26:09.444381 32051 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 12:26:09.512596 31514 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:09.539873 31869 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.747s	user 0.010s	sys 0.000s
I20260709 12:26:09.540537 31869 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.482470 31909 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.548046 26198 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:09.547672 31895 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:09.552532 31895 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:09.552361 32049 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:26:09.550324 32003 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:09.524948 32040 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:09.566289 32055 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:26:09.558109 32003 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.569046 32003 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:09.564592 31567 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.000s
I20260709 12:26:09.569907 31567 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.553601 31895 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:09.567988 32054 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 12:26:09.571954 31769 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.690s	user 0.004s	sys 0.003s
I20260709 12:26:09.452596 31948 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:09.593044 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:09.606638 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 214623us
I20260709 12:26:09.608294 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1968) took 1118 ms. Trace:
I20260709 12:26:09.579033 31095 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.580503 31987 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:09.583053 31793 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:09.572050 31567 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.574728 32008 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.615918 26050 rpcz_store.cc:276] 0709 12:26:08.489445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.773988 (+284543us) service_pool.cc:224] Handling call
0709 12:26:08.831150 (+ 57162us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.901207 (+ 70057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.901266 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.996443 (+ 95177us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b2c000014c8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:26:08.996866 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.196001 (+199135us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:09.301507 (+105506us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.301759 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.302774 (+  1015us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.607896 (+305122us) raft_consensus.cc:1685] finished
0709 12:26:09.608005 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.608174 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61280,"prepare.queue_time_us":87954,"prepare.run_wall_time_us":37,"raft.queue_time_us":83228,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":35926,"spinlock_wait_cycles":16306816,"thread_start_us":268299,"threads_started":2,"wal-append.queue_time_us":302991}
I20260709 12:26:09.621240 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:09.643302 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.644723 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.645135 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.645406 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.646173 26788 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:09.646205 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.647317 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.647890 26788 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.649273 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.650120 26790 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:26:09.650231 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.649950 32008 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.656673 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.656978 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:09.658937 26050 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:09.652437 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:09.660403 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 248956us
I20260709 12:26:09.618986 31979 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:09.663620 26783 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:09.619835 31909 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:09.613886 31960 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.640127 31987 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.670531 31960 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.670665 31987 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.674502 31782 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.613725 31769 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:09.681492 31563 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.003s
I20260709 12:26:09.665828 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.682435 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.690083 26788 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:09.690085 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.691509 26783 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 13: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:09.691325 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.692221 26784 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:09.666602 32015 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.004s
I20260709 12:26:09.664857 31793 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:09.699800 26199 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:09.700888 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.701519 26784 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.704025 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:09.695662 26788 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:09.695842 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.705061 26783 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.706264 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:09.581480 31818 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.709565 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:09.694458 31869 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:09.713903 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:09.715082 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.723019 26788 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:09.726905 26788 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:09.699975 31793 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.727672 26050 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:09.730276 26784 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:09.733568 26050 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:09.708372 32015 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:09.750931 26783 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:09.729828 31962 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:09.736462 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.755585 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99462.2us
I20260709 12:26:09.692466 32069 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 12:26:09.759670 32062 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:26:09.760792 32058 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.150s
I20260709 12:26:09.762439 31961 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:09.713169 31909 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:09.713380 32072 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.779959 26050 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:09.720712 31957 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:09.756467 31664 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:26:09.784857 31782 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:09.786149 32073 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:09.694219 31948 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.788939 31769 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:09.790557 31818 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:09.822285 31991 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:09.804324 32072 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.827183 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 969) took 1833 ms. Trace:
I20260709 12:26:09.827561 26791 rpcz_store.cc:276] 0709 12:26:07.993978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.994088 (+   110us) service_pool.cc:224] Handling call
0709 12:26:07.994353 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.680894 (+686541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.826875 (+1145981us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54001162d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:09.827115 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423740160}
I20260709 12:26:09.730648 32032 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.732841 31793 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:09.836550 32032 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.839735 32032 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.824090 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.806893 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:09.842167 26195 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:09.842394 26788 raft_consensus.cc:2376] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18. Re-sending same reply.
I20260709 12:26:09.807258 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3217) took 1013 ms. Trace:
I20260709 12:26:09.843195 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.808075 26047 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:09.843080 26046 rpcz_store.cc:276] 0709 12:26:08.793660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.823264 (+ 29604us) service_pool.cc:224] Handling call
0709 12:26:09.807020 (+983756us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:09.807172 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":534062848}
I20260709 12:26:09.845171 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:09.829250 32072 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:09.846220 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:09.846869 26046 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:09.847595 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.794023 32013 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:09.848014 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.848119 26046 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:09.862839 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.877643 26197 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:09.867285 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3213) took 1132 ms. Trace:
I20260709 12:26:09.877954 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:09.878119 26054 rpcz_store.cc:276] 0709 12:26:08.735059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.139846 (+404787us) service_pool.cc:224] Handling call
0709 12:26:09.140104 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.352448 (+212344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.352479 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.352520 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:09.352554 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.859958 (+507404us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:09.860033 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.860225 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487859584}
I20260709 12:26:09.879598 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.880092 26053 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:09.862203 31782 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:09.880999 31782 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.883618 26046 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:09.890204 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.890995 26198 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:09.895699 31812 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:09.855759 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1317) took 1618 ms. Trace:
I20260709 12:26:09.261514 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:09.899276 26782 rpcz_store.cc:276] 0709 12:26:08.236947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.239327 (+  2380us) service_pool.cc:224] Handling call
0709 12:26:08.239624 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.734203 (+494579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.855438 (+1121235us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:09.855677 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939840768}
I20260709 12:26:09.892168 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.907644 26198 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:09.912442 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:09.918977 26783 raft_consensus.cc:2368] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18.
I20260709 12:26:09.920776 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.853003 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:09.925238 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:09.930239 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:09.931186 26198 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.936084 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:09.937040 26197 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:09.942016 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.943271 26195 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:09.906432 31782 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:09.781466 32015 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 12:26:09.920349 31991 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:09.923686 32002 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:09.956085 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:09.958555 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:09.939956 32069 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:09.969046 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.973398 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2351) took 1613 ms. Trace:
I20260709 12:26:09.983457 26196 rpcz_store.cc:276] 0709 12:26:08.359981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.408796 (+ 48815us) service_pool.cc:224] Handling call
0709 12:26:08.409112 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.213772 (+804660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.973047 (+759275us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54002b4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:09.973316 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1838558080}
I20260709 12:26:09.959735 31913 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.978657 32008 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:09.974316 31986 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:09.973480 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:09.984419 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.961764 32003 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:09.992122 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 11.
W20260709 12:26:09.995885 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63859.1us
I20260709 12:26:09.995937 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:09.998211 26783 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:10.000360 26756 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:09.986443 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:10.011464 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:09.998229 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:10.007475 32013 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:10.007875 26054 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:10.009322 32045 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:09.999858 31514 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
W20260709 12:26:09.961594 31612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.013s	sys 0.007s
W20260709 12:26:10.026783 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.043227 26198 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:09.974390 32069 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.050879 32013 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:10.048141 31664 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:26:10.053454 31664 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:10.053934 31664 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:09.994642 32003 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:10.054955 32003 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:10.054762 31664 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.990432 31913 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:09.994182 31986 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.058459 31913 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.059069 31986 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:09.998629 31968 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:10.056116 31664 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:10.061447 31968 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.048382 31862 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:10.064581 31612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.013s	sys 0.008s
I20260709 12:26:10.067526 32008 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:10.067950 32049 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:10.046466 26756 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:10.047370 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:10.069614 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.070005 26196 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:09.994818 31973 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:09.961463 31991 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.054447 31957 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:10.072685 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:10.082453 26045 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.961105 32002 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.087055 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.087836 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.071775 31913 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:09.994421 31793 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.073825 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:10.074311 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:10.074833 31915 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:10.075819 31514 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:10.076174 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 1992) took 2188 ms (client timeout 1734 ms). Trace:
I20260709 12:26:10.076418 31991 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:10.072938 31812 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:10.074862 31973 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:10.077392 31968 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:10.077915 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:10.109611 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.079319 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:10.089807 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.091002 26196 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.092192 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.111697 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:10.112000 26125 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:10.092450 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.094339 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1962) took 1702 ms. Trace:
W20260709 12:26:10.094237 26203 rpcz_store.cc:269] 0709 12:26:07.887779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.887890 (+   111us) service_pool.cc:224] Handling call
0709 12:26:10.075945 (+2188055us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:10.076102 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270759168}
I20260709 12:26:10.047768 32045 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:10.095942 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.114321 26121 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:10.113690 26052 rpcz_store.cc:276] 0709 12:26:08.474858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.526737 (+ 51879us) service_pool.cc:224] Handling call
0709 12:26:08.544118 (+ 17381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.085992 (+541874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.094107 (+1008115us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b540022b354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:10.094283 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217569024}
I20260709 12:26:10.097436 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.043463 26197 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:10.098547 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.115808 26122 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:26:10.093715 31913 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.093909 31915 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:10.124899 31915 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.126236 31915 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:10.127424 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2012) took 1638 ms. Trace:
I20260709 12:26:10.107815 31612 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:10.128885 31612 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:10.127831 26202 rpcz_store.cc:276] 0709 12:26:08.489291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.565920 (+ 76629us) service_pool.cc:224] Handling call
0709 12:26:08.566377 (+   457us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.608295 (+ 41918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.608329 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:08.667287 (+ 58958us) log.cc:844] Serialized 52 byte log entry
0709 12:26:08.860562 (+193275us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:08.892352 (+ 31790us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.952839 (+ 60487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.952931 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.124452 (+1171521us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b18003a1740. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:10.127156 (+  2704us) raft_consensus.cc:1685] finished
0709 12:26:10.127248 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.127372 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86474,"prepare.queue_time_us":37233,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":87038,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":8670,"spinlock_wait_cycles":13366016,"thread_start_us":128291,"threads_started":1,"wal-append.queue_time_us":188765}
I20260709 12:26:10.064606 31862 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.107951 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:10.109951 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.132846 26129 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:10.068625 32049 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.179915 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.097934 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.116479 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.212913 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:10.214039 26052 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:10.118105 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.214399 32045 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.215888 26123 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:10.215955 32045 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:10.118736 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 182968us
I20260709 12:26:10.217769 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.218935 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.119673 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.089881 31960 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:10.220427 32008 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:10.111187 26127 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:10.103792 31956 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:10.221138 32008 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:10.221501 31956 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.216849 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.222723 26123 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:10.122262 32051 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:10.123064 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.123196 26198 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:10.224606 26124 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:10.224913 32051 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:10.225306 32051 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:10.226171 32051 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.227411 32051 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:10.120179 32030 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:10.228448 32030 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.228847 32030 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.223996 31956 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:10.180150 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:10.109078 31812 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.233033 26122 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:10.143029 31862 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:10.234342 31812 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.103828 32072 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:10.236671 32072 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.239106 31812 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.239701 32072 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:10.239764 31812 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.243512 31812 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.185626 31793 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:10.087226 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87010.9us
W20260709 12:26:10.244801 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:10.188968 26203 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:26:10.190982 32003 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.246608 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:10.188040 32049 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.193892 31913 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.129894 31612 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.100471 32002 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:10.202900 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.181059 31973 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.108522 31957 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.212688 31770 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:10.212879 26128 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.118280 26045 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:10.215152 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.117142 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.269079 26121 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:10.269062 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.270555 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.271394 26045 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:10.271660 26129 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:10.247787 26791 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:10.272446 26791 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:10.248809 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:10.274101 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.274928 26045 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:10.276139 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.279439 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:10.249631 26196 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:10.249814 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:10.251788 26124 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:10.281530 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.282938 26124 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:10.193743 32069 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.254966 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.258119 31664 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:26:10.256790 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:10.254034 31924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.066s	user 0.005s	sys 0.033s
I20260709 12:26:10.252934 31960 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:10.266913 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.288254 26196 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:10.248941 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:10.308827 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:10.309432 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60582us
I20260709 12:26:10.299507 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:10.257565 31970 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:26:10.277078 26791 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.311190 26791 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.313361 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.313776 26127 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:10.314164 26122 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:10.306228 31924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.005s	sys 0.033s
I20260709 12:26:10.315650 26201 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:10.317323 31237 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.319728 32003 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.324460 31960 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.269623 31957 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.333084 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.336954 31770 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.310163 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:10.343577 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.348209 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:10.267366 32002 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:26:10.355300 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.356078 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:10.331734 31957 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.362731 26123 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:10.358384 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:10.366721 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.369947 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:10.371289 32013 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.371819 32013 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:10.372293 32013 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.375825 26791 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:26:10.376443 26791 pending_rounds.cc:85] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 11
I20260709 12:26:10.376828 26791 pending_rounds.cc:107] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:26:10.377223 26791 raft_consensus.cc:2889] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.366415 31957 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:10.345784 32069 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:10.385517 26124 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:26:10.386059 26124 pending_rounds.cc:85] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 6
I20260709 12:26:10.386305 26124 pending_rounds.cc:107] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:26:10.386535 26124 raft_consensus.cc:2889] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.388970 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.390810 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.388801 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:10.363273 31968 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:10.398263 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.367169 31664 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:10.414606 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:10.397912 32015 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:10.364012 32003 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:10.429709 32003 consensus_queue.cc:1243] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:10.415879 31948 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:10.363771 32008 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:26:10.396754 31957 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:10.433652 26128 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:10.434931 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:10.473484 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.508927 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.539723 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.584128 31237 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.584789 31237 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:10.585781 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 325295us
I20260709 12:26:10.587512 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:10.593073 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 158264us
I20260709 12:26:10.594151 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:10.429791 32030 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.424536 31960 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:10.437413 32003 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:10.604141 31957 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:10.617344 31957 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:10.618590 31957 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.620848 26122 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:10.622030 32030 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.603477 31948 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.591964 31664 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:10.599543 31991 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.608431 32003 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:10.597424 31862 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.629060 31664 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:10.652163 26129 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:10.667654 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 319548us
W20260709 12:26:10.666428 31981 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.005s	sys 0.009s
W20260709 12:26:10.670408 31981 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.005s	sys 0.009s
I20260709 12:26:10.683290 26195 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 12:26:10.685464 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 341638us
I20260709 12:26:10.693511 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2360) took 1241 ms. Trace:
I20260709 12:26:10.695684 26198 rpcz_store.cc:276] 0709 12:26:09.452339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.227494 (+775155us) service_pool.cc:224] Handling call
0709 12:26:10.227804 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.228361 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:10.435200 (+206839us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.691921 (+256721us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.693143 (+  1222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.693194 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:10.693239 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.693317 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.693450 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":408483,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":1422,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":83,"thread_start_us":461675,"threads_started":2}
W20260709 12:26:10.706528 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.740128 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:10.688931 31968 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:10.714396 31986 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:10.748533 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2361) took 1276 ms. Trace:
I20260709 12:26:10.752584 26203 rpcz_store.cc:276] 0709 12:26:09.472014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.259314 (+787300us) service_pool.cc:224] Handling call
0709 12:26:10.259736 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.260019 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:10.438248 (+178229us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.747929 (+309681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.748045 (+   116us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:10.748115 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.748225 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.748416 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":379189,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":1679,"raft.queue_time_us":284,"thread_start_us":479814,"threads_started":2}
I20260709 12:26:10.758179 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:10.760547 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2027) took 1086 ms. Trace:
I20260709 12:26:10.743736 31973 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.762667 26203 rpcz_store.cc:276] 0709 12:26:09.674163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.759751 (+1085588us) service_pool.cc:224] Handling call
0709 12:26:10.760019 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.760203 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.760271 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.760314 (+    43us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:10.760344 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.760396 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.760496 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.777498 26203 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 18. Ops: []
I20260709 12:26:10.778224 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 913) took 1074 ms. Trace:
I20260709 12:26:10.770138 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3221) took 1297 ms. Trace:
I20260709 12:26:10.770275 32102 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.746767 32069 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:10.771875 32113 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:10.772324 31968 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.719698 32103 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:10.752660 32091 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:26:10.688031 32013 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.778898 26048 rpcz_store.cc:276] 0709 12:26:09.473537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.566255 (+ 92718us) service_pool.cc:224] Handling call
0709 12:26:09.566626 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.588998 (+ 22372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.589042 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.589104 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:09.589153 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.769862 (+1180709us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b540022bd54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:10.769908 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.770067 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50788096}
I20260709 12:26:10.779947 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2365) took 1153 ms. Trace:
I20260709 12:26:10.778720 26203 rpcz_store.cc:276] 0709 12:26:09.703994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.776801 (+1072807us) service_pool.cc:224] Handling call
0709 12:26:10.777316 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.777944 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.778169 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.684221 32003 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.771349 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1974) took 2192 ms. Trace:
W20260709 12:26:10.773980 32090 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:26:10.790302 26198 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:10.693419 31664 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:10.780994 31924 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:10.805536 32113 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.819471 26203 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:10.856472 31862 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.819964 26047 rpcz_store.cc:276] 0709 12:26:08.578805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.104377 (+525572us) service_pool.cc:224] Handling call
0709 12:26:09.807842 (+703465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.843890 (+ 36048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.771135 (+927245us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b540008e8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:10.771287 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616711424}
I20260709 12:26:10.856781 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 914) took 1151 ms. Trace:
I20260709 12:26:10.857398 26203 rpcz_store.cc:276] 0709 12:26:09.705712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.818856 (+1113144us) service_pool.cc:224] Handling call
0709 12:26:10.819151 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.856423 (+ 37272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.856707 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.862470 32103 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.863405 32103 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.866382 32103 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:10.867337 32103 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.781062 31973 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.871222 31973 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:10.836947 31631 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:10.886540 32045 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.001s
I20260709 12:26:10.829331 31664 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.830410 32003 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.818544 26196 rpcz_store.cc:276] 0709 12:26:09.626987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.350660 (+723673us) service_pool.cc:224] Handling call
0709 12:26:10.351321 (+   661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.354332 (+  3011us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:10.687470 (+333138us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.779536 (+ 92066us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.779588 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:10.779635 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.779774 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.779891 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":300327,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":92,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":5283,"thread_start_us":410813,"threads_started":2}
I20260709 12:26:10.901676 26196 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:10.834545 32013 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:10.835476 31977 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:10.836922 32105 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:10.847227 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.848870 32069 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.789049 32102 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:10.958125 32102 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:10.854354 31770 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.005s
W20260709 12:26:10.783891 32092 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 12:26:10.992822 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.103744 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.117794 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3235) took 1108 ms. Trace:
I20260709 12:26:11.118304 26049 rpcz_store.cc:276] 0709 12:26:10.009633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.061983 (+ 52350us) service_pool.cc:224] Handling call
0709 12:26:10.062594 (+   611us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.063125 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.063155 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.071160 (+  8005us) log.cc:844] Serialized 590 byte log entry
0709 12:26:10.295889 (+224729us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:10.296012 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.300076 (+  4064us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.117503 (+817427us) raft_consensus.cc:1685] finished
0709 12:26:11.117527 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.117726 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1305,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":224011,"threads_started":1,"wal-append.queue_time_us":342591}
I20260709 12:26:10.991446 32103 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.095153 31770 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:10.883803 32113 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:11.141328 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:11.153177 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:11.212401 26128 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:11.155696 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:11.159821 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:11.240793 26203 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:26:11.244341 32111 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:26:11.219046 32045 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.259070 32045 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:11.264070 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.009s	sys 0.008s
W20260709 12:26:11.264477 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.009s	sys 0.008s
W20260709 12:26:11.265399 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 671307us
I20260709 12:26:11.266290 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:11.267448 26045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.007s	sys 0.008s
W20260709 12:26:11.267895 26045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.007s	sys 0.009s
I20260709 12:26:11.261852 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2031) took 1449 ms. Trace:
I20260709 12:26:11.276181 26198 rpcz_store.cc:276] 0709 12:26:09.812415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.882737 (+1070322us) service_pool.cc:224] Handling call
0709 12:26:10.883072 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.883296 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:10.883575 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.261510 (+377935us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.261563 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.261607 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.261674 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.261789 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":526,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":103,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":86,"thread_start_us":377250,"threads_started":1}
I20260709 12:26:11.216698 31991 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:11.261770 32045 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:11.402174 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 135973us
W20260709 12:26:11.425866 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.416070 31973 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:11.469774 26754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.022s	sys 0.080s
W20260709 12:26:11.478643 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.504947 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.023s	sys 0.080s
I20260709 12:26:11.419654 32045 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.525954 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.531585 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.578539 32103 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:11.668857 26787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.030s	sys 0.038s
I20260709 12:26:11.528565 31770 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:11.563472 31973 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:11.670462 26787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.030s	sys 0.038s
I20260709 12:26:11.558939 32108 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:11.559129 32113 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:11.667977 31968 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.016s	sys 0.016s
I20260709 12:26:11.598042 32115 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:11.558416 32045 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:11.687703 32115 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:11.568080 31664 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.680s	user 0.013s	sys 0.001s
W20260709 12:26:11.670893 26756 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.000s	sys 0.012s
I20260709 12:26:11.684500 32113 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:11.691401 26756 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.000s	sys 0.012s
W20260709 12:26:11.688074 32116 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:26:11.680152 31991 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.684324 31948 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.051s	user 0.005s	sys 0.009s
W20260709 12:26:11.691968 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 872475us
I20260709 12:26:11.693863 31991 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.695005 31948 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:11.695851 32113 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.677960 31770 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:11.686269 31968 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.016s	sys 0.017s
W20260709 12:26:11.710182 32060 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 2.212s	user 0.012s	sys 0.048s
I20260709 12:26:11.707410 32118 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.682119 32108 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.717720 32113 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:26:11.707535 32103 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:26:11.719660 32118 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:11.714811 31957 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.017s	sys 0.017s
I20260709 12:26:11.721309 32115 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:11.732146 31957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.018s	sys 0.017s
W20260709 12:26:11.739194 32098 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.994s	user 0.006s	sys 0.016s
I20260709 12:26:11.717573 31973 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:11.743882 32101 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.994s	user 0.012s	sys 0.034s
I20260709 12:26:11.740388 32040 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:11.757709 32069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.017s	sys 0.068s
W20260709 12:26:11.778296 32069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.017s	sys 0.069s
I20260709 12:26:11.779842 32108 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:11.789039 32118 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.827581 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 1995) took 1582 ms. Trace:
I20260709 12:26:11.828661 26045 rpcz_store.cc:276] 0709 12:26:10.245291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.346331 (+101040us) service_pool.cc:224] Handling call
0709 12:26:10.346966 (+   635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.744363 (+397397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.744426 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.745526 (+  1100us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.268388 (+522862us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:11.268449 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.268553 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.826526 (+557973us) raft_consensus.cc:1685] finished
0709 12:26:11.826622 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.827204 (+   582us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13139,"raft.queue_time_us":35599,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":890630,"threads_started":2,"wal-append.queue_time_us":927835}
W20260709 12:26:11.832010 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.002s	sys 0.005s
W20260709 12:26:11.832741 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.002s	sys 0.005s
I20260709 12:26:11.774633 32040 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:11.857242 32052 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:11.858163 31770 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:11.858824 32052 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.858557 32040 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:11.860013 31875 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:11.736985 31968 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:11.861097 31875 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:11.862290 31968 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.863919 31875 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.867835 32040 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:11.873505 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 730) took 2173 ms. Trace:
I20260709 12:26:11.870671 31991 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.874315 31875 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:11.874320 26050 rpcz_store.cc:276] 0709 12:26:09.699951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.758505 (+ 58554us) service_pool.cc:224] Handling call
0709 12:26:09.758763 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.782382 (+ 23619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.872476 (+2090094us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b540035fcd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:11.873202 (+   726us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48291712}
I20260709 12:26:11.878719 32056 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:11.899420 32127 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.886862 32056 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:11.907246 32056 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.901857 32052 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.909209 31968 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.899928 32045 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.900609 32127 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.927948 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3303) took 1824 ms. Trace:
I20260709 12:26:11.928756 26120 rpcz_store.cc:276] 0709 12:26:10.103436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.103561 (+   125us) service_pool.cc:224] Handling call
0709 12:26:10.103846 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.235960 (+132114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.236018 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.236062 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:10.236094 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.927023 (+1690929us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400186ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:11.927138 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.927539 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303351552}
I20260709 12:26:11.942114 32127 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.943640 31237 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.949033 31968 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.950035 31968 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.958680 32056 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.958947 31986 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:11.959707 31986 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:11.959635 32056 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:11.962045 31895 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:11.962698 32056 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.964332 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1332) took 2801 ms. Trace:
I20260709 12:26:11.965795 31968 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:11.964864 26787 rpcz_store.cc:276] 0709 12:26:09.150743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.212573 (+ 61830us) service_pool.cc:224] Handling call
0709 12:26:10.773278 (+1560705us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:26:10.773484 (+   206us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.820351 (+ 46867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.820385 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.870719 (+ 50334us) log.cc:844] Serialized 52 byte log entry
0709 12:26:11.675756 (+805037us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:11.854250 (+178494us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.855282 (+  1032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.951176 (+ 95894us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b54004f79d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:11.951280 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.951376 (+    96us) raft_consensus.cc:1685] finished
0709 12:26:11.951403 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.951767 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111056,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":170423,"raft.run_cpu_time_us":611,"raft.run_wall_time_us":8427,"spinlock_wait_cycles":110526592,"thread_start_us":972446,"threads_started":2,"wal-append.queue_time_us":860936}
I20260709 12:26:11.967742 31818 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:11.968677 31818 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:11.970599 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.974464 32015 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:11.980090 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:11.999403 32015 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.980876 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.005177 26124 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:11.981806 32130 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:12.005746 32015 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.981770 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:12.006284 32130 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.007292 26127 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:12.007107 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:11.982575 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:11.979176 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.008975 26784 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:12.009441 26128 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
W20260709 12:26:12.010481 32097 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.235s	user 0.008s	sys 0.017s
I20260709 12:26:12.010792 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2033) took 2095 ms. Trace:
I20260709 12:26:12.011199 26203 rpcz_store.cc:276] 0709 12:26:09.915014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.858393 (+943379us) service_pool.cc:224] Handling call
0709 12:26:10.858747 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.241386 (+382639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.010148 (+768762us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400217ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:12.010494 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":840452864}
I20260709 12:26:12.011689 32130 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:11.984361 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:11.984520 32135 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.017369 32135 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:11.985309 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:12.019367 26789 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:11.986243 31916 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.005185 32106 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.012900 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3241) took 1908 ms. Trace:
I20260709 12:26:12.005373 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2017) took 2787 ms. Trace:
I20260709 12:26:11.972188 31818 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.983603 26120 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:26:11.984989 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:11.998456 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.063704 26120 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
I20260709 12:26:12.060943 26201 rpcz_store.cc:276] 0709 12:26:09.217936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.218088 (+   152us) service_pool.cc:224] Handling call
0709 12:26:09.218450 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.316089 (+1097639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.004822 (+1688733us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:12.005163 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2522788096}
W20260709 12:26:11.994086 32099 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.222s	user 0.015s	sys 0.010s
I20260709 12:26:11.983464 31956 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.992424 31961 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:12.067233 26120 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:26:12.076987 26127 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 12:26:11.992487 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 293945us
I20260709 12:26:12.094419 26127 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:12.096823 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.096905 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.100335 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.100683 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:12.070921 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.101693 26127 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:12.073344 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.072444 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.068552 31916 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:11.992201 32003 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.100s	user 0.013s	sys 0.010s
I20260709 12:26:12.100852 32135 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:11.986949 31381 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.075693 26120 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:12.111093 32002 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:12.111675 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.104841 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.114471 31916 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.115099 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.118022 26125 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:12.123026 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.112002 32002 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.125434 26202 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:12.121503 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.126580 32113 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.129638 32113 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.114583 31956 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:12.135258 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.126283 32003 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.060792 26053 rpcz_store.cc:276] 0709 12:26:10.104720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.219604 (+114884us) service_pool.cc:224] Handling call
0709 12:26:10.219934 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.247473 (+ 27539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.247507 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.248139 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.638231 (+390092us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:10.714650 (+ 76419us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.718733 (+  4083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.718875 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.991005 (+1272130us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:12.012281 (+ 21276us) raft_consensus.cc:1685] finished
0709 12:26:12.012383 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.012679 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3671,"prepare.queue_time_us":494347,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":4458,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":2155,"spinlock_wait_cycles":50690304,"thread_start_us":439680,"threads_started":2,"wal-append.queue_time_us":432567}
I20260709 12:26:12.137660 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.074815 32106 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:12.142556 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.144753 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.143278 31381 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.144727 32040 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:12.162420 31924 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:12.142442 32003 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.162591 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.167328 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.167979 26120 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:12.168432 26203 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:12.171717 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.157816 31961 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.169286 31924 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:12.177662 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:12.178172 32002 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:12.183032 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61109.6us
I20260709 12:26:12.184023 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:12.184052 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3239) took 2098 ms. Trace:
I20260709 12:26:12.185338 26052 rpcz_store.cc:276] 0709 12:26:10.085425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.214997 (+129572us) service_pool.cc:224] Handling call
0709 12:26:10.215343 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.329648 (+114305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.329679 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.330131 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.607169 (+277038us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:10.607240 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.696195 (+1088955us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:12.183436 (+487241us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:12.183509 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.183578 (+    69us) raft_consensus.cc:1685] finished
0709 12:26:12.183594 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.183848 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1433,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":301822208,"thread_start_us":276530,"threads_started":1,"wal-append.queue_time_us":357261}
I20260709 12:26:12.187942 31818 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:12.188547 31916 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:12.188669 31818 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:12.189205 31916 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:12.190416 26124 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:12.191279 31818 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.191479 31916 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.193284 26120 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:12.191855 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.178757 32040 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.197422 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2440) took 1937 ms. Trace:
I20260709 12:26:12.199329 32040 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.197984 26121 rpcz_store.cc:276] 0709 12:26:10.259537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.282972 (+ 23435us) service_pool.cc:224] Handling call
0709 12:26:10.301632 (+ 18660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.302145 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.302200 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.302816 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.640289 (+337473us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:10.640349 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.640451 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.992843 (+1352392us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:12.196915 (+204072us) raft_consensus.cc:1685] finished
0709 12:26:12.196939 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.197240 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9541,"raft.queue_time_us":124,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":14231,"spinlock_wait_cycles":5628160,"thread_start_us":323428,"threads_started":1,"wal-append.queue_time_us":455661}
I20260709 12:26:12.189106 31956 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.201439 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 731) took 2501 ms. Trace:
I20260709 12:26:12.202129 26046 rpcz_store.cc:276] 0709 12:26:09.700229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.877627 (+177398us) service_pool.cc:224] Handling call
0709 12:26:09.883193 (+  5566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.884119 (+   926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.200846 (+2316727us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b54000f8550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:12.201181 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12041088}
I20260709 12:26:12.204977 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:12.211481 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.210969 31862 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.216702 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 180397us
I20260709 12:26:12.217099 31862 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.217485 31986 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.219899 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2024) took 2813 ms. Trace:
I20260709 12:26:12.220429 26195 rpcz_store.cc:276] 0709 12:26:09.406071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.045005 (+638934us) service_pool.cc:224] Handling call
0709 12:26:10.045375 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.683975 (+638600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.219347 (+1535372us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:12.219720 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466190080}
I20260709 12:26:12.220269 26127 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:12.222777 31986 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:12.223701 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3314) took 1872 ms (client timeout 1572 ms). Trace:
I20260709 12:26:12.223604 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:12.224375 26122 rpcz_store.cc:269] 0709 12:26:10.351146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.366406 (+ 15260us) service_pool.cc:224] Handling call
0709 12:26:12.223283 (+1856877us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002ca1d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:12.223493 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581961088}
I20260709 12:26:12.212641 26121 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:12.235189 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.181733 31381 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.248890 32113 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.272545 32040 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.286616 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.274709 31986 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.287585 32129 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.288573 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:12.285023 32003 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.275980 32002 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.304167 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 120276us
I20260709 12:26:12.321352 32002 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.325173 32002 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.326138 32106 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.332491 32120 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:12.333434 32120 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.337450 31631 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.338467 31631 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.307220 32129 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.356148 32148 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:12.356993 32148 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.358295 32129 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.353930 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.359408 32148 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.355366 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.355618 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.361240 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:12.361822 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 905) took 3737 ms. Trace:
I20260709 12:26:12.361186 32133 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:12.364146 32133 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:12.346334 32002 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.327734 31986 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.367748 32002 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.327147 32040 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.370654 31986 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.346303 32106 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.362356 26194 rpcz_store.cc:276] 0709 12:26:08.624029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.782530 (+158501us) service_pool.cc:224] Handling call
0709 12:26:08.838602 (+ 56072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.892523 (+ 53921us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:08.910639 (+ 18116us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.067106 (+1156467us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b180047b960. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:26:12.008518 (+1941412us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b180047b960. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b77d9d3c7 00007f5b743af9a7 00007f5b743bf0bf 00007f5b743bf038
0709 12:26:12.181505 (+172987us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.286665 (+105160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.296824 (+ 10159us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:12.305152 (+  8328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.361422 (+ 56270us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:12.361471 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.361651 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11901,"prepare.queue_time_us":389824,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":2177,"raft.queue_time_us":3979,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":25016,"spinlock_wait_cycles":172339200,"thread_start_us":438021,"threads_started":3,"wal-append.queue_time_us":1616327,"wal-append.run_cpu_time_us":3102,"wal-append.run_wall_time_us":1484918}
I20260709 12:26:12.377410 31961 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.394652 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.394877 31924 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:12.397852 26126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3300) took 2301 ms (client timeout 1693 ms). Trace:
I20260709 12:26:12.398355 31924 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:12.398447 26126 rpcz_store.cc:269] 0709 12:26:10.096850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.097135 (+   285us) service_pool.cc:224] Handling call
0709 12:26:12.397555 (+2300420us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54002ca6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:12.397746 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4802726656}
I20260709 12:26:12.399402 31924 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.400375 31924 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.400956 31813 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.401894 31813 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.402731 32040 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.403208 31924 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.395660 26203 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:12.405014 31813 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:12.406129 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91595.2us
I20260709 12:26:12.408941 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:12.409145 32106 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.411859 32106 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.413054 31924 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:12.414309 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.414812 31924 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:12.416054 26790 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:12.418764 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.419456 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:12.420188 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.423192 26789 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:12.376679 31862 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.422387 26785 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:12.420673 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.433895 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:12.433436 26046 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:12.387008 31631 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.434772 32003 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.421241 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.420478 31986 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:12.439929 26783 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:12.439206 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.442360 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.445242 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.446594 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:12.447520 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66266.9us
I20260709 12:26:12.447541 26052 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:12.448837 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.449769 26127 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:26:12.450809 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 318002us
I20260709 12:26:12.446512 26784 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:12.450990 26122 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:12.448900 32147 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.452962 32147 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.454021 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.455542 32147 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.456228 31962 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:12.444860 32003 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.456941 31962 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.459298 32003 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.459496 31962 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.460866 32145 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:12.461771 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.459910 32003 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:12.464396 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.453589 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:12.464218 26126 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:12.466271 26789 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:12.468386 32145 consensus_queue.cc:1243] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:12.474251 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.445328 32040 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:12.462272 32147 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.460103 31962 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:12.469388 31986 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:12.473426 31875 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:12.499629 31960 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:26:12.501818 32040 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:12.504328 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.504918 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.505373 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.493157 32145 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.520135 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 111626us
I20260709 12:26:12.519590 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.496943 31875 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.519759 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.522312 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.471227 31813 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:26:12.526081 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.529197 26122 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:12.530417 26127 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:12.530751 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.529965 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.542523 26046 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:12.543853 26052 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:12.531359 26053 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:12.548024 26120 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:12.542471 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:12.553025 26124 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:12.546941 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:12.526746 26093 connection.cc:256] RPC call timeout handler was delayed by -1.2978682729999491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 12:26:12.556998 26093 connection.cc:256] RPC call timeout handler was delayed by -1.197268212000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 12:26:12.558624 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.555390 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.525961 31875 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:12.561671 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.496557 32003 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.512570 31960 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:26:12.532351 32040 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.508951 32147 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.568449 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.572047 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.572741 26046 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 12.
I20260709 12:26:12.586932 31869 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.587694 31869 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.588369 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2447) took 2217 ms. Trace:
I20260709 12:26:12.588904 26129 rpcz_store.cc:276] 0709 12:26:10.329689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.340499 (+ 10810us) service_pool.cc:224] Handling call
0709 12:26:10.361776 (+ 21277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.652718 (+290942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.496114 (+1843396us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54001c4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:12.547433 (+ 51319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667084032}
I20260709 12:26:12.589603 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:12.590366 31869 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.590456 26126 raft_consensus.cc:2376] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18. Re-sending same reply.
I20260709 12:26:12.591488 31751 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.591624 32120 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:12.604185 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.600596 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:12.588402 26199 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:12.556586 32148 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.590476 26129 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:12.561654 32150 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:12.620306 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.618374 26129 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:12.573390 31769 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.621626 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.622087 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.622728 26202 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:12.626003 26195 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:12.573482 31381 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:12.625686 26197 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
W20260709 12:26:12.629799 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.555755 31962 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.631345 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.632085 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.631811 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.632030 26053 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:12.631573 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.634814 26784 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:12.601874 31968 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:12.574150 32002 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.636849 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:12.641300 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.643779 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.644060 26046 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:12.592118 31751 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.648473 32061 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:26:12.650501 31751 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.643339 32150 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.652295 26789 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:12.653039 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:12.654997 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:12.655335 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:12.654937 26128 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:12.629619 31381 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.666370 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:12.594755 32130 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 12:26:12.670459 32149 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.190s	user 0.000s	sys 0.002s
I20260709 12:26:12.670816 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:12.671597 26199 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.648475 31769 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.672605 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.673955 26045 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:12.674225 31769 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.676714 31968 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.679386 31968 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:12.680675 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.682471 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.683506 26199 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:12.684748 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.695482 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.683909 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.690006 26050 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 12:26:12.709996 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 115411us
I20260709 12:26:12.597716 31793 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:26:12.715828 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.598861 32145 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.726262 26124 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:12.733956 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:12.716856 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 95473us
I20260709 12:26:12.736625 26195 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:12.736284 26094 connection.cc:256] RPC call timeout handler was delayed by -1.481892219999736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
I20260709 12:26:12.732944 26199 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:12.599707 31986 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:12.668516 31381 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:12.748600 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.742971 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.743870 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.746881 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:12.667644 32002 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:12.750538 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.749593 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.751369 26124 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:12.752071 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.748754 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:12.751143 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.688753 32148 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.777029 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.741237 32147 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.854394 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:12.861148 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.861721 32148 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.775519 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.862067 32133 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.863059 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.754527 26782 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:12.864404 31968 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.868320 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.769318 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.766889 26128 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:12.823663 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 168805us
I20260709 12:26:12.806524 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:12.814600 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 193689us
I20260709 12:26:12.735406 31631 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:12.767234 26784 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 12:26:12.778892 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3414922199999637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
W20260709 12:26:12.892364 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2298922199997833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 12:26:12.894372 26123 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:12.896395 31968 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.901263 26128 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:12.902596 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:12.905025 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.905788 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:12.906383 32040 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:12.896423 32133 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.901684 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:12.914913 26790 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:12.772020 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 175113us
I20260709 12:26:12.815371 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.864012 26129 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:12.886312 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.796084 26093 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:12.875770 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.890146 26197 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:12.755729 26199 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:12.772616 32106 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:12.936570 26094 connection.cc:256] RPC call timeout handler was delayed by -1.089492220000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 12:26:12.938055 31968 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:12.913470 26787 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:12.980010 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:12.981465 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43782.8us
I20260709 12:26:12.891742 32003 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:12.910157 32133 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:12.985100 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80868.3us
I20260709 12:26:12.993268 26195 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:12.939083 31895 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:13.007968 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.991982 26203 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:13.001022 26785 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:12.939235 31960 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:13.015257 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.017339 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:12.968045 32106 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:13.015746 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:12.901283 32145 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:13.026746 26129 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:13.032699 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 290730us
I20260709 12:26:13.012292 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:13.037984 26128 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:13.009820 32133 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.041442 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.042187 26122 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:12.988782 32061 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 12:26:13.045895 32161 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:26:13.046494 26200 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:13.047644 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.048339 26200 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:13.040266 32150 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:13.053810 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:13.045910 32168 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 12:26:13.059005 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 153376us
I20260709 12:26:13.061640 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:13.033605 32003 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:26:13.081787 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.086354 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.087612 26051 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:13.096807 32162 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:13.096910 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45498.2us
I20260709 12:26:13.097553 32162 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:13.099289 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.104980 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:13.105959 26045 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:13.106611 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48567.2us
I20260709 12:26:13.107438 32162 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:13.110212 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68788.2us
I20260709 12:26:13.110563 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.096390 31381 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.113270 26129 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:13.112170 31968 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:13.120157 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.120493 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.121264 26045 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:13.052637 32133 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.129881 26196 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:13.131768 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:13.135277 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 118341us
I20260709 12:26:13.133056 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.127739 32145 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.146018 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:13.131824 32148 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:13.170509 31895 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:13.172902 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46321.9us
I20260709 12:26:13.172689 32174 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:26:13.173640 32061 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:13.173846 32165 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:26:13.175546 31875 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.000s
W20260709 12:26:13.176132 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.176652 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 116895us
I20260709 12:26:13.177846 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:13.200161 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.178539 26046 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:26:13.180444 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47974.3us
I20260709 12:26:13.182654 31381 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:13.188030 32148 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:13.233079 26120 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:26:13.236987 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.240952 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 141824us
I20260709 12:26:13.123240 32106 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.224196 31961 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:13.230438 32148 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:13.267002 26202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:13.268004 26202 raft_consensus.cc:606] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.240710 31956 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.223762 31875 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.268429 26202 raft_consensus.cc:614] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:13.268718 31956 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.270021 26791 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:13.270401 31875 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:13.273515 26202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:13.274273 26202 raft_consensus.cc:606] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.274721 26202 raft_consensus.cc:614] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:13.279495 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.280392 26045 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:13.231962 32061 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:13.258791 31961 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:13.296882 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.292321 26195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:13.302955 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:13.307197 26195 raft_consensus.cc:606] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.307197 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.242669 32145 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.301604 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:13.259212 31916 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.285796 31961 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:13.310124 31631 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:13.310751 32150 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:13.300923 31769 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
W20260709 12:26:13.313385 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54398.5us
I20260709 12:26:13.325873 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.293381 32174 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:13.341979 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.349493 26046 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:13.259429 31986 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:26:13.342777 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:13.302980 31968 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:13.369831 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52588.5us
I20260709 12:26:13.322384 31751 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:13.319722 32003 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.373271 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:13.341261 31793 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:26:13.378417 31956 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:13.398281 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.381068 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:13.382202 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:13.382580 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.349785 32040 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.384718 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:13.385288 26049 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:13.386067 32106 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.386125 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.386617 32174 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.388823 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.391273 31962 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:13.318625 31631 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:13.383786 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.375033 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:13.393198 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64673.7us
I20260709 12:26:13.301354 32129 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:26:13.324697 32133 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:13.383612 26128 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:13.347512 31769 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:26:13.406270 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 77020.4us
I20260709 12:26:13.406226 26048 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:13.378566 31381 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Committing config change with OpId 15.12: config changed from index 3 to 12, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:13.410696 31631 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:13.409896 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:13.396858 32147 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:13.340600 32145 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:13.392611 31751 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.295714 32061 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:13.421489 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:13.432720 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.401021 32162 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:13.423233 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.436765 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:13.440698 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:13.444489 26048 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:13.433780 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.446228 26049 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:13.441651 26045 raft_consensus.cc:2376] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16. Re-sending same reply.
I20260709 12:26:13.466961 26052 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
W20260709 12:26:13.468849 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47448us
I20260709 12:26:13.411767 32181 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.434861 31751 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.432410 32147 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:13.490152 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 370056us
I20260709 12:26:13.490885 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.491974 26046 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:13.494930 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.494328 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.459894 32040 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:13.499171 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.428509 31962 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:26:13.501859 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:13.432890 31916 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:13.511646 32192 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:13.487525 32147 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:13.490223 32003 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:13.513935 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:13.522351 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:13.527947 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 159025us
W20260709 12:26:13.516088 26168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.511s (SENT) [suppressed 19 similar messages]
I20260709 12:26:13.531078 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:13.489521 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:13.549919 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 738) took 1041 ms. Trace:
I20260709 12:26:13.550429 26050 rpcz_store.cc:276] 0709 12:26:12.509340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.519505 (+ 10165us) service_pool.cc:224] Handling call
0709 12:26:13.549495 (+1029990us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400171954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:13.549850 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306926848}
I20260709 12:26:13.550447 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.522377 32003 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.563135 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:13.545605 32178 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 12:26:13.570863 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:13.570739 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45183.4us
I20260709 12:26:13.545301 31813 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.569119 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84132.4us
I20260709 12:26:13.580353 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.549881 31956 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:13.607982 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.489084 31968 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.608996 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:13.611341 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55043.5us
I20260709 12:26:13.561048 31895 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:13.610769 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 128952us
I20260709 12:26:13.631553 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.612260 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:13.510627 32040 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.638967 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.641152 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.643419 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.639969 26126 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:13.567793 32003 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:13.654657 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:13.653767 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:13.656961 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141334us
I20260709 12:26:13.657873 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.658095 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.658377 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.580746 31769 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:13.663525 26122 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:13.665563 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.611579 31956 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:13.672495 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 118066us
I20260709 12:26:13.677021 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.677474 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.682096 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:13.685712 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.689957 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.642894 32181 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:13.632843 31895 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:13.696826 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.701965 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.715094 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.716447 26126 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:13.713482 26197 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:13.664145 31769 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.684765 31956 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:13.734004 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 121514us
I20260709 12:26:13.732793 26789 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:13.745966 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52207.7us
I20260709 12:26:13.703344 32181 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.717691 31895 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:13.756021 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:13.727581 32176 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.354s	user 0.002s	sys 0.000s
W20260709 12:26:13.734524 32168 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:26:13.760798 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:13.748550 26786 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.010s
W20260709 12:26:13.791277 26786 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.000s	sys 0.011s
I20260709 12:26:13.789006 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.731977 31769 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:13.684172 32173 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.256s
W20260709 12:26:13.797765 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.804785 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.805686 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.788972 26048 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
W20260709 12:26:13.763651 32179 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:26:13.713979 32148 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:13.684453 31813 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:13.811800 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45491us
I20260709 12:26:13.814239 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:13.816368 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 139506us
I20260709 12:26:13.757927 32181 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:13.756680 32174 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:13.824765 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.825408 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:13.831396 26127 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:13.790356 32129 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.835867 26054 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:13.800336 31986 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.826287 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:13.828334 32185 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.332s	user 0.000s	sys 0.002s
I20260709 12:26:13.864925 26127 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:13.832417 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:13.849884 32199 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:26:13.851145 32184 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260709 12:26:13.874436 32183 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.338s	user 0.000s	sys 0.002s
I20260709 12:26:13.828879 32162 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:13.871384 26048 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:13.831908 32197 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:13.888464 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62232.6us
I20260709 12:26:13.893410 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:13.899711 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:13.903141 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.950093 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50470us
W20260709 12:26:13.944868 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 120363us
W20260709 12:26:13.954133 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.849884 32200 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:13.957767 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:13.957834 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 248971us
I20260709 12:26:13.959215 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:13.958608 26048 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:13.895454 31986 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:13.949093 32162 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:14.006494 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:14.009362 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 272927us
I20260709 12:26:13.962455 32197 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:14.029155 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74693.5us
I20260709 12:26:13.943791 32129 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.077095 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:13.991796 31916 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:13.990978 32200 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:14.279870 26789 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.001s
I20260709 12:26:14.281137 26789 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:26:14.272389 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 327208us
I20260709 12:26:14.286883 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.286265 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.291697 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.292860 26789 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
W20260709 12:26:14.254827 32189 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:26:14.301014 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.298799 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.311185 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.311861 26789 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:14.260340 32196 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.261516 32040 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.315302 26784 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:14.234526 32194 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:14.165503 31916 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:14.327669 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:14.318419 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 293692us
W20260709 12:26:14.331709 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 301652us
I20260709 12:26:14.333182 26789 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:14.334344 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:14.335845 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.219640 32213 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:14.333364 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:14.342144 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.309417 31813 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.089275 32162 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.224s
I20260709 12:26:14.344924 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.230336 32205 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.456s
I20260709 12:26:14.347435 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.349740 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.351639 26788 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:26:14.359834 32145 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.363859 26784 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:14.364866 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:14.366590 32208 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.369s	user 0.000s	sys 0.001s
I20260709 12:26:14.366487 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:14.366958 32129 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.365623 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:14.373692 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.330312 32196 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.391664 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:14.392428 26199 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:14.366542 32204 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.395001 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:14.398499 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 428924us
I20260709 12:26:14.400332 26199 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:26:14.425957 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92659.9us
W20260709 12:26:14.426918 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.428419 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:14.428030 26123 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:14.427125 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.437168 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.431713 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:14.426437 31986 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:14.446033 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.424934 32195 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:14.456219 32218 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:14.456949 32218 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.459681 32218 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.460355 32224 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:14.460851 32224 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:14.461275 32224 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.466842 31793 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.467450 31793 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.469581 31793 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.471638 32150 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:14.472177 32150 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:14.477053 32227 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:26:14.477588 32196 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.477880 26126 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:14.479218 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3350) took 1508 ms. Trace:
I20260709 12:26:14.479605 26120 rpcz_store.cc:276] 0709 12:26:12.970986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.039075 (+ 68089us) service_pool.cc:224] Handling call
0709 12:26:13.232558 (+193483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.233667 (+  1109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.479010 (+1245343us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54003d1b50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:14.479141 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433712384}
W20260709 12:26:14.480772 31769 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.650s	user 0.007s	sys 0.004s
I20260709 12:26:14.481315 32195 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:14.481767 32003 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.808s	user 0.006s	sys 0.005s
I20260709 12:26:14.485868 26203 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:14.451354 26127 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
W20260709 12:26:14.456151 32221 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:26:14.501596 32232 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:14.504347 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:14.504788 31986 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:14.505461 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:14.506191 32239 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:14.507197 31793 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:14.509127 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:14.509853 26045 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:14.510921 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:14.456080 32225 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 12:26:14.501603 32219 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:26:14.453771 32181 consensus_meta.cc:287] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.612s	user 0.001s	sys 0.011s
W20260709 12:26:14.453245 31751 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.959s	user 0.006s	sys 0.006s
I20260709 12:26:14.512094 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:14.519266 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3260) took 1137 ms. Trace:
I20260709 12:26:14.526887 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:14.499830 26127 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:26:14.533475 26050 raft_consensus.cc:1217] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 15.11->[15.12-15.12]   Dedup: 15.12->[]
W20260709 12:26:14.541814 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.515057 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:14.523396 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3274) took 1080 ms. Trace:
W20260709 12:26:14.544903 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 116401us
W20260709 12:26:14.516638 32226 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:26:14.598592 26127 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:14.549108 26122 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:14.549753 26045 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:14.601107 26050 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:26:14.530957 32147 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.975s	user 0.010s	sys 0.000s
W20260709 12:26:14.527693 31381 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.819s	user 0.010s	sys 0.000s
I20260709 12:26:14.513113 31793 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:14.525192 32148 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.605906 32150 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.600883 31751 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.607128 32150 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:14.609134 31751 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.606727 31793 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.516355 31769 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.616612 26201 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:14.616850 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 185214us
I20260709 12:26:14.618395 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:14.618878 32106 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.211s	user 0.008s	sys 0.006s
I20260709 12:26:14.619670 32205 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.620199 32205 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:14.620662 32205 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.620715 32191 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:14.532364 31895 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.001s
I20260709 12:26:14.623206 32235 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:26:14.624518 32200 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:14.625959 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 110958us
I20260709 12:26:14.602931 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:14.601630 26046 rpcz_store.cc:276] 0709 12:26:13.442987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.504018 (+ 61031us) service_pool.cc:224] Handling call
0709 12:26:13.523829 (+ 19811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.715709 (+191880us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:13.715762 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.729726 (+ 13964us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:13.752879 (+ 23153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.522931 (+770052us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:14.523006 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.523275 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378595200}
W20260709 12:26:14.513859 31631 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.045s	user 0.013s	sys 0.001s
I20260709 12:26:14.600360 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:14.513258 32198 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.619603 26051 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:14.603345 26052 rpcz_store.cc:276] 0709 12:26:13.381900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.383462 (+  1562us) service_pool.cc:224] Handling call
0709 12:26:14.519022 (+1135560us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b5400030054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:14.519172 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133770112}
I20260709 12:26:14.634627 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:14.606666 31956 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.817s	user 0.006s	sys 0.006s
I20260709 12:26:14.639372 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:14.517176 32195 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.624004 31895 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.604063 32196 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:14.616065 31769 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.533854 32003 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.636868 32235 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:14.637737 32198 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.639351 32192 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:14.676988 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:14.677882 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44633.4us
W20260709 12:26:14.676301 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76051.4us
I20260709 12:26:14.598269 32181 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.679906 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:14.640008 32147 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.680294 31924 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:14.680451 32198 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.680455 32192 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.648665 31956 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.680971 32191 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.716748 32147 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:14.717670 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.720537 32244 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:14.724700 32240 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.725247 32240 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:14.725657 32240 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.709230 32192 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.727397 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.705130 32236 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.178s
I20260709 12:26:14.728770 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.694465 31924 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:14.729344 26120 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:14.729651 31924 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:14.730233 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.677253 32196 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.738046 26126 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:14.734001 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:14.737426 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:14.640367 31631 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.701882 31895 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.739499 26202 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:14.742882 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.743393 26120 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.739940 32235 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:26:14.742271 32244 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:26:14.750219 32205 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.752351 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3351) took 1408 ms. Trace:
I20260709 12:26:14.751835 32224 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.752748 26124 rpcz_store.cc:276] 0709 12:26:13.344174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.368486 (+ 24312us) service_pool.cc:224] Handling call
0709 12:26:13.368854 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.450469 (+1081615us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b18003c18a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b7439dbfb 00007f5b743a864b
0709 12:26:14.453545 (+  3076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.453594 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.454227 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.542862 (+ 88635us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:14.542904 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.542970 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.752020 (+209050us) raft_consensus.cc:1685] finished
0709 12:26:14.752041 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.752290 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122880,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":15802,"spinlock_wait_cycles":12931968,"thread_start_us":193440,"threads_started":2,"wal-append.queue_time_us":199916}
I20260709 12:26:14.710999 32191 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.755422 32218 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:14.756647 32218 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:14.756994 32218 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:14.757897 32218 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:14.759665 32241 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:14.737951 31961 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.365s	user 0.012s	sys 0.003s
I20260709 12:26:14.760505 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:14.738862 32147 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.770701 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:14.769213 26047 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:14.769340 32150 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.772605 32150 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.772965 32150 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.773435 32003 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:14.775097 32160 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.884s	user 0.008s	sys 0.010s
I20260709 12:26:14.778927 32040 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:14.779403 32040 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:14.781450 32181 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:14.782686 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89534.4us
W20260709 12:26:14.783394 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 930) took 1284 ms (client timeout 1528 ms). Trace:
W20260709 12:26:14.784101 26202 rpcz_store.cc:269] 0709 12:26:13.498720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.525025 (+ 26305us) service_pool.cc:224] Handling call
0709 12:26:14.783063 (+1258038us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002b3754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:14.783253 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2726971136}
I20260709 12:26:14.786123 26197 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:14.769878 31895 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:14.787942 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2394) took 1127 ms. Trace:
I20260709 12:26:14.793443 26197 rpcz_store.cc:276] 0709 12:26:13.660082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.478770 (+818688us) service_pool.cc:224] Handling call
0709 12:26:14.785849 (+307079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.786638 (+   789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.787681 (+  1043us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002b3750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:14.787844 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705423232}
I20260709 12:26:14.760556 32200 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.760887 32191 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:14.792269 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2390) took 1318 ms (client timeout 1576 ms). Trace:
I20260709 12:26:14.792553 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 748) took 1045 ms. Trace:
W20260709 12:26:14.813668 26201 rpcz_store.cc:269] 0709 12:26:13.474057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.561470 (+ 87413us) service_pool.cc:224] Handling call
0709 12:26:14.792004 (+1230534us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:14.792166 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2265997568}
I20260709 12:26:14.789009 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:14.813733 26045 rpcz_store.cc:276] 0709 12:26:13.746655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.600807 (+854152us) service_pool.cc:224] Handling call
0709 12:26:14.604793 (+  3986us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.632985 (+ 28192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.633036 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.647637 (+ 14601us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.754140 (+106503us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:14.769038 (+ 14898us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.770044 (+  1006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.770132 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.792312 (+ 22180us) raft_consensus.cc:1685] finished
0709 12:26:14.792334 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.792472 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14746,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":2034,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":2134,"thread_start_us":66556,"threads_started":1,"wal-append.queue_time_us":100109}
I20260709 12:26:14.795213 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.816121 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:14.818313 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:14.803284 26120 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:14.752710 32205 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:14.833400 31875 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.834306 31875 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.839445 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:14.768496 31961 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:14.840934 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2050) took 2161 ms (client timeout 2061 ms). Trace:
W20260709 12:26:14.841527 26196 rpcz_store.cc:269] 0709 12:26:12.679151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.754028 (+ 74877us) service_pool.cc:224] Handling call
0709 12:26:14.840583 (+2086555us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400162e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:14.840778 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457171712}
I20260709 12:26:14.842276 31961 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:14.796110 32147 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.820024 32200 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.845034 32147 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.846552 32200 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.817775 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:14.849115 31956 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.795924 32040 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.850680 32198 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.853068 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3270) took 1462 ms. Trace:
I20260709 12:26:14.853633 26054 rpcz_store.cc:276] 0709 12:26:13.391124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.435107 (+ 43983us) service_pool.cc:224] Handling call
0709 12:26:13.477153 (+ 42046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.838542 (+361389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.852703 (+1014161us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:14.852980 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807632640}
W20260709 12:26:14.857623 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.828996 32235 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:14.833691 32188 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.860991 32188 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.861624 31875 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.862855 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:14.863337 32188 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.863808 26787 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:14.863981 32194 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:14.864732 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46522.6us
I20260709 12:26:14.865139 31793 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:14.866500 31631 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.867250 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.867475 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.867843 32224 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.868410 31916 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:14.868868 31916 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:14.869240 31916 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.871300 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.871978 26125 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:14.859678 26122 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:14.770872 32244 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.869172 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:14.859032 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 931) took 1350 ms. Trace:
I20260709 12:26:14.831060 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1362) took 1406 ms. Trace:
I20260709 12:26:14.904855 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.875123 26124 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:14.904549 26203 rpcz_store.cc:276] 0709 12:26:13.508891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.401512 (+892621us) service_pool.cc:224] Handling call
0709 12:26:14.437012 (+ 35500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.498877 (+ 61865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.856224 (+357347us) spinlock_profiling.cc:243] Waited 9.62 ms on lock 0x7b5400398854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:14.858967 (+  2743us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22117120}
W20260709 12:26:14.910626 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.30332e+06us
I20260709 12:26:14.915999 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1323) took 2012 ms. Trace:
W20260709 12:26:14.920403 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53639.9us
I20260709 12:26:14.918226 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1358) took 1494 ms. Trace:
I20260709 12:26:14.923743 26123 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:14.918237 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1357) took 1494 ms. Trace:
I20260709 12:26:14.907218 31956 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.908120 32244 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:14.921689 26783 rpcz_store.cc:276] 0709 12:26:13.423674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.427809 (+  4135us) service_pool.cc:224] Handling call
0709 12:26:13.447478 (+ 19669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.461887 (+ 14409us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:13.631111 (+169224us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.917910 (+1286799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.917963 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:14.918007 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.918069 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.918171 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":980117,"prepare.queue_time_us":201533,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4817,"thread_start_us":401645,"threads_started":2,"wal-append.queue_time_us":671200}
I20260709 12:26:14.925944 26785 rpcz_store.cc:276] 0709 12:26:13.423360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.425947 (+  2587us) service_pool.cc:224] Handling call
0709 12:26:13.436630 (+ 10683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.484769 (+ 48139us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:13.810352 (+325583us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.917963 (+1107611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.918005 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:14.918054 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.918106 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.918184 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":975296,"prepare.queue_time_us":257462,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":325414,"threads_started":2,"wal-append.queue_time_us":438566}
I20260709 12:26:14.935355 32196 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:14.936937 31924 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.937472 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.938099 26128 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.939596 31924 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:14.916450 26786 rpcz_store.cc:276] 0709 12:26:12.903352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.924407 (+ 21055us) service_pool.cc:224] Handling call
0709 12:26:12.938461 (+ 14054us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:13.057174 (+118713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.057213 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:13.114360 (+ 57147us) log.cc:844] Serialized 52 byte log entry
0709 12:26:13.191696 (+ 77336us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:13.791910 (+600214us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.915653 (+1123743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.915747 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.915787 (+    40us) raft_consensus.cc:1685] finished
0709 12:26:14.915798 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.915924 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1123472,"prepare.queue_time_us":60919,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":7404,"raft.queue_time_us":37570,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":9783,"thread_start_us":736523,"threads_started":3,"wal-append.queue_time_us":266069}
I20260709 12:26:14.919140 32235 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.929893 31956 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.933662 32244 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:14.974732 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 105676us
I20260709 12:26:14.957015 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:14.972015 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.980104 26128 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:14.981348 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:14.924685 32173 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:14.931233 31813 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:14.956156 32162 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:15.030345 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 98096.9us
I20260709 12:26:15.031010 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.037397 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:14.994438 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2395) took 1334 ms. Trace:
I20260709 12:26:15.040036 26200 rpcz_store.cc:276] 0709 12:26:13.660402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.481361 (+820959us) service_pool.cc:224] Handling call
0709 12:26:14.481749 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.508517 (+ 26768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.508545 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.509072 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.625322 (+116250us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:14.983131 (+357809us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.993958 (+ 10827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.994092 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.994133 (+    41us) raft_consensus.cc:1685] finished
0709 12:26:14.994144 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.994347 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36181,"prepare.queue_time_us":302943,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":76329,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":457808,"threads_started":2,"wal-append.queue_time_us":182051}
I20260709 12:26:15.033267 26128 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:14.975099 31956 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.945835 32212 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 12:26:15.034420 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.30649e+06us
I20260709 12:26:14.985008 32148 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.057765 32218 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:15.059444 32253 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:14.954113 32150 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:14.984115 32228 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:14.971159 32147 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:26:15.032032 32204 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:15.031642 31962 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:15.090837 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:14.978344 32235 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.104108 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1006) took 1384 ms. Trace:
I20260709 12:26:15.103295 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1361) took 1678 ms. Trace:
I20260709 12:26:15.112162 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1005) took 1393 ms. Trace:
I20260709 12:26:15.107303 26790 rpcz_store.cc:276] 0709 12:26:13.424682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.424806 (+   124us) service_pool.cc:224] Handling call
0709 12:26:13.438020 (+ 13214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.438332 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:13.485927 (+ 47595us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.477826 (+991899us) spinlock_profiling.cc:243] Waited 912 us on lock 0x7b18003ff220. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:26:15.102886 (+625060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:15.102953 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:15.103005 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.103077 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.103221 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":432333,"prepare.queue_time_us":24708,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":400102,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"spinlock_wait_cycles":2097536,"thread_start_us":232985,"threads_started":2,"wal-append.queue_time_us":1123944}
I20260709 12:26:15.112922 26789 rpcz_store.cc:276] 0709 12:26:13.719003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.384392 (+665389us) service_pool.cc:224] Handling call
0709 12:26:14.384864 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.394014 (+  9150us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:14.431268 (+ 37254us) log.cc:844] Serialized 14 byte log entry
0709 12:26:15.111732 (+680464us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:15.111783 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:15.111838 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.111926 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.112084 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":323941,"prepare.queue_time_us":36703,"prepare.run_cpu_time_us":3121,"prepare.run_wall_time_us":8121,"raft.queue_time_us":300009,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"spinlock_wait_cycles":354176,"thread_start_us":334736,"threads_started":2,"wal-append.queue_time_us":297863}
W20260709 12:26:15.119544 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 164640us
I20260709 12:26:15.053510 32162 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.137624 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.106037 26788 rpcz_store.cc:276] 0709 12:26:13.719387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.390554 (+671167us) service_pool.cc:224] Handling call
0709 12:26:14.390911 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.425760 (+ 34849us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:14.454857 (+ 29097us) log.cc:844] Serialized 14 byte log entry
0709 12:26:15.103758 (+648901us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:15.103811 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:15.103861 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.103935 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.104050 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":284517,"prepare.queue_time_us":46662,"prepare.run_cpu_time_us":232,"prepare.run_wall_time_us":231,"raft.queue_time_us":293078,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":287474,"threads_started":2,"wal-append.queue_time_us":257738}
I20260709 12:26:15.105114 32197 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:15.157948 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:15.159238 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:15.158027 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:15.117281 32198 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.119693 32211 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:15.164448 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47548.2us
I20260709 12:26:15.092926 32228 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.162925 26045 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:15.103639 31962 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.163691 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.162173 32196 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:15.174248 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.175760 32228 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.176182 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.178123 26052 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.162276 32197 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.181268 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:15.182904 32197 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.162062 32249 raft_consensus.cc:2764] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:15.190486 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:15.196168 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1363) took 1553 ms. Trace:
I20260709 12:26:15.196434 31962 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.166226 32240 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.196691 26791 rpcz_store.cc:276] 0709 12:26:13.643354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.644805 (+  1451us) service_pool.cc:224] Handling call
0709 12:26:13.673914 (+ 29109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.872017 (+1198103us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:14.872051 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.872569 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.958140 (+ 85571us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:14.958188 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.958276 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.195908 (+237632us) raft_consensus.cc:1685] finished
0709 12:26:15.195929 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.196100 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1240341,"raft.queue_time_us":1202240,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":13037,"thread_start_us":85076,"threads_started":1,"wal-append.queue_time_us":215973}
I20260709 12:26:15.199270 26052 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:15.199810 32240 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:15.200465 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.201020 32235 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:15.205736 32162 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.206213 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1012) took 1481 ms. Trace:
I20260709 12:26:15.189324 32211 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:15.207701 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.220048 32212 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.229462 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55635.8us
I20260709 12:26:15.181289 32150 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.236629 25948 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 9 to 14, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:15.202656 32192 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.224445 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:15.193270 32196 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:15.240219 32148 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.242638 32211 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.242432 31956 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.232668 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.244930 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:15.246831 31956 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:15.247422 32186 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.247911 32264 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:15.250106 32264 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:15.250530 32264 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.253543 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3269) took 1862 ms. Trace:
I20260709 12:26:15.248906 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:15.254442 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.240921 32266 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:15.255816 32266 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:15.255365 32186 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.188005 32191 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:15.255959 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3252) took 2139 ms (client timeout 1511 ms). Trace:
I20260709 12:26:15.256889 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:15.256099 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:15.258656 32238 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.796s	user 0.011s	sys 0.004s
I20260709 12:26:15.274267 26126 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:15.259737 32212 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.225854 26784 rpcz_store.cc:276] 0709 12:26:13.725212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.403144 (+677932us) service_pool.cc:224] Handling call
0709 12:26:14.433500 (+ 30356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.929054 (+495554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.929083 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.936061 (+  6978us) log.cc:844] Serialized 26 byte log entry
0709 12:26:15.084919 (+148858us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:15.105011 (+ 20092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.105107 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.205971 (+100864us) raft_consensus.cc:1685] finished
0709 12:26:15.205992 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.206144 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495131,"raft.queue_time_us":507194,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"thread_start_us":148324,"threads_started":1,"wal-append.queue_time_us":207328}
I20260709 12:26:15.256307 32266 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.279775 31631 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:15.280217 31916 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.241889 32196 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:15.282068 32196 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.272680 32186 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.284482 32186 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:15.287317 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0790107960001478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 12:26:15.288939 31956 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.289230 26200 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:15.289623 31956 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.257059 32252 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:26:15.291302 32249 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:15.291684 32249 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:15.293004 32249 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:14.905009 26782 rpcz_store.cc:276] 0709 12:26:13.425547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.431827 (+  6280us) service_pool.cc:224] Handling call
0709 12:26:13.460104 (+ 28277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.460896 (+   792us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:13.501737 (+ 40841us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.830437 (+1328700us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b180044da60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:26:14.830620 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.830692 (+    72us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:14.830744 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.830823 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.830968 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113516,"prepare.queue_time_us":160168,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1285804,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":27869568,"thread_start_us":71210,"threads_started":1,"wal-append.queue_time_us":192689,"wal-append.run_cpu_time_us":2389,"wal-append.run_wall_time_us":1195066}
I20260709 12:26:15.294530 32249 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.295816 26199 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:15.276021 32192 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:15.297317 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.254011 26053 rpcz_store.cc:276] 0709 12:26:13.390833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.413154 (+ 22321us) service_pool.cc:224] Handling call
0709 12:26:13.557167 (+144013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.578317 (+ 21150us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:14.023250 (+444933us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.299131 (+275881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.299192 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:14.299238 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.299319 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.253335 (+954016us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400171950. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:15.253473 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40773,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":35905,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":15898,"spinlock_wait_cycles":314112768,"thread_start_us":625140,"threads_started":2,"wal-append.queue_time_us":336946}
I20260709 12:26:15.300426 32192 raft_consensus.cc:2764] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:26:15.302942 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.307539 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51535.8us
I20260709 12:26:15.310107 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3361) took 1586 ms. Trace:
W20260709 12:26:15.310755 32163 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.858s	user 0.002s	sys 0.000s
I20260709 12:26:15.311378 31956 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:15.312635 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.312855 31916 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:15.314517 32192 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:15.315343 32192 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:15.315721 32192 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.318147 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2370) took 2789 ms. Trace:
W20260709 12:26:15.318416 32233 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.839s	user 0.000s	sys 0.005s
I20260709 12:26:15.319885 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3359) took 1756 ms. Trace:
I20260709 12:26:15.320345 26129 rpcz_store.cc:276] 0709 12:26:13.563210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.606790 (+ 43580us) service_pool.cc:224] Handling call
0709 12:26:13.607174 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.994017 (+386843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.994047 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.026735 (+ 32688us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.316039 (+289304us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:14.316086 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.316178 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.317614 (+1001436us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:15.319571 (+  1957us) raft_consensus.cc:1685] finished
0709 12:26:15.319590 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.319802 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14546,"raft.queue_time_us":72057,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2995,"spinlock_wait_cycles":51225856,"thread_start_us":425397,"threads_started":2,"wal-append.queue_time_us":268687}
I20260709 12:26:15.322379 31968 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.322988 31968 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.323441 31631 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:15.324609 31631 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.326050 31631 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.327037 26049 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:15.335180 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.337051 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.338089 26120 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:15.340101 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.342530 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.344645 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.328130 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:15.349797 26126 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:15.351123 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:15.272501 26051 rpcz_store.cc:269] 0709 12:26:13.116688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.123166 (+  6478us) service_pool.cc:224] Handling call
0709 12:26:15.255728 (+2132562us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400229a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:15.255851 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3418840320}
I20260709 12:26:15.353327 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:15.353631 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 96794.3us
I20260709 12:26:15.354364 26196 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:15.354264 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.310642 26127 rpcz_store.cc:276] 0709 12:26:13.724086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.450717 (+726631us) service_pool.cc:224] Handling call
0709 12:26:14.451132 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:14.956102 (+504970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.956132 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:14.975918 (+ 19786us) log.cc:844] Serialized 52 byte log entry
0709 12:26:15.189913 (+213995us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:15.189966 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.190994 (+  1028us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.309810 (+118816us) raft_consensus.cc:1685] finished
0709 12:26:15.309832 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.310007 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22612,"raft.queue_time_us":29049,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3539,"thread_start_us":439234,"threads_started":2,"wal-append.queue_time_us":272622}
I20260709 12:26:15.354871 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.355908 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.357165 26127 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:15.359735 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:15.360384 32106 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.360718 32231 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:15.360934 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.361150 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:15.361831 26122 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:15.360962 32106 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.361662 32231 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:15.362855 32269 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:15.362941 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.363438 32269 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.363961 32269 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.365706 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:15.318673 26194 rpcz_store.cc:276] 0709 12:26:12.528326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.535329 (+  7003us) service_pool.cc:224] Handling call
0709 12:26:12.569613 (+ 34284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:12.614334 (+ 44721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.614376 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:12.638537 (+ 24161us) log.cc:844] Serialized 52 byte log entry
0709 12:26:12.873094 (+234557us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:12.987503 (+114409us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.987850 (+   347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.987937 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.408949 (+1421012us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b1800016960. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:15.309951 (+901002us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:15.317902 (+  7951us) raft_consensus.cc:1685] finished
0709 12:26:15.317929 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.318085 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8219,"prepare.queue_time_us":172633,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":20803,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":25185,"spinlock_wait_cycles":75786752,"thread_start_us":235284,"threads_started":2,"wal-append.queue_time_us":346118}
I20260709 12:26:15.365087 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:15.372311 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.374441 26122 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:15.375020 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.375257 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:15.328850 31986 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:26:15.325330 31968 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.386440 26122 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:15.384622 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.386377 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:15.391721 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.392467 26196 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.393440 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:15.388554 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:15.398959 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:15.400434 26195 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:15.258780 31793 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:15.376369 32173 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.405722 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:15.406546 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92495.1us
I20260709 12:26:15.363842 32231 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.402982 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:15.399667 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3264) took 2014 ms (client timeout 2366 ms). Trace:
I20260709 12:26:15.380482 32150 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:15.406373 26791 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:15.387039 32162 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.383447 32113 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:15.390515 32252 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:15.414438 26782 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:15.402408 26788 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 3 to 12, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:15.419528 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.420142 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:15.429711 26199 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:15.410931 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:15.425346 26203 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:26:15.422024 26047 rpcz_store.cc:269] 0709 12:26:13.385515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.388427 (+  2912us) service_pool.cc:224] Handling call
0709 12:26:15.399482 (+2011055us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54003593d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:15.399605 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3128020736}
I20260709 12:26:15.380232 31962 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:26:15.455900 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.437691 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.408948 32228 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.441169 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:15.444821 32262 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.445439 32187 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:26:15.450340 32106 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.249392 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:15.473450 25948 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 11 to 14, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 14 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:15.475481 25948 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:15.405721 32173 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.371639 32264 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.438478 32263 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:15.434016 32150 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:15.386932 32133 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:26:15.381166 32244 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.412858 32276 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.477485 25948 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:15.521405 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.426862 32162 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:15.429054 32113 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:15.458173 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.515108 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:15.462657 32228 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:15.522991 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.486267 32173 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.436509 32147 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:26:15.530743 26196 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:15.532541 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.462566 32196 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:15.582787 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46117.1us
W20260709 12:26:15.586391 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49166.8us
I20260709 12:26:15.591660 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:15.531773 32162 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.592500 26196 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:15.535683 32228 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.532992 32113 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.598778 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.599457 26199 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:26:15.599687 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141626us
W20260709 12:26:15.605716 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 193441us
I20260709 12:26:15.611553 26125 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:15.585453 32217 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.619467 26784 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:15.591495 32234 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:15.629215 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.636695 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.636773 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.638239 26123 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:15.534204 32276 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.652555 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.659497 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.683373 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.684337 26791 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:15.684477 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:15.693979 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.697739 32040 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.700917 32264 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.705437 32244 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:15.649631 32234 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:15.707171 32234 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.709498 32196 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:15.715616 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2007) took 2094 ms. Trace:
I20260709 12:26:15.716055 26050 rpcz_store.cc:276] 0709 12:26:13.621521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.519812 (+898291us) service_pool.cc:224] Handling call
0709 12:26:14.533243 (+ 13431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.601599 (+ 68356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.715409 (+1113810us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b54003589d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:15.715551 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29107072}
I20260709 12:26:15.723198 26791 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:15.725263 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.728895 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.732355 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.645747 32150 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.719822 31631 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.746982 26196 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:15.748201 32286 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.757788 32286 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.761444 32286 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:15.769507 26168 leader_election.cc:336] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.572s (SENT)
I20260709 12:26:15.770172 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:15.771070 32244 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:15.780648 31962 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:15.781368 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.782335 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.783355 32285 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:15.783735 32285 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:15.784566 32003 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.785390 32003 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:15.786211 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.786875 26051 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:15.788522 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:15.789738 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.790403 26054 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.782297 31962 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:26:15.722309 32192 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.751890 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.753602 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:15.804672 26048 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:15.784173 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.749409 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:15.805994 26196 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
W20260709 12:26:15.798365 32249 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.007s
I20260709 12:26:15.810554 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:15.815399 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:15.817142 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:15.822929 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:15.823980 26048 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:15.729343 32186 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.745393 31631 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.827857 32264 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:15.827708 31962 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.850056 32264 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:15.853343 32264 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.856446 26782 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:15.858052 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3353) took 2513 ms. Trace:
I20260709 12:26:15.858618 26121 rpcz_store.cc:276] 0709 12:26:13.344908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.368329 (+ 23421us) service_pool.cc:224] Handling call
0709 12:26:13.384583 (+ 16254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.565934 (+181351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.565972 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.628971 (+ 62999us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.944279 (+315308us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:14.020051 (+ 75772us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.276635 (+256584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.276747 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.463979 (+187232us) spinlock_profiling.cc:243] Waited 7.08 ms on lock 0x7b180041ce60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:15.306945 (+842966us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b180041ce60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240
0709 12:26:15.308117 (+  1172us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b180041ce60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1
0709 12:26:15.855354 (+547237us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b180041ce60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:15.857759 (+  2405us) raft_consensus.cc:1685] finished
0709 12:26:15.857791 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.857970 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53460,"prepare.queue_time_us":70438,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":193692,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":53451,"spinlock_wait_cycles":76284288,"thread_start_us":387115,"threads_started":2,"wal-append.queue_time_us":412431}
I20260709 12:26:15.893913 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:15.894981 32262 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:15.895442 32262 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:15.896485 32262 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.897655 32262 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:15.899672 32282 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 12:26:15.868759 32276 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:15.870916 32186 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.872431 32192 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.876490 32217 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.877406 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 223401us
I20260709 12:26:15.879784 32212 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:15.889863 32286 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:15.916374 32286 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:15.917579 32286 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.919447 32286 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.919284 32173 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:15.921974 32280 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 12:26:15.891053 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47633.4us
I20260709 12:26:15.927433 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:15.929132 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:15.930267 32265 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:15.930742 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.930858 32265 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.891390 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:15.895984 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.933506 32265 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:15.949117 32281 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.291s	user 0.004s	sys 0.000s
I20260709 12:26:15.954300 26045 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:26:15.957412 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48063.2us
I20260709 12:26:15.958161 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.958395 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.867468 32269 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.914171 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:15.914042 32264 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.915159 26129 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:15.904138 32217 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:15.926573 32287 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:26:15.968943 26120 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:15.961882 26122 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:15.970793 32265 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:15.971297 32265 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:15.971678 32265 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.972617 32264 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:15.973284 32260 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:15.963487 32266 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.000s
I20260709 12:26:15.974941 32266 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:15.976150 26789 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:15.977910 31968 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.827492 32285 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:15.988547 32191 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:15.989363 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.990268 26051 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:15.991132 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:15.992499 31793 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:15.993572 31793 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.889997 31631 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:16.005712 32284 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.264s	user 0.000s	sys 0.002s
I20260709 12:26:16.023286 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:15.992291 26054 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:26:16.038401 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 107750us
I20260709 12:26:15.964339 32113 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:16.049206 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.091773 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.092726 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.093632 26049 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:26:16.086650 26053 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:16.097795 26121 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:16.097905 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.102743 25948 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 9 to 13, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 13 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:16.115353 25948 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 10, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 10 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:16.091904 32106 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:16.127158 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 352190us
I20260709 12:26:16.134923 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:16.144482 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45690.3us
I20260709 12:26:16.111107 32147 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:16.105327 31631 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.146112 26788 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:16.115036 32040 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.150619 25948 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 11 to 12, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:16.152258 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:16.175169 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:16.126266 32262 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.194267 25948 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:16.131279 32003 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.125552 32106 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:16.112015 32162 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.111853 32228 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.144292 32266 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.150558 32252 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:16.194808 31962 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:16.212474 26121 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:16.224911 31962 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:16.225627 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 276) took 46948 ms (client timeout 15000 ms). Trace:
I20260709 12:26:16.221625 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:16.190996 31631 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:16.208873 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63526.9us
I20260709 12:26:16.238744 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.227311 32269 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.228993 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:16.218912 26050 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.231387 32265 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:16.227751 25946 rpcz_store.cc:269] 0709 12:25:29.277047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.277433 (+   386us) service_pool.cc:224] Handling call
0709 12:26:16.225343 (+46947910us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:05.810800 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:05.860887 (+ 50087us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:05.860944 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:26:05.860967 (+    23us) tablet.cc:662] Decoding operations
0709 12:26:05.862457 (+  1490us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:05.862581 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:05.862613 (+    32us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:26:05.991237 (+128624us) tablet.cc:701] Row locks acquired
0709 12:26:06.016085 (+ 24848us) write_op.cc:260] PREPARE: finished
0709 12:26:06.066514 (+ 50429us) write_op.cc:270] Start()
0709 12:26:06.101275 (+ 34761us) write_op.cc:276] Timestamp: P: 1783599966047514 usec, L: 0
0709 12:26:06.110207 (+  8932us) op_driver.cc:348] REPLICATION: starting
0709 12:26:06.201261 (+ 91054us) log.cc:844] Serialized 6779 byte log entry
0709 12:26:07.043051 (+841790us) op_driver.cc:464] REPLICATION: finished
0709 12:26:07.395987 (+352936us) write_op.cc:301] APPLY: starting
0709 12:26:07.396138 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:07.865834 (+469696us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:07.865945 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:08.253983 (+388038us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:08.254381 (+   398us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 12:26:08.254463 (+    82us) write_op.cc:312] APPLY: finished
0709 12:26:08.255010 (+   547us) log.cc:844] Serialized 156 byte log entry
0709 12:26:08.381660 (+126650us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:08.394321 (+ 12661us) write_op.cc:454] Released schema lock
0709 12:26:08.474076 (+ 79755us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:08.576948 (+102872us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:08.593277 (+ 16329us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1179687,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":232321,"apply.run_cpu_time_us":37990,"apply.run_wall_time_us":1163571,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75289,"num_ops":17,"prepare.queue_time_us":164880,"prepare.run_cpu_time_us":89979,"prepare.run_wall_time_us":673774,"raft.queue_time_us":398646,"raft.run_cpu_time_us":14264,"raft.run_wall_time_us":165102,"replication_time_us":922137,"thread_start_us":808696,"threads_started":5,"wal-append.queue_time_us":573445,"wal-append.run_cpu_time_us":33455,"wal-append.run_wall_time_us":2605873}]]}
I20260709 12:26:16.194998 32262 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:16.234678 31962 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.188984 32148 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:16.284709 32148 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:16.285471 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46849.9us
I20260709 12:26:16.287603 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:16.242821 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62906.2us
W20260709 12:26:16.243253 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91130.5us
I20260709 12:26:16.204931 25948 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:16.249366 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:16.298813 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 514672us
W20260709 12:26:16.202988 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50623.5us
I20260709 12:26:16.234766 32113 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:16.221385 32294 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:26:16.155684 32150 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 12:26:16.272553 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:16.281564 32147 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.191881 32194 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:26:16.223475 31968 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:16.194069 32274 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:16.221011 32228 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.218524 32162 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.266251 32191 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.299974 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.302042 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.342326 26789 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:16.199083 32040 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.304047 25948 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:16.308192 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.301184 31961 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.309140 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.353686 26194 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:26:16.310673 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.355020 26199 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:16.274046 32262 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.314699 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.287931 32148 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.316476 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:16.316489 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.318342 32266 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:16.361315 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.331300 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.360594 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.335377 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.338171 26202 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:16.364714 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:16.344449 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56976.9us
I20260709 12:26:16.366442 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.366634 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.295578 32217 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:16.319507 32224 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:26:16.353726 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.358779 26050 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:16.319571 32147 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.357203 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.265527 32276 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:16.293694 32264 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:16.332824 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.385026 26049 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
I20260709 12:26:16.346856 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.369721 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.371125 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.386173 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.377007 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.387250 26194 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:16.353039 32244 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:16.380894 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.389755 26197 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:16.391340 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.391775 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.392692 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.392154 25948 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:16.351881 32173 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.381944 26201 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:16.368160 32148 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:16.375950 32162 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:16.386238 26053 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.404294 26196 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:16.397680 32113 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:16.396248 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.405964 26197 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:16.367254 26202 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.396225 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.407773 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:16.397660 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.411204 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.387187 32264 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:16.400310 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.415017 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.325790 32269 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:26:16.375630 31968 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:16.364921 32266 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:16.395781 32228 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.398707 32244 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:16.360664 31961 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:16.415057 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.398068 32187 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.398339 32240 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:16.409627 32113 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:16.430559 25948 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 7 to 12, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:16.437790 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.400370 32150 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:16.421533 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.401809 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.414021 32200 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:16.414474 32264 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:16.413939 32234 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:16.420743 32212 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.417246 32239 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:16.421044 32266 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.414199 32162 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:16.427896 32235 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.449050 26051 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.446777 26053 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:16.449052 26201 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:16.448969 32234 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:16.452553 32228 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:16.450076 25948 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:16.464340 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.468192 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 156385us
W20260709 12:26:16.457595 32293 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:26:16.443210 32113 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.469288 26202 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:16.488780 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.491295 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.496520 26786 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:16.490309 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.457862 32265 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:16.503110 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.504598 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.505694 26202 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:16.493808 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.509708 26053 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:16.494601 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.493456 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.499310 26195 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:16.510612 26791 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:16.464464 32195 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:16.502202 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.517616 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:16.502288 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.511852 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:16.518728 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94935.8us
I20260709 12:26:16.466279 31631 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:16.520457 26198 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:16.484246 32133 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:16.521862 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.524019 26202 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:26:16.504251 32298 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:26:16.495648 31956 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.532868 26198 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:16.463871 32212 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:16.517832 32286 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.598s	user 0.016s	sys 0.000s
I20260709 12:26:16.493206 31968 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:16.538583 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.541481 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:16.544921 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.492936 32228 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.548785 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.548799 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.549449 26791 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:16.550429 26201 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:16.551309 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.551529 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.553936 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.554688 26790 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:16.555342 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:16.551944 26783 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:16.557950 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:16.567240 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79067.1us
I20260709 12:26:16.569234 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.568154 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.571460 26053 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:16.559031 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:16.580078 25948 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: config changed from index 3 to 10, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:16.583240 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:16.585186 26785 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:16.594010 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74379.1us
I20260709 12:26:16.603034 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.604382 25948 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:16.618235 25948 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 10 to 11, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 11 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:16.533493 32133 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:16.516456 32244 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.618462 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:16.630986 25948 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 11 to 12, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:16.633247 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:16.633899 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 76036us
W20260709 12:26:16.634644 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75693.9us
I20260709 12:26:16.635253 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:16.553570 31968 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:16.635636 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:16.636545 25948 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:16.639423 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.642511 32317 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:26:16.645123 31986 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:16.578235 32173 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:26:16.616925 31961 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:16.655632 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:16.662485 26789 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:16.663304 26785 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:16.662925 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:16.676295 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105832us
I20260709 12:26:16.628294 32235 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:16.678762 25948 catalog_manager.cc:5697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 9 to 12, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 12 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
W20260709 12:26:16.684257 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65902.4us
I20260709 12:26:16.685485 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.687059 26786 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:16.693367 32309 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:16.693571 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3387) took 1092 ms. Trace:
I20260709 12:26:16.548381 32234 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.691746 26784 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:16.694023 26127 rpcz_store.cc:276] 0709 12:26:15.600739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.600879 (+   140us) service_pool.cc:224] Handling call
0709 12:26:15.601269 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.601752 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:15.633701 (+ 31949us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.116696 (+482995us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.116870 (+   174us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.142253 (+ 25383us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.142391 (+   138us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:16.142445 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.142508 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.693337 (+550829us) raft_consensus.cc:1685] finished
0709 12:26:16.693361 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.693498 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":282509,"prepare.queue_time_us":35430,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":4451,"raft.queue_time_us":62397,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":22338,"spinlock_wait_cycles":33838720,"thread_start_us":414840,"threads_started":1,"wal-append.queue_time_us":684712}
I20260709 12:26:16.656698 32265 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:16.704726 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:16.704048 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.720909 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:16.720340 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.726188 26049 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:16.727569 32326 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.727813 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.729194 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.729786 26045 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:16.730309 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.730829 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.731232 26048 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:16.729848 26046 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:16.732477 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.732678 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.733495 26046 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:16.733724 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:16.733848 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48450.2us
I20260709 12:26:16.734382 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.734303 32148 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:16.734727 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.614418 32162 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.735657 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:16.721328 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:16.739246 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:16.720328 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 85162.7us
I20260709 12:26:16.748447 26201 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.749186 26045 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.749431 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:16.743419 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:16.746934 26129 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:26:16.751343 26127 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:16.757933 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:16.680718 32235 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:16.770272 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:16.734330 32301 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 12:26:16.733852 32310 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:16.695181 32320 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:16.773617 32235 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:16.734745 32306 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:26:16.775759 26049 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:16.706682 32265 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:16.782053 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47418.7us
I20260709 12:26:16.764092 31986 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:16.702370 32317 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:26:16.702071 32252 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
W20260709 12:26:16.776206 32303 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 12:26:16.763839 32234 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.761924 32186 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:16.745854 32148 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:16.774618 32150 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.784122 32304 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 12:26:16.777818 31631 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.809171 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:16.792459 32265 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.790323 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:16.788165 31924 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:16.817941 32320 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:16.833084 26045 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:16.819434 32308 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:16.874074 32148 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:16.845505 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 96794.5us
I20260709 12:26:16.875078 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:16.846131 26047 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:16.843183 31962 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
I20260709 12:26:16.843482 32212 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:16.853391 31631 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.842105 31895 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:16.879901 26121 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:16.872932 32133 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:16.880975 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:26:16.852305 32305 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.295s	user 0.000s	sys 0.002s
I20260709 12:26:16.852121 32150 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:16.890904 32320 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:16.892329 31962 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:16.893251 32315 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.201s	user 0.003s	sys 0.000s
I20260709 12:26:16.846622 32228 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:16.895490 32312 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:26:16.844087 32331 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:26:16.910293 31631 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.926991 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.933825 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:16.934903 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:16.920576 31968 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:16.927873 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:16.923906 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85258.9us
I20260709 12:26:16.929014 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:16.921752 32304 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:16.909786 31895 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:16.882017 31924 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:16.955207 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 292379us
W20260709 12:26:16.943244 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68267.8us
I20260709 12:26:16.957242 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:16.944762 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:16.949923 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.005213 26201 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 11.
I20260709 12:26:16.958468 32324 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:16.958850 32316 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:26:16.958459 32338 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:16.966579 32313 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.384s	user 0.000s	sys 0.002s
I20260709 12:26:16.968993 26121 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:16.970662 32212 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:16.970778 32147 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 12:26:16.971971 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:16.959703 31895 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:16.979030 32285 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:17.026191 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54319.2us
I20260709 12:26:16.981858 32040 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:16.994446 26094 connection.cc:256] RPC call timeout handler was delayed by -2.6320653930001754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:26:17.029861 26094 connection.cc:256] RPC call timeout handler was delayed by -2.0860065010001563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 12:26:17.030527 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:16.997211 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.006322 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:17.032895 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.020776 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.035030 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.022193 26195 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:17.027175 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.038177 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:17.038333 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.038177 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.047065 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.049961 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.042166 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.053669 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:17.048269 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.054390 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:17.060827 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.023144 32338 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.065203 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.024610 32192 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:17.071270 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:17.108902 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.049038 32249 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.043474 31968 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.058702 32162 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.046185 32133 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.111729 26051 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:17.112339 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51660.3us
I20260709 12:26:17.050275 32040 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:17.043104 31961 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:17.054894 32218 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.112834 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.117277 26197 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:17.117986 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62305.9us
I20260709 12:26:17.057828 32224 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.068533 31895 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.070919 32269 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:17.090595 32262 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.001s
I20260709 12:26:17.123615 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.049222 31986 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.029373 32147 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.081845 32192 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:17.131639 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 66502.2us
I20260709 12:26:17.132932 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.039409 32341 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
W20260709 12:26:17.121587 32113 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.006s
I20260709 12:26:17.150753 32113 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:17.134711 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71444.6us
I20260709 12:26:17.072290 32307 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.056082 32186 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:26:17.116645 32327 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:17.117834 32228 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.145362 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.145901 31986 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.146448 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.118678 32211 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:26:17.137476 32133 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.146833 32269 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:17.147955 32147 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.143981 32040 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.150164 31962 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:17.152555 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.110965 32340 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 12:26:17.154407 32266 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.000s
I20260709 12:26:17.206925 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:17.223058 32263 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.241044 32266 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.182361 32326 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.234236 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:17.242556 32326 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:17.143095 32323 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.243388 32266 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:17.143863 31961 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.244622 26787 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:17.158155 32342 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.245086 31961 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.233677 26054 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 3 to 12, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:17.156112 32327 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:17.206470 26129 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:17.246863 32327 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.192574 32150 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:17.164956 32322 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.166316 26782 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.250036 31924 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:17.232251 26047 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.251549 26047 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:17.252393 26047 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:17.253633 26200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2408) took 1823 ms (client timeout 1543 ms). Trace:
I20260709 12:26:17.198649 32231 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.253722 26194 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:17.254208 26200 rpcz_store.cc:269] 0709 12:26:15.430633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.430754 (+   121us) service_pool.cc:224] Handling call
0709 12:26:17.253427 (+1822673us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54002b5054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:17.253566 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068122368}
W20260709 12:26:17.210639 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.255481 32307 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.155613 32338 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.238332 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:17.257057 26045 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:17.257236 32338 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.197191 32228 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.258076 32228 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:17.256275 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 103755us
I20260709 12:26:17.259927 32307 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:26:17.187549 32300 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.773s	user 0.000s	sys 0.004s
I20260709 12:26:17.209303 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.260979 32252 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:17.261220 26128 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.261370 32252 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.254390 32231 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.262310 32252 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.265247 32195 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.265728 26127 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:17.265820 32195 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.267094 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.267258 32106 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.268170 26047 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.268848 32106 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.269443 32252 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:17.184597 32194 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:17.271224 32329 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.377s	user 0.001s	sys 0.001s
I20260709 12:26:17.270813 32194 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:17.262938 32337 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.387s	user 0.005s	sys 0.000s
I20260709 12:26:17.273589 32194 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.243525 32326 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.208783 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.275465 26123 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.275105 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.276448 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:17.278061 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 131672us
I20260709 12:26:17.278538 26121 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:17.276947 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.278806 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.245780 32342 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:17.220875 32339 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 12:26:17.287516 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 142226us
I20260709 12:26:17.281845 32326 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.282809 32231 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:17.218691 32187 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.006s
I20260709 12:26:17.274403 32228 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:17.289397 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 110853us
I20260709 12:26:17.289585 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.299036 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.301877 26128 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:17.144313 32262 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.304400 26053 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.309854 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:17.305505 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:17.307744 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:17.307703 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:17.312626 26785 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:17.312690 26782 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:17.314574 32192 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.315833 32342 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:17.316741 32191 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.974s	user 0.012s	sys 0.001s
I20260709 12:26:17.317180 32191 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.318109 26196 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:17.318295 32195 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.307034 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:17.286339 32040 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:17.320811 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.322506 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:17.324256 26202 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:17.325908 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.289359 32200 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:17.327760 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:17.230219 32173 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.353850 32173 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.355098 26784 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:17.303855 32187 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.315866 32192 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.357053 26787 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:17.357596 32173 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.331534 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:17.336666 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.220165 32113 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:17.359655 31895 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:17.320070 32228 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.360173 31895 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.361398 31895 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.362149 32265 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.365298 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:17.339445 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.328042 32211 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:26:17.340299 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:17.339983 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.340975 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:17.342900 32195 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.347469 32231 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.348169 31924 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.304131 32147 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:26:17.353232 26123 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:17.336067 32285 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.356108 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:17.362576 32264 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.899s	user 0.007s	sys 0.004s
I20260709 12:26:17.334213 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:17.340324 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.376878 26129 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:17.377408 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.339206 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:17.390810 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.326081 32347 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.196468 31631 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:17.419916 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.398887 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:17.446183 26791 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.449085 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.449999 26202 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:17.378731 32196 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:17.440205 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101133us
I20260709 12:26:17.443367 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:17.452493 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75166.7us
I20260709 12:26:17.383081 32148 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:17.455878 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:17.459537 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.395485 32249 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.464066 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:17.439908 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 99672us
I20260709 12:26:17.392858 31924 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.487915 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:17.491703 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.507354 26046 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13.
I20260709 12:26:17.493295 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.512867 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:17.448559 32266 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:17.517468 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.558991 26124 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:17.392582 32327 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.558245 26791 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
W20260709 12:26:17.562945 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45591.2us
W20260709 12:26:17.564889 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100883us
I20260709 12:26:17.574383 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.494451 32194 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:17.399778 32239 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 12:26:17.577128 26201 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:26:17.457192 32228 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.486054 32162 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.581203 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.452606 32195 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:26:17.494784 32196 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.401059 32318 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:26:17.513641 32113 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:17.513981 32269 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:17.522156 32308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.009s	sys 0.025s
I20260709 12:26:17.449204 32264 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.590049 32200 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:17.560688 32286 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:17.558652 32341 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.569975 32323 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.583350 32148 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:17.577044 32194 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.518980 32324 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.582350 32266 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:17.619490 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.519831 32322 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 12:26:17.612147 32269 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:17.613330 32308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.021s	sys 0.032s
I20260709 12:26:17.622223 32113 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:17.639609 26788 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:17.630889 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 263943us
I20260709 12:26:17.650310 26121 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:17.583609 32249 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 12:26:17.662633 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 375365us
W20260709 12:26:17.663298 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.636638 32148 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.509292 32133 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.121s
I20260709 12:26:17.655224 32269 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.645002 32262 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.654425 32308 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.690639 26791 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:17.672451 32148 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.667438 32187 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.633258 32341 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:17.695271 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 182522us
W20260709 12:26:17.689407 32235 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.880s	user 0.012s	sys 0.002s
I20260709 12:26:17.677356 32249 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.692143 32195 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.692468 32211 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.712348 32249 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.712528 32211 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.708781 32266 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.697077 32262 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.733074 32342 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.727082 32249 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:17.752921 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 84790.4us
I20260709 12:26:17.733724 32330 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.712309 32195 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:17.761283 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99030.7us
W20260709 12:26:17.734305 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 246442us
I20260709 12:26:17.736773 31631 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.752249 32342 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.711756 32235 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.751888 32262 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:17.706791 32187 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:17.774591 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.678457 32196 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.772153 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:17.786065 26129 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:17.705121 32194 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.789242 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:17.792544 26195 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:17.766235 32330 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:17.793959 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:17.796574 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:17.778796 32357 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.767315 32249 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:26:17.779732 32327 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.805451 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:17.813184 26051 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:17.813836 32262 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.815207 32262 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:17.815770 32106 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 12:26:17.816514 32249 consensus_queue.cc:1243] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:17.819180 32357 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.821461 32357 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.822037 32357 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.822908 26050 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:17.805019 32195 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:17.825914 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.826601 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.828626 26789 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:17.829744 32359 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:17.830113 32195 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:17.796726 32252 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.004s
I20260709 12:26:17.758002 32113 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.802934 32364 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:26:17.836874 32240 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:17.837365 32240 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:17.824599 31962 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.838956 32326 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 12:26:17.838517 32240 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.794950 31895 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:26:17.797858 32264 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:17.838593 31962 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:17.861186 32224 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.725s	user 0.007s	sys 0.004s
I20260709 12:26:17.822434 32357 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.861757 32224 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.862458 32357 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.840955 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:17.842604 26121 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:26:17.870796 26121 pending_rounds.cc:85] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 10
I20260709 12:26:17.871060 26121 pending_rounds.cc:107] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:26:17.844995 26123 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:17.871304 26121 raft_consensus.cc:2889] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.872094 26198 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:17.845821 26051 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 13.
I20260709 12:26:17.872749 26123 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:17.772924 32308 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.838995 32235 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:17.857071 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:17.877839 26122 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:17.846369 26129 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
W20260709 12:26:17.865249 32234 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.020s	user 0.013s	sys 0.000s
I20260709 12:26:17.888278 26127 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:17.864188 32357 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.792302 32211 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.900269 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.901611 32195 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.901657 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.902397 32195 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.903383 26094 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
W20260709 12:26:17.897627 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48042.9us
I20260709 12:26:17.898157 32375 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:17.904595 32252 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.905051 32375 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:17.905424 32375 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.906315 32375 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.906783 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:17.907526 32204 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:17.907851 32331 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:17.909024 32204 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:17.909456 32204 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.909853 32204 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:17.913369 26786 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:17.912674 32375 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.868916 32187 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:17.918430 32269 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.885066 32361 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.920200 32361 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.802754 32342 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.905412 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.922528 32361 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.922856 32342 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:17.923214 32320 consensus_queue.cc:1175] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 16 status { last_received { term: 16 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 16 index: 9 } }
I20260709 12:26:17.920979 32347 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.925469 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:17.884828 32192 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:17.929744 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:17.935686 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.942929 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:17.943728 26791 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.943284 32330 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.944381 32364 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.945160 32347 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:26:17.945809 32231 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.003s
I20260709 12:26:17.946290 32231 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.946765 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:17.948562 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:17.950242 32192 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:17.950619 32252 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:17.951505 32252 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:17.951857 32252 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:17.952740 32252 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:17.963722 26787 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:17.953142 32192 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:17.886471 32318 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:17.950770 26129 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:17.957147 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 944) took 1519 ms. Trace:
I20260709 12:26:17.991386 26196 rpcz_store.cc:276] 0709 12:26:16.437372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.437527 (+   155us) service_pool.cc:224] Handling call
0709 12:26:17.956908 (+1519381us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400335254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:17.957048 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2022351744}
I20260709 12:26:17.957149 32234 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.946043 32347 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:17.967932 26791 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:17.989617 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 408459us
I20260709 12:26:18.004253 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.988788 32040 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.998443 32234 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.007433 32368 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.007956 32368 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.008569 31631 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:17.988024 32364 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:18.009613 32364 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.010201 32368 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:18.010128 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 213874us
I20260709 12:26:18.010855 32106 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:18.011407 32106 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:17.987715 32362 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:18.011639 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.012434 26196 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:18.012573 32106 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.015694 32269 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.011377 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.010541 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.018187 26125 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:18.021576 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.023417 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.023741 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:18.025228 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:17.999214 32307 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.999714 32228 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.027416 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.017140 32252 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:18.000157 32347 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.029726 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.016451 32234 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:18.031541 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:18.032536 32147 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.961126 31924 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.033043 32162 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:18.025800 32269 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:18.034684 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.054149 26125 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:18.037652 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.037973 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.043496 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:18.043576 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.057003 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:18.057232 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:18.057684 26791 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:18.020463 32286 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:18.034104 32367 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:18.030251 32231 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.046525 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:18.049230 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 86582.6us
I20260709 12:26:18.070546 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:18.042452 32285 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.997083 32318 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.060357 26054 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:18.060428 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.076200 26045 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:18.045495 26787 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:26:18.064360 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.065371 26052 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:18.068262 26790 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:18.053498 32162 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:18.072891 26782 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:18.048465 32368 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:18.010658 32364 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.072016 32195 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.052556 32113 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:18.078363 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.083609 26049 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:18.064435 32314 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.095198 32286 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.091804 32368 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:18.110334 32368 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:18.090253 32162 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.076040 31924 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.098130 26045 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:26:18.103780 32318 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.072157 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.109350 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.074976 32234 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:26:18.100250 32347 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.090745 32148 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.141021 26202 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:18.143074 32148 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.119467 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:18.117306 32204 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.146742 26199 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:18.147588 32204 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:18.149127 32376 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:18.150707 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.124419 26201 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:18.151376 26122 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:18.124500 32113 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:18.125217 26123 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:18.152846 32113 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.151499 26782 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:18.077636 32285 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.157083 26054 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:18.158277 32285 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.162818 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2427) took 1166 ms. Trace:
I20260709 12:26:18.059441 26196 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:18.129487 26125 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:18.149020 26789 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:18.165444 26789 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:18.168270 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.168897 26201 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
W20260709 12:26:18.151430 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.131281 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59217.7us
I20260709 12:26:18.170059 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.170778 26196 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.163477 26198 rpcz_store.cc:276] 0709 12:26:16.996465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.066071 (+ 69606us) service_pool.cc:224] Handling call
0709 12:26:17.066486 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.872527 (+806041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.162472 (+289945us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b5400300554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:18.162727 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852277504}
I20260709 12:26:18.172649 26122 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:18.166597 26789 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.170964 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:18.171783 32357 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:18.186404 26045 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:18.187493 31962 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.191890 32370 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:18.192530 32370 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.194576 26054 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:18.197705 32265 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.198061 26789 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:18.198724 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.193943 32370 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.201287 32370 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.203611 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:18.207050 26047 pending_rounds.cc:85] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 11
I20260709 12:26:18.187791 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.207724 26047 pending_rounds.cc:107] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:26:18.208006 26045 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:18.208389 26047 raft_consensus.cc:2889] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:18.208746 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.210419 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:18.211256 26782 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:18.214735 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48515.2us
I20260709 12:26:18.215267 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.218120 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:18.222455 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.237332 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:18.237519 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:18.210637 32162 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.241824 26196 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:18.243922 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.244043 26789 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:18.244351 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.248044 26122 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:18.251852 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.254298 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.254921 26051 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.255141 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.248584 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:18.220221 32192 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:18.229113 32200 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:18.246865 26194 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:18.229933 32331 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:18.259969 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.229633 32285 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:18.270957 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.266726 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:18.271759 26052 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:18.276167 26054 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:18.268121 26051 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:18.276104 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.253031 32362 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.245473 32321 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:18.279953 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.243024 32367 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:18.240806 32388 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:18.229768 32352 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:26:18.281567 26194 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:18.279604 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.261752 32231 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:18.282143 26045 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
W20260709 12:26:18.283560 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.265519 32200 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.244678 32252 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:18.240180 32269 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:26:18.282585 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.287801 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:18.319465 26199 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:18.277920 32162 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.313215 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:18.245172 32320 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:26:18.252847 32106 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
W20260709 12:26:18.353204 32381 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:26:18.355252 32106 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:18.355705 32106 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:18.356354 32148 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:18.286549 32321 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.358248 32321 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.358610 32368 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.286760 32388 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.363538 32388 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.365401 32147 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:18.366971 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.367789 32269 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.282836 32386 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:18.368541 32269 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.318774 32231 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:18.371083 32231 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.372370 32269 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.313313 32200 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.368552 32386 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.382377 32386 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:18.388499 26094 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.24.71.190:42395, user_credentials={real_user=slave}} blocked reactor thread for 51304.5us
I20260709 12:26:18.391095 32314 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:18.391664 32314 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:18.392522 32314 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.399420 32318 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:18.401857 32368 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.403702 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.404557 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1018) took 1084 ms. Trace:
I20260709 12:26:18.404922 26128 rpcz_store.cc:276] 0709 12:26:17.319782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.320834 (+  1052us) service_pool.cc:224] Handling call
0709 12:26:18.403528 (+1082694us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:26:18.404479 (+   951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3374464}
I20260709 12:26:18.403723 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1030) took 1098 ms. Trace:
W20260709 12:26:18.408095 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.409049 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1013) took 1142 ms. Trace:
I20260709 12:26:18.409490 26127 rpcz_store.cc:276] 0709 12:26:17.266632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.266821 (+   189us) service_pool.cc:224] Handling call
0709 12:26:18.408858 (+1142037us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:18.408984 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323455360}
I20260709 12:26:18.406713 26784 rpcz_store.cc:276] 0709 12:26:17.305835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.306241 (+   406us) service_pool.cc:224] Handling call
0709 12:26:18.403508 (+1097267us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b54000587d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:18.403636 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38863744}
I20260709 12:26:18.317806 26202 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:18.271581 32192 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.445783 32192 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.235296 31962 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:26:18.313717 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:18.320922 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.271539 32342 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:26:18.450783 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3310) took 1169 ms. Trace:
I20260709 12:26:18.324261 32367 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.280931 32331 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:18.336599 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:18.337827 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:18.418912 26050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2019) took 2004 ms (client timeout 1512 ms). Trace:
W20260709 12:26:18.455008 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141311us
W20260709 12:26:18.455643 26050 rpcz_store.cc:269] 0709 12:26:16.414701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.414827 (+   126us) service_pool.cc:224] Handling call
0709 12:26:18.418679 (+2003852us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:18.418834 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3205223296}
I20260709 12:26:18.419826 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2429) took 1422 ms. Trace:
W20260709 12:26:18.420769 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.07611e+06us
I20260709 12:26:18.458104 26195 rpcz_store.cc:276] 0709 12:26:16.997030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.068187 (+ 71157us) service_pool.cc:224] Handling call
0709 12:26:17.068530 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.793064 (+724534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.419614 (+626550us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54002175d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:18.419772 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1664571648}
I20260709 12:26:18.458529 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.428061 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3313) took 1141 ms. Trace:
I20260709 12:26:18.459933 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.436249 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.437186 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.438295 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:18.461432 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.460024 26046 rpcz_store.cc:276] 0709 12:26:17.286441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.286589 (+   148us) service_pool.cc:224] Handling call
0709 12:26:18.427612 (+1141023us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400227c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:18.427929 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444329856}
I20260709 12:26:18.439285 32314 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.439555 32285 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:18.441908 32341 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.288683 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:18.451238 26048 rpcz_store.cc:276] 0709 12:26:17.284955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.285074 (+   119us) service_pool.cc:224] Handling call
0709 12:26:17.285408 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.350118 (+ 64710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.350154 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.350616 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.589286 (+238670us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:17.589331 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.450315 (+860984us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540008e3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:18.450368 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.450417 (+    49us) raft_consensus.cc:1685] finished
0709 12:26:18.450435 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.450691 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1147,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":147967872,"thread_start_us":238259,"threads_started":1,"wal-append.queue_time_us":354107}
W20260709 12:26:18.452745 31968 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:26:18.450799 32236 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:18.475594 32341 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.475819 32236 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:18.476822 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 188219us
I20260709 12:26:18.468053 32285 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:18.479993 32341 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:18.480580 32341 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.480270 32236 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.481007 32341 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.480734 32285 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.481878 32236 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:18.482028 32285 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:18.464105 32265 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.464094 32266 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:18.470568 32342 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:18.484683 32342 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:18.489756 26048 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:18.496980 26784 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:18.498667 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.500154 26787 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:18.500540 32266 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:18.500969 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:18.501032 32266 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.506549 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.509181 26782 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:18.499884 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:18.533820 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.480774 32331 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.452062 32200 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.540850 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.541476 26168 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13.
I20260709 12:26:18.541115 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.541879 26168 leader_election.cc:403] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:18.508649 32265 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.544710 32224 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.546339 32320 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13.
I20260709 12:26:18.506543 32266 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.549511 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1376) took 2285 ms. Trace:
I20260709 12:26:18.485611 32342 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.577862 26788 rpcz_store.cc:276] 0709 12:26:16.264512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.316406 (+ 51894us) service_pool.cc:224] Handling call
0709 12:26:16.316703 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.657749 (+1341046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.549266 (+891517us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:18.549446 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3041897728}
I20260709 12:26:18.583298 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.584118 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.565788 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.585323 26201 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:18.583437 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.588167 26198 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:18.591063 26199 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:18.594022 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.570040 32240 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.532660 32340 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:18.555498 32388 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.564795 32200 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.579828 32234 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:18.588521 32218 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:18.615294 26045 raft_consensus.cc:2368] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:18.560114 32386 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.601787 32397 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:18.589128 32265 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:18.590106 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:18.571336 32321 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:18.624078 25938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80243us
I20260709 12:26:18.595662 32369 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:26:18.589901 32224 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.622938 26196 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:18.567718 32106 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:18.639513 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 96492.9us
W20260709 12:26:18.613204 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 223269us
I20260709 12:26:18.619984 32218 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:18.670693 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.617192 32234 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.693583 32265 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.694646 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.695720 32224 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:18.699100 32393 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:18.700831 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.661123 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:18.725008 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64025.4us
I20260709 12:26:18.725788 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:18.728751 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:18.734856 32218 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.735638 32307 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:18.736204 32307 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:18.736336 32386 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.736634 32234 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.737429 32249 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:26:18.737972 32399 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:26:18.738912 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.739475 26053 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:18.739575 26125 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:18.740152 32194 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:18.740746 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.741348 26128 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.675037 32341 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.744112 26053 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:18.746112 32388 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.746471 32192 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:18.746866 32192 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.747141 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.715961 32397 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:18.754240 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.755059 26053 raft_consensus.cc:2368] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16.
I20260709 12:26:18.757083 26052 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:18.743145 31895 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:18.766230 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.729998 32361 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:18.765434 31895 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:18.768298 31895 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.772724 32397 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.773811 32341 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:18.775063 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:18.736666 32307 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:18.776787 26046 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:18.777546 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.778342 26125 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:18.777005 32367 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:18.780603 32367 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.784977 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.785822 26788 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:18.786837 26128 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:18.788823 32361 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:18.789309 32401 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:18.789721 32361 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:18.790230 32361 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:18.790547 32361 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.791407 32361 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.747666 32192 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.793766 32192 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.793429 32362 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:26:18.783458 32367 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.793385 26203 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:18.807058 26121 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:18.809748 32361 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.810982 32211 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:18.795187 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1019) took 1467 ms. Trace:
I20260709 12:26:18.817317 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.818632 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:18.816728 26124 rpcz_store.cc:276] 0709 12:26:17.327797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.327990 (+   193us) service_pool.cc:224] Handling call
0709 12:26:17.328308 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.579550 (+251242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.794845 (+1215295us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400185bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:18.795106 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529918720}
I20260709 12:26:18.823815 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.827708 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:18.832022 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.837649 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:18.835916 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 336080us
I20260709 12:26:18.841632 26129 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:18.839190 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:18.832897 32192 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:18.833223 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.844542 26757 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.839843 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:18.836382 32263 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.841554 32362 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:26:18.843753 32307 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.858865 32307 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:18.838814 32194 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:18.836906 32388 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.839149 32386 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:26:18.862001 32386 consensus_queue.cc:1243] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:18.845984 32234 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:26:18.845378 32407 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:18.847558 32218 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:18.834853 32367 raft_consensus.cc:2764] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:26:18.850924 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.836376 32341 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.876767 32341 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.870048 32218 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:18.877875 32218 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:18.879283 32341 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.870325 32113 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:18.870469 32194 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:26:18.880888 32113 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:18.881335 32307 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:18.881899 32307 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.832566 26128 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:18.841907 32322 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:18.855247 32263 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.856729 32362 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:18.867308 26094 leader_election.cc:400] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16.
I20260709 12:26:18.839718 32338 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:26:18.875277 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:18.889950 26789 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:18.852475 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:18.890935 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55010.6us
I20260709 12:26:18.893406 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:18.894886 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.895748 26050 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:18.898144 26788 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:18.899036 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.898044 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.899712 26195 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:18.888641 32362 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:18.903578 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:18.920547 26196 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:18.919826 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:18.921902 26197 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:18.923341 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:18.924181 26046 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.900660 32367 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:18.900776 32388 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:18.926810 32386 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:18.934089 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:18.930245 31962 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.928675 32249 raft_consensus.cc:2764] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:18.897899 32407 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:18.933338 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 88909.7us
I20260709 12:26:18.909790 32383 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:26:18.927888 32409 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:26:18.958216 31962 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.971139 31962 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:18.933050 31956 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:26:18.935772 32322 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:18.905921 32040 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:26:18.978173 26046 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:18.926457 32194 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:26:18.881860 32113 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:18.927964 32234 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:26:18.933300 31895 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:18.984642 31956 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.919834 32307 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.022745 32407 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.023716 32307 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:19.024075 26199 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:19.029052 32370 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.029281 32321 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:19.029874 32396 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.397s	user 0.001s	sys 0.001s
I20260709 12:26:19.029896 32370 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.031843 32347 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.036793 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2079) took 1097 ms. Trace:
I20260709 12:26:19.037272 26203 rpcz_store.cc:276] 0709 12:26:17.939506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.939683 (+   177us) service_pool.cc:224] Handling call
0709 12:26:19.036616 (+1096933us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b5400399754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:19.036728 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1945372544}
I20260709 12:26:19.040624 32218 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.048735 32218 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.050884 26200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:19.051721 26200 raft_consensus.cc:606] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:18.955369 32388 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.052110 26200 raft_consensus.cc:614] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:19.054231 32388 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:19.006734 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69353us
I20260709 12:26:19.060978 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:19.061911 26046 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:19.062388 26050 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:19.012205 32374 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.011495 31956 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.013002 31895 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:19.013505 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.007315 32249 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:26:19.013840 32040 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:19.101290 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:19.015007 32263 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.096390 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:19.097285 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:19.111446 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.117854 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.010216 32236 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:19.144580 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:19.145615 26197 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.140370 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.146468 26195 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:19.146814 26196 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:19.144512 26201 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:19.127293 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:19.157162 26756 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.310s (SENT) [suppressed 18 similar messages]
I20260709 12:26:19.161132 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.138451 32383 consensus_queue.cc:1243] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:19.166421 26201 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:19.167634 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.169174 26201 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:19.092478 32352 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.171265 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:19.171849 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:19.133251 32386 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.176075 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:19.131899 32040 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:19.173868 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.131903 32211 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.138280 32367 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:19.148869 31895 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.161258 32252 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:19.161617 32263 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.163676 32364 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.198657 32211 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.203523 26196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:19.203293 31895 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.204310 26196 raft_consensus.cc:606] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:19.205516 32307 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:19.206182 31895 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.207479 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.208182 31956 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.208803 32367 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.210481 32040 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.208200 26788 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:19.099706 32374 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.223357 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.229641 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:19.230558 26049 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:19.160912 32236 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.232997 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:19.233352 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59566.5us
I20260709 12:26:19.239017 26197 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:19.239104 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.211941 32040 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.236616 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:19.245990 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.246845 26196 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:19.213634 32252 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:19.220139 32211 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.186851 32386 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.180811 32352 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.264942 26195 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:19.268297 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:19.247057 32285 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:19.254340 32307 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.221709 32364 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.254832 32252 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.250633 32416 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.221247 32263 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:19.302394 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.301987 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 119366us
W20260709 12:26:19.364895 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 123059us
I20260709 12:26:19.259290 32388 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:26:19.283857 32307 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:19.368587 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62653.5us
I20260709 12:26:19.373153 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.373297 26054 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:19.370628 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 213511us
I20260709 12:26:19.375672 32308 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:19.288688 32416 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.380098 26049 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:19.384713 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:19.376209 32308 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:19.391116 26052 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:19.286266 32113 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:19.385880 26049 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:26:19.368422 32417 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:26:19.393103 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:19.406319 26046 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:19.367805 32191 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:19.413964 32420 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.414885 32420 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.418344 26788 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:19.427356 32211 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:26:19.445115 32415 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.445863 32415 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.379684 32307 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:19.454680 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.375722 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.466002 26046 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:19.466884 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91250.4us
I20260709 12:26:19.397889 32308 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:19.477479 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 79033.3us
I20260709 12:26:19.486780 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.484802 26046 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:26:19.382654 32416 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.473793 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.408139 32383 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.491048 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:19.495414 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.453720 32191 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:19.496404 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 129667us
I20260709 12:26:19.492560 32321 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
W20260709 12:26:19.554473 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.505357 32421 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:26:19.565616 32415 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.566264 32415 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:19.566992 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.538745 32420 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.567622 26782 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:19.494011 32340 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:26:19.497241 32361 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:26:19.570955 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:19.572022 32420 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.572444 32420 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.572778 32420 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.572826 32191 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.574182 32191 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:19.578547 32252 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.581110 32420 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.590106 32236 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.592051 32364 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.502190 32428 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.593324 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.499843 32418 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.508487 32323 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:19.594058 32323 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.594463 32323 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.509935 32386 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:19.596963 26045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2037) took 1321 ms (client timeout 1534 ms). Trace:
W20260709 12:26:19.597390 26045 rpcz_store.cc:269] 0709 12:26:18.280918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.286581 (+  5663us) service_pool.cc:224] Handling call
0709 12:26:19.596767 (+1310186us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:19.596902 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369934208}
I20260709 12:26:19.510227 32352 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:19.599836 32370 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:19.511323 32405 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.892s	user 0.001s	sys 0.016s
W20260709 12:26:19.507695 32422 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:26:19.521687 32402 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:19.601817 32402 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:19.601954 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2521) took 1166 ms. Trace:
I20260709 12:26:19.602335 26127 rpcz_store.cc:276] 0709 12:26:18.435236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.435880 (+   644us) service_pool.cc:224] Handling call
0709 12:26:19.601782 (+1165902us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:19.601897 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1856758528}
I20260709 12:26:19.602712 32402 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.603634 26094 leader_election.cc:400] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:19.603790 32402 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.540541 32266 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:19.499033 32425 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.606637 32369 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:19.607263 32369 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:19.506337 32263 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.535190 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1350) took 1602 ms. Trace:
I20260709 12:26:19.609519 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:19.534580 32407 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:19.610801 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.610962 32407 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.527119 32330 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:19.609088 26786 rpcz_store.cc:276] 0709 12:26:17.932833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.933026 (+   193us) service_pool.cc:224] Handling call
0709 12:26:18.403636 (+470610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.403856 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.403887 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:18.403931 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:18.403975 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.404038 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.535013 (+1130975us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54000587d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:19.535131 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051279616}
I20260709 12:26:19.614041 32407 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.475883 32265 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:19.617172 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.618491 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.503923 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.552641 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.619629 26194 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
W20260709 12:26:19.619814 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116023us
I20260709 12:26:19.620639 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.622058 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.622370 32194 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:19.623337 32361 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:19.463145 32211 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:19.624406 32263 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:19.624348 32211 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.625345 32263 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:19.554211 32383 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.628983 32362 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.630226 32383 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.593029 32428 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.566825 32415 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.538574 32314 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:19.634101 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.634455 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.634953 26194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:19.525048 32347 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:19.594058 32418 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.649365 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.470681 32367 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:26:19.635160 26121 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:19.412426 32113 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.635742 26194 raft_consensus.cc:606] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:19.625844 32263 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.652789 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:19.637014 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:19.637732 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.654837 26785 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:19.638192 32040 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:19.638478 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.658543 26791 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:19.638955 26783 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:19.639302 32386 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:19.608656 26782 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:19.660965 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.630419 32362 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.641556 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.642558 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.643429 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.666543 26128 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:19.667122 26197 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:19.643908 26129 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:19.644618 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.646226 26123 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:19.668602 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.600003 26200 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:19.669507 26128 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:19.671432 26199 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:19.671653 26124 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:19.672549 26129 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:19.671451 26785 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:19.673429 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.674237 26122 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:19.677248 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:19.655767 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43779.7us
I20260709 12:26:19.654392 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:19.657563 26121 raft_consensus.cc:1217] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Deduplicated request from leader. Original: 13.7->[15.8-21.11]   Dedup: 17.9->[19.10-21.11]
I20260709 12:26:19.662986 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:19.635743 32200 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:19.741559 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:19.741478 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:19.742616 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59367us
I20260709 12:26:19.743608 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:19.702247 26129 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:26:19.744328 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107390us
I20260709 12:26:19.744825 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:19.743568 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:19.746573 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:19.650986 32285 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:19.746907 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:19.747730 26786 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.655407 32347 consensus_queue.cc:1243] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:26:19.748375 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 94072us
I20260709 12:26:19.607028 32425 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.679644 26121 pending_rounds.cc:85] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 9
I20260709 12:26:19.657953 32263 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.657508 31962 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.770956 26121 pending_rounds.cc:107] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:26:19.778550 26121 raft_consensus.cc:2889] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.769809 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.769562 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:19.773273 26048 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:19.672365 32236 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.679370 32266 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.786666 26783 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:19.640534 32314 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:19.789995 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:19.792783 26045 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:19.679960 32414 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.790733 26050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.104s	sys 0.177s
I20260709 12:26:19.791436 26790 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:19.793586 26050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.104s	sys 0.177s
I20260709 12:26:19.778626 32347 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:19.658432 32318 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.120s
I20260709 12:26:19.663604 32338 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.115s
I20260709 12:26:19.776588 32407 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:19.776037 32425 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.671280 32040 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.788053 32200 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.769433 32269 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.777905 31962 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:19.834240 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:19.861223 26124 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:19.788488 32252 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.791751 32420 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.868387 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:19.863232 31962 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.868062 32338 consensus_queue.cc:1243] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:19.829973 32347 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:19.842326 32194 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:19.823638 32364 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.860473 32269 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.865099 32211 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.795590 32266 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:19.929003 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 251862us
I20260709 12:26:19.798229 32414 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.833896 32407 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.942327 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:19.935376 32269 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.950484 32266 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:19.950840 32113 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:19.951079 32266 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:19.951618 32266 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:19.947082 32438 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:26:19.956875 32418 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:19.957538 32266 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:19.959584 32403 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.272s	user 0.000s	sys 0.003s
I20260709 12:26:19.963682 26122 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:19.967729 26786 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:19.867713 32307 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.984886 32308 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:19.986215 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.989782 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.992848 26198 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:19.999132 32236 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:19.969250 32342 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:19.936470 32364 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:19.976320 32415 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.022444 32106 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:20.030459 32429 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.045259 32429 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.030987 26123 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:20.036037 26791 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:20.035171 26129 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:20.055860 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2448) took 1557 ms. Trace:
I20260709 12:26:19.982277 32224 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:20.056357 26202 rpcz_store.cc:276] 0709 12:26:18.498459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.498570 (+   111us) service_pool.cc:224] Handling call
0709 12:26:18.498859 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.552316 (+ 53457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.552350 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.552884 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.658181 (+105297us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:18.682299 (+ 24118us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.682628 (+   329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.682728 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.026263 (+343535us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b180000d660. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:19.958962 (+932699us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:20.055647 (+ 96685us) raft_consensus.cc:1685] finished
0709 12:26:20.055669 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.055799 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50616,"prepare.queue_time_us":24613,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":45095,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":211,"spinlock_wait_cycles":6385920,"thread_start_us":104847,"threads_started":1,"wal-append.queue_time_us":134084}
I20260709 12:26:20.068434 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:20.073129 26123 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:20.079459 26129 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:19.980597 32269 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.086467 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:20.087743 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:20.088541 26791 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:20.072681 32314 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:20.010835 32347 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.091586 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:20.052675 32438 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:26:20.074859 32429 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.035104 32194 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.088078 32249 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:20.122291 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.079221 32285 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:20.098985 32416 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.000s
I20260709 12:26:20.135283 26791 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:20.095688 32428 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.151548 26129 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:20.119824 32314 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.158450 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:20.161211 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 109107us
I20260709 12:26:20.126662 32249 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.094079 32386 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:26:20.175863 32456 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:20.148609 32444 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:20.150894 32429 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:20.185045 32418 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:20.169466 32314 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:20.190586 32323 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.000s
I20260709 12:26:20.191617 32269 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.169849 32249 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.172436 32428 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:20.153632 32416 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.219579 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:20.219460 32447 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:26:20.211026 32414 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:20.236204 32451 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:20.212105 32429 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:20.236956 32445 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:20.240144 32314 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.232774 32459 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:20.241547 32330 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.000s
I20260709 12:26:20.242811 32307 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.244447 32407 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.247422 32415 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.254930 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.297760 26791 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17.
I20260709 12:26:20.215596 32323 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.263437 32364 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.216426 32285 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.219115 32440 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.207782 32444 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.228832 32265 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:20.277985 32434 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 12:26:20.299350 26018 leader_election.cc:400] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17.
I20260709 12:26:20.367399 26018 leader_election.cc:403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:20.278170 32263 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.291080 31968 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.270416 31962 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.288952 32459 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.275224 32416 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.317227 32323 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:20.318584 32458 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:20.321883 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.384136 26784 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:20.311357 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:20.385542 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74327.6us
I20260709 12:26:20.386622 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:20.230324 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:20.306872 32431 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.364s	user 0.000s	sys 0.001s
W20260709 12:26:20.268136 32191 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.002s
I20260709 12:26:20.363549 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:20.392202 26791 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:20.243848 32192 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.279799 32435 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 12:26:20.241621 32200 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.277045 32321 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:20.233807 32418 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:20.294668 32269 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.377823 31968 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.416997 32269 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:20.374367 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75162.8us
I20260709 12:26:20.230298 32428 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.419266 32432 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.419804 32432 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.420642 32428 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:20.389324 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 202861us
I20260709 12:26:20.401206 26196 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:20.778504 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:20.780018 32330 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:20.780273 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 971) took 1138 ms. Trace:
I20260709 12:26:20.780447 32330 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.381356 32386 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:20.781085 32330 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.327750 32415 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.416282 26791 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:20.365682 32383 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:20.785979 32469 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 12:26:20.786154 32383 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.786716 32465 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:20.421931 32432 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.424227 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.780722 26196 rpcz_store.cc:276] 0709 12:26:19.641852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.643057 (+  1205us) service_pool.cc:224] Handling call
0709 12:26:19.675363 (+ 32306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.778005 (+1102642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.780002 (+  1997us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:20.780188 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666408320}
I20260709 12:26:20.789472 26782 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:20.788561 26757 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:20.390569 32429 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.424402 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2050) took 1189 ms. Trace:
I20260709 12:26:20.424314 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.406844 32321 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:20.800820 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.398661 32307 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:20.438586 31962 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.439308 32374 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.441408 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.443475 32308 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.365927 32440 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.455276 32263 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.424228 32249 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.837849 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:20.457811 32436 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.486s	user 0.002s	sys 0.000s
W20260709 12:26:20.459412 32461 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:26:20.464684 32398 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.846s	user 0.004s	sys 0.016s
I20260709 12:26:20.471336 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:20.473548 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:20.489295 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.425043 32231 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:20.498778 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:20.419288 32269 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.501869 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3337) took 1333 ms. Trace:
I20260709 12:26:20.407472 32192 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.505555 32444 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.505963 32367 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.506971 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:20.895371 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 93324.5us
I20260709 12:26:20.895753 26047 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:20.495918 32211 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.509063 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.498598 32323 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:20.516844 32369 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:20.513069 32404 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.828s	user 0.018s	sys 0.014s
I20260709 12:26:20.511958 32040 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.501124 32347 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.527266 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.527720 26195 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:20.528007 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:20.909511 26202 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:20.846865 32370 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.666805 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:20.676898 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.910764 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:20.912719 26197 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:20.913712 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.914517 26200 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:20.915719 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.673570 32106 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:20.916425 26200 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:20.916561 32106 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.917553 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.918802 32106 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.703944 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.608515 32418 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.922736 26197 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
W20260709 12:26:20.732993 32472 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:20.741197 32459 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.701334 32266 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:20.926000 32266 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.698191 32331 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:20.928690 32331 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.931751 32331 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:20.701784 32362 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.765542 32194 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.911789 32323 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.945451 32362 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.947006 32323 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.792330 26790 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:20.793040 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:20.793754 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:20.949373 26121 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:20.950536 31968 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.909662 32211 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.791982 26053 rpcz_store.cc:276] 0709 12:26:19.235521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.248787 (+ 13266us) service_pool.cc:224] Handling call
0709 12:26:19.249240 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.376680 (+127440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.376712 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.377203 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.804541 (+427338us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:19.804584 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.804768 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.279239 (+474471us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:20.414885 (+135646us) raft_consensus.cc:1685] finished
0709 12:26:20.414910 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.424335 (+  9425us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17063,"raft.queue_time_us":40274,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":69606,"spinlock_wait_cycles":69544704,"thread_start_us":551056,"threads_started":2,"wal-append.queue_time_us":496089}
I20260709 12:26:20.788170 32465 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:26:20.794924 26128 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:20.805245 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.393909 32364 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:20.952934 32415 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.953115 26201 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:20.953568 32370 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.837911 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.955798 32370 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:20.957681 32437 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.033s	user 0.004s	sys 0.000s
I20260709 12:26:20.890312 26049 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.891424 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:20.960335 32479 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.141s	user 0.000s	sys 0.003s
I20260709 12:26:20.960637 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1397) took 1501 ms. Trace:
I20260709 12:26:20.780403 32407 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 12:26:20.960983 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69641.3us
I20260709 12:26:20.962036 32407 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:20.803567 32383 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.802924 32307 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.963238 32307 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.854419 26051 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:20.965498 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:20.961017 26788 rpcz_store.cc:276] 0709 12:26:19.459144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.459545 (+   401us) service_pool.cc:224] Handling call
0709 12:26:19.460084 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.460941 (+   857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.460988 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.461638 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.836212 (+374574us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:19.843740 (+  7528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.843846 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.956938 (+1113092us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:20.960322 (+  3384us) raft_consensus.cc:1685] finished
0709 12:26:20.960345 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.960577 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2299,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11641,"spinlock_wait_cycles":49009024,"thread_start_us":374157,"threads_started":1,"wal-append.queue_time_us":395387}
W20260709 12:26:20.965538 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54828.5us
W20260709 12:26:20.969195 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 176240us
I20260709 12:26:20.972671 26049 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:20.903563 32192 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.964650 32383 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.904860 32367 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.813781 32113 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:20.907405 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.988772 26789 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:20.977629 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:20.910022 26194 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:20.652565 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:20.704079 32425 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.000s
I20260709 12:26:20.934578 32249 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:20.911831 32321 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:20.941229 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:20.945488 32194 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:20.970575 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:20.974506 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.016204 26197 raft_consensus.cc:2410] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:20.889122 26050 rpcz_store.cc:276] 0709 12:26:19.168182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.170351 (+  2169us) service_pool.cc:224] Handling call
0709 12:26:19.170626 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.170957 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.170982 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.185778 (+ 14796us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.793994 (+608216us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:19.794030 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.794087 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.306061 (+511974us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:20.488928 (+182867us) raft_consensus.cc:1685] finished
0709 12:26:20.488953 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.501804 (+ 12851us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21188,"raft.queue_time_us":2515,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":15455,"spinlock_wait_cycles":69614464,"thread_start_us":581257,"threads_started":1,"wal-append.queue_time_us":651550}
I20260709 12:26:20.977276 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:20.903966 32442 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.814142 32439 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:20.986084 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:20.964774 32465 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.969539 32040 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:20.991269 26198 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:26:20.953115 32364 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:20.949683 32440 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:20.993229 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:20.976265 31962 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.970427 32308 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:21.007673 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:20.944165 32266 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.033303 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:26:21.153942 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 120780us
I20260709 12:26:20.984993 32269 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.978914 32285 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:20.948163 32362 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.138043 26121 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:20.987869 32444 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.968753 32347 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.187299 32347 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:21.141973 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 148839us
I20260709 12:26:20.969808 31968 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:21.189625 31968 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:21.190200 31968 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.190699 31968 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:21.192636 32428 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.193437 32428 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.190717 26129 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:21.143481 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.143589 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:21.200608 32428 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.207337 26124 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:21.208272 31968 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.208868 32444 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.211611 31968 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.142875 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:21.218127 31968 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.163069 32266 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:21.219102 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.163204 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.048605 32040 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.030455 32331 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:21.218796 31968 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.046650 32439 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.152611 31962 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.226966 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1365) took 1589 ms. Trace:
I20260709 12:26:21.171594 32330 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.227771 32386 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17.
I20260709 12:26:21.158964 32476 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.228691 32476 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.229168 32331 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.230033 32330 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.230890 26048 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:21.231005 32476 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.231314 32331 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.141038 32240 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.140041 32465 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.232579 32240 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.234627 32465 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.234817 32240 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:21.235175 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.235486 32211 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.168717 32269 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.140539 31961 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:21.237362 31961 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.237771 32235 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.238513 32269 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.238322 32235 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.138595 32442 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.239367 31961 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.239903 31961 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.240579 32235 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.212759 32200 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:21.227813 31962 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.241468 32200 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.161748 32285 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.241984 32439 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.242143 32285 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.243347 32211 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.243844 32211 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.245764 32388 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.246326 32388 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.247359 32211 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.247292 32307 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.159497 32321 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.249892 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.250312 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.214442 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.251106 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:21.251004 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.241561 26200 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:21.216861 32194 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.218127 26125 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:21.219260 32266 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.220685 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.265727 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:20.988987 32481 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:21.272806 26122 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:20.998883 32432 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.162s
I20260709 12:26:21.224637 32040 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.260533 32200 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:21.221843 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.284073 26201 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.152997 32150 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:21.240553 31961 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.288496 31968 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.241190 32442 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.181195 32368 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:20.999614 32113 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.047228 32402 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:21.227378 26785 rpcz_store.cc:276] 0709 12:26:19.637746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.741681 (+103935us) service_pool.cc:224] Handling call
0709 12:26:20.291696 (+550015us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.319144 (+ 27448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.319208 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.319895 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.529044 (+209149us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:20.529084 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.529149 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.704692 (+175543us) raft_consensus.cc:1685] finished
0709 12:26:20.704714 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.226678 (+521964us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54004f79d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:21.226901 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28430,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":39341,"spinlock_wait_cycles":1264090368,"thread_start_us":208771,"threads_started":1,"wal-append.queue_time_us":380932}
I20260709 12:26:21.259413 26194 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:20.994805 32407 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.163s
I20260709 12:26:21.265749 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.313723 26124 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:26:21.314822 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:21.315415 26124 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.268496 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.250464 31962 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.321319 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.322611 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.269524 26128 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:21.270112 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.270774 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:21.140463 32364 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:20.983860 32383 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:21.222030 32484 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:26:21.273648 26202 raft_consensus.cc:2368] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.274724 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.331028 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:21.277972 32464 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.603s	user 0.009s	sys 0.002s
I20260709 12:26:21.279487 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.274724 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.335644 26195 raft_consensus.cc:2368] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:21.335570 26197 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:21.281137 32388 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.159402 32308 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:21.289996 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 319464us
I20260709 12:26:21.340129 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.340932 26195 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:21.341812 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.211874 32444 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.342777 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.280751 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.291460 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.146463 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.293258 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:21.297111 26201 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:21.297762 26125 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:21.169315 32419 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.002012 32425 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.356305 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.323661 26121 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:21.356735 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.327793 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:21.357342 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.357803 26051 raft_consensus.cc:2368] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.358052 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.320978 32402 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.329982 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:21.276401 32347 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.361627 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.336550 26791 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:21.336661 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.280735 32440 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:21.337870 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.338932 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:21.346946 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:21.353885 26124 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:21.358645 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.358654 26784 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:21.355872 32191 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.366150 26094 leader_election.cc:400] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.398316 26094 leader_election.cc:403] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:21.399247 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:21.362524 32402 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.359304 26126 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.325632 32150 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:21.314822 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.321177 32368 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.331400 32442 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.328694 32323 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:21.363345 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.362202 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:21.446538 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.446668 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.404995 26045 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:21.406299 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.329260 31962 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.382664 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.406132 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.366866 26128 raft_consensus.cc:2368] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:21.458086 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.379734 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.407418 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:21.459317 26786 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:21.380841 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:21.425202 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.382344 26198 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:21.460799 26789 raft_consensus.cc:2368] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15.
I20260709 12:26:21.439240 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.392542 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.393076 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.436030 26194 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:21.463738 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.401481 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.464449 26046 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:21.464751 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57415.3us
I20260709 12:26:21.404614 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:21.466105 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61616us
I20260709 12:26:21.466691 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.366930 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.465663 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.462007 26784 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:21.451531 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:21.364908 32430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.016s
I20260709 12:26:21.422183 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.473524 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.474756 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.423727 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.432301 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.458384 26199 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:26:21.464900 26782 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:21.366511 26052 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.476191 26199 pending_rounds.cc:85] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 10
I20260709 12:26:21.466781 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.476634 26199 pending_rounds.cc:107] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:26:21.469285 26019 leader_election.cc:400] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:21.477180 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.476209 26203 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:21.406653 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3456) took 1253 ms. Trace:
I20260709 12:26:21.378733 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:21.400429 32211 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:21.477991 26123 rpcz_store.cc:276] 0709 12:26:20.153266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.153418 (+   152us) service_pool.cc:224] Handling call
0709 12:26:20.159360 (+  5942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.159807 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.159849 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.160394 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.372213 (+211819us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:20.372285 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.372358 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.277352 (+904994us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:21.406360 (+129008us) raft_consensus.cc:1685] finished
0709 12:26:21.406383 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.406594 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12247,"raft.queue_time_us":114,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":3016,"spinlock_wait_cycles":8800512,"thread_start_us":198696,"threads_started":1,"wal-append.queue_time_us":331914}
I20260709 12:26:21.482013 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.470429 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.400280 32370 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.400142 32308 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:21.472548 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47459.4us
I20260709 12:26:21.467530 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.476962 26199 raft_consensus.cc:2889] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.407487 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.498900 26054 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:21.498975 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.428633 32420 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.384305 32194 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.509371 32266 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:21.509665 32194 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:21.513499 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 134907us
W20260709 12:26:21.514815 32488 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:21.508680 32150 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.482645 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.484730 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:21.479369 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.033s	sys 0.009s
I20260709 12:26:21.519547 26200 raft_consensus.cc:2368] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.422358 32240 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.482687 26790 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:21.516366 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.485607 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:21.488021 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.472070 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.524844 26203 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:26:21.525012 26194 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:21.444288 32249 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.488111 26050 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:21.499935 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.530722 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.477548 26019 leader_election.cc:403] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:21.483560 32106 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:21.433463 32307 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.360843 32419 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.444698 32476 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.431849 32383 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.503793 26052 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:21.505543 32483 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:21.555637 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 86498.5us
I20260709 12:26:21.497038 32490 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:21.507169 32465 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.508666 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:21.519598 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.034s	sys 0.009s
W20260709 12:26:21.564613 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 654143us
I20260709 12:26:21.405172 31961 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:21.508003 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.483698 32430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.000s	sys 0.016s
I20260709 12:26:21.522189 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.525198 26789 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:21.494297 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.525902 26126 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:21.584959 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.585894 26126 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:21.555190 31968 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:21.587527 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.588186 26120 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:21.526497 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.468271 32489 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:21.417342 32369 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.590472 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.591670 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.586977 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:21.540212 32249 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.593055 26126 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.594040 32285 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.595050 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:21.596175 32419 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.597157 32494 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:21.597615 32040 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.598800 32113 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:21.598912 31968 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:21.600593 26053 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:21.602144 26127 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:21.513526 32491 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:21.534152 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.573542 32106 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:21.602564 32367 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.534566 32370 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.603626 32370 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.604068 32370 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.604589 31962 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.604947 26201 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:21.605333 32362 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:21.605855 32362 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.599858 32040 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.606357 26201 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:21.607503 32425 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.534690 32240 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.609624 32368 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.562901 32476 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.610718 32240 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.489828 32211 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.611510 32211 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.375890 32347 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.612421 32476 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.613931 32347 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.614734 26122 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
W20260709 12:26:21.614882 32487 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:21.480459 32415 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.616430 32415 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.617417 26126 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:21.575465 32465 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.618288 32465 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.619053 32415 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.397662 32191 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.480880 32429 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:21.456645 32323 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.621474 32191 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:21.538684 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.450984 32442 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.622196 26200 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.401053 32439 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.505744 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:21.626796 26782 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:21.504383 32420 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.560122 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.580181 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.581107 26045 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:21.642019 26203 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.599979 31968 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.574091 32269 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:21.623010 26197 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:21.500656 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.433940 32440 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.625138 26194 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:21.562660 32483 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.629287 32362 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.630833 32442 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:21.624792 32485 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.603637 32106 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.626113 32490 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:21.634451 26787 raft_consensus.cc:2368] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20.
I20260709 12:26:21.634903 32494 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.682274 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:21.683214 26122 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:21.638353 32465 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.639163 32211 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.622233 32323 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:21.646853 26054 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:21.594650 32249 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.639628 32200 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.652449 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:21.662086 26200 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:21.656224 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:21.704768 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:21.739400 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 87047.9us
I20260709 12:26:21.671120 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:21.741472 26202 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.633790 32439 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.675303 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:21.646184 32420 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.749524 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.677780 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.749841 26200 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:21.750322 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:21.680050 26052 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:26:21.686327 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 235287us
I20260709 12:26:21.688097 26782 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:21.694757 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.664248 32483 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.697703 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:21.740031 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83901.5us
I20260709 12:26:21.758240 26124 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.669061 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.747000 26053 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:21.760668 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.754880 26126 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:21.756853 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.759124 26194 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:21.764566 26094 leader_election.cc:400] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.672258 32481 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:26:21.764889 26094 leader_election.cc:403] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:21.693594 32362 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.781656 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.781988 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.765125 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:21.766610 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:21.771600 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.769459 26124 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:21.777374 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:21.786095 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:21.787791 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.788712 26203 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:21.793730 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.794494 26782 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:21.795464 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:21.783398 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 268117us
I20260709 12:26:21.772573 32249 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:21.798863 32323 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:21.799098 32442 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.799737 32040 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:21.800307 32330 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:21.800532 32429 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.800719 32368 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:21.795310 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:21.750481 32269 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:21.697074 32494 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.807730 26167 leader_election.cc:400] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15.
I20260709 12:26:21.945027 26167 leader_election.cc:403] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:26:21.945626 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 160795us
I20260709 12:26:21.758313 32266 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.810283 32347 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 12:26:21.791956 26202 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:21.829811 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.835721 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.684266 32485 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.849869 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.830560 32231 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.859335 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:21.957926 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 98673us
I20260709 12:26:21.958997 32440 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:21.881248 26053 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:21.961299 32224 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:21.810029 32307 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:21.963714 32307 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:21.964999 32307 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:21.806754 32415 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:21.969722 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:21.971470 32224 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:21.972844 31962 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.973594 31962 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.973644 32231 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.976765 32224 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:21.703366 32492 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 12:26:21.919373 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.865113 32249 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.865788 32263 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:21.864125 32323 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.993999 32323 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.868863 32490 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:21.943966 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148975us
I20260709 12:26:21.997680 32419 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:21.913653 32442 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:21.997932 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:21.999042 32428 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:21.793162 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.000263 32442 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:21.947310 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:22.003472 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56240.8us
I20260709 12:26:21.805751 32362 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.009727 32362 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:21.917078 32364 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.578s	user 0.005s	sys 0.009s
I20260709 12:26:22.010637 32364 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:22.014380 26787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1366) took 2376 ms (client timeout 2173 ms). Trace:
W20260709 12:26:22.014976 26787 rpcz_store.cc:269] 0709 12:26:19.638001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.638265 (+   264us) service_pool.cc:224] Handling call
0709 12:26:20.837594 (+1199329us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:26:22.014033 (+1176439us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:22.014247 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1834626304}
I20260709 12:26:22.016966 32285 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.021291 32191 raft_consensus.cc:2764] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:26:21.999791 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:21.989732 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:21.944263 32429 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:21.954335 32402 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 12:26:21.922215 32194 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:22.030155 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.004s	sys 0.011s
I20260709 12:26:21.955595 32150 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:21.949235 32266 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:21.944469 32269 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.037045 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.921837 32330 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:21.949044 32113 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.049767 32483 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:22.053294 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.004s	sys 0.011s
W20260709 12:26:22.081282 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 583261us
I20260709 12:26:22.032887 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:21.988147 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:22.058064 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.062224 26787 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:21.955829 32491 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 14. Current term is 15.
I20260709 12:26:21.987835 32249 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:21.994000 32200 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.052006 32465 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.084719 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.083840 26202 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
W20260709 12:26:22.091545 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 103925us
W20260709 12:26:22.085876 31895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.002s	sys 0.013s
W20260709 12:26:22.094811 31895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.002s	sys 0.013s
I20260709 12:26:22.094559 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:22.066740 32266 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.055274 32442 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:22.054247 32285 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:22.066576 32269 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.119377 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.125034 26786 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:22.128393 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.036612 32370 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.036420 32263 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.109771 32442 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.085391 32367 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:22.143713 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.000s	sys 0.019s
I20260709 12:26:22.055671 32402 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:22.151319 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66687.1us
I20260709 12:26:22.155900 26020 leader_election.cc:400] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:22.066104 32428 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:26:22.158216 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:22.152942 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.000s	sys 0.020s
W20260709 12:26:22.149013 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45874.9us
W20260709 12:26:22.159687 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 674117us
I20260709 12:26:22.079346 32113 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.129897 32211 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.170121 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.109030 32321 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:22.150754 32442 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:22.140584 32504 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:22.178020 26123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.021s	sys 0.040s
I20260709 12:26:22.179098 26195 raft_consensus.cc:2368] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:22.186510 26053 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:22.170312 26128 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:22.192363 26123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.021s	sys 0.040s
I20260709 12:26:22.157629 32368 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:22.172186 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:22.210141 32263 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.193156 32330 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.174007 32367 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:22.219570 32367 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.220086 32367 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:22.199854 32368 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.213639 32364 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.234836 26784 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:22.221114 32211 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:22.242424 32485 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.241279 32368 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:22.252712 32211 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.215307 32231 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:22.278371 32499 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.330s	user 0.001s	sys 0.003s
I20260709 12:26:22.277017 32429 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.281505 32494 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.306478 32483 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.320540 32370 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:22.329538 32508 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:22.323588 32402 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:22.372438 32420 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.002s
I20260709 12:26:22.373327 32420 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.379519 32420 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.420293 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.423648 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:22.197165 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.424763 26046 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:22.425155 26045 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 14.
I20260709 12:26:22.427088 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.429056 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.429747 26051 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:22.433096 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:22.436334 26050 raft_consensus.cc:2368] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:22.432204 26049 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:22.441334 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.444561 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.453433 26050 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:22.456425 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:22.457129 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 433486us
I20260709 12:26:22.462435 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:22.217671 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45587.8us
I20260709 12:26:22.465627 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:22.467998 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:22.470371 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 292864us
I20260709 12:26:22.498225 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 986) took 1154 ms. Trace:
I20260709 12:26:22.286583 32211 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.386269 32442 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:22.504733 26203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.013s	sys 0.017s
W20260709 12:26:22.505380 26203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.013s	sys 0.017s
I20260709 12:26:22.506875 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 991) took 1147 ms. Trace:
I20260709 12:26:22.507301 32442 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.507331 32113 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:22.509769 32113 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:22.510931 32113 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.514075 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2482) took 1090 ms. Trace:
W20260709 12:26:22.518278 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2061) took 1188 ms (client timeout 1543 ms). Trace:
W20260709 12:26:22.519068 26047 rpcz_store.cc:269] 0709 12:26:21.329922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.330154 (+   232us) service_pool.cc:224] Handling call
0709 12:26:22.517607 (+1187453us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b540022c254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:22.517993 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288997120}
I20260709 12:26:22.523779 32113 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.524778 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1370) took 1168 ms. Trace:
I20260709 12:26:22.521529 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2103) took 1238 ms. Trace:
I20260709 12:26:22.480007 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.527078 32231 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:22.499320 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 404720us
I20260709 12:26:22.529510 26201 rpcz_store.cc:276] 0709 12:26:21.282595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.607351 (+324756us) service_pool.cc:224] Handling call
0709 12:26:21.607665 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.608037 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.608069 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.644524 (+ 36455us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.040639 (+396115us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:22.040680 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.040744 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.520874 (+480130us) raft_consensus.cc:1685] finished
0709 12:26:22.520923 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.521245 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":395733,"threads_started":1,"wal-append.queue_time_us":599087}
I20260709 12:26:22.541225 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2117) took 1108 ms. Trace:
I20260709 12:26:22.541908 26197 rpcz_store.cc:276] 0709 12:26:21.432536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.445452 (+ 12916us) service_pool.cc:224] Handling call
0709 12:26:22.540565 (+1095113us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:22.540939 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321172224}
I20260709 12:26:22.507417 26203 rpcz_store.cc:276] 0709 12:26:21.359457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.001167 (+641710us) service_pool.cc:224] Handling call
0709 12:26:22.001570 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.001918 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.001962 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.002525 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.506243 (+503718us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:22.506359 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.506503 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.506565 (+    62us) raft_consensus.cc:1685] finished
0709 12:26:22.506576 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.506772 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33698,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":27839,"thread_start_us":502874,"threads_started":1,"wal-append.queue_time_us":342164}
I20260709 12:26:22.499032 26200 rpcz_store.cc:276] 0709 12:26:21.343596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.750772 (+407176us) service_pool.cc:224] Handling call
0709 12:26:21.751090 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.751416 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.751442 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.769208 (+ 17766us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.224844 (+455636us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:22.224903 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.224974 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.497390 (+272416us) raft_consensus.cc:1685] finished
0709 12:26:22.497444 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.497886 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":739,"raft.queue_time_us":760,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":11883,"thread_start_us":419858,"threads_started":1,"wal-append.queue_time_us":518724}
I20260709 12:26:22.553579 32370 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:22.554311 32370 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:22.554903 32370 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:22.556587 32364 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:22.554319 32265 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:22.561306 32265 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:22.562446 32429 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.568873 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:22.563151 32265 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.571630 32265 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.571925 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:22.525353 26785 rpcz_store.cc:276] 0709 12:26:21.356599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.356724 (+   125us) service_pool.cc:224] Handling call
0709 12:26:21.357053 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:21.360306 (+  3253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.360355 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:21.366994 (+  6639us) log.cc:844] Serialized 52 byte log entry
0709 12:26:21.737126 (+370132us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.078032 (+340906us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.078255 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.078330 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.250836 (+172506us) spinlock_profiling.cc:243] Waited 75.1 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:22.523910 (+273074us) raft_consensus.cc:1685] finished
0709 12:26:22.524083 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.524471 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2790,"prepare.queue_time_us":335445,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1274,"raft.queue_time_us":17352,"raft.run_cpu_time_us":1095,"raft.run_wall_time_us":88712,"spinlock_wait_cycles":172699392,"thread_start_us":566031,"threads_started":2,"wal-append.queue_time_us":369595}
I20260709 12:26:22.569268 26783 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:22.514654 26202 rpcz_store.cc:276] 0709 12:26:21.423458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.138374 (+714916us) service_pool.cc:224] Handling call
0709 12:26:22.138736 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.150799 (+ 12063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.150832 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.151316 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.513121 (+361805us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:22.513238 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.513382 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.513501 (+   119us) raft_consensus.cc:1685] finished
0709 12:26:22.513519 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.513827 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21706,"raft.queue_time_us":16101,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10724,"thread_start_us":361066,"threads_started":1,"wal-append.queue_time_us":198385}
I20260709 12:26:22.563702 32429 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.571630 32420 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:22.608246 32420 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:22.608287 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1053) took 1208 ms. Trace:
I20260709 12:26:22.609980 32420 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.615890 32364 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:22.616982 32364 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.619151 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2476) took 1712 ms. Trace:
I20260709 12:26:22.619863 32364 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.607582 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.596163 32415 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:22.684125 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:22.631773 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.639900 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.696961 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.697574 32368 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.678454 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:22.705202 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 97883.7us
I20260709 12:26:22.705554 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2106) took 1421 ms. Trace:
W20260709 12:26:22.676976 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.625967 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:22.705983 26198 rpcz_store.cc:276] 0709 12:26:21.283525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.615805 (+332280us) service_pool.cc:224] Handling call
0709 12:26:21.616158 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.623672 (+  7514us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:22.160401 (+536729us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.704378 (+543977us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b18003361c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:26:22.704884 (+   506us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.704983 (+    99us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:22.705124 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.705226 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.705379 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368545,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":222,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":213,"spinlock_wait_cycles":90102656,"thread_start_us":885943,"threads_started":2,"wal-append.queue_time_us":371784}
I20260709 12:26:22.641157 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.640064 31962 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.642464 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.708945 32364 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.709203 26048 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:22.607685 32429 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.620573 32485 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.710880 32535 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260709 12:26:22.710937 32485 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.642649 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1054) took 1064 ms. Trace:
I20260709 12:26:22.712520 26128 rpcz_store.cc:276] 0709 12:26:21.577992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.579853 (+  1861us) service_pool.cc:224] Handling call
0709 12:26:22.642058 (+1062205us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b54001f77d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:22.642372 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202749312}
I20260709 12:26:22.713028 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.713388 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.706432 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:22.713959 26197 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.714191 26203 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.715571 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1060) took 1060 ms. Trace:
I20260709 12:26:22.716041 26126 rpcz_store.cc:276] 0709 12:26:21.655361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.705903 (+ 50542us) service_pool.cc:224] Handling call
0709 12:26:21.706306 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.762711 (+ 56405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.714778 (+952067us) spinlock_profiling.cc:243] Waited 48.2 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:22.715334 (+   556us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110930304}
I20260709 12:26:22.710850 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:22.656939 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.650889 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2462) took 2125 ms. Trace:
I20260709 12:26:22.673532 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1060) took 1023 ms. Trace:
I20260709 12:26:22.678848 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:22.734005 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:22.734912 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:22.728642 26786 rpcz_store.cc:276] 0709 12:26:21.649735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.829511 (+179776us) service_pool.cc:224] Handling call
0709 12:26:22.672820 (+843309us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54006bf8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:22.673129 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387865856}
I20260709 12:26:22.727041 26199 rpcz_store.cc:276] 0709 12:26:20.525330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.525457 (+   127us) service_pool.cc:224] Handling call
0709 12:26:20.666527 (+141070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.498419 (+831892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.649874 (+1151455us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b5400302354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:22.650574 (+   700us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820212992}
I20260709 12:26:22.719163 32415 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.736164 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:22.634423 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.747329 26053 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:22.741896 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.749747 26789 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:22.753232 26047 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.754930 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.756794 26202 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:22.757772 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1420) took 1337 ms. Trace:
I20260709 12:26:22.758220 26782 rpcz_store.cc:276] 0709 12:26:21.442652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.986785 (+544133us) service_pool.cc:224] Handling call
0709 12:26:21.987053 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.987297 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:22.276320 (+289023us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.757047 (+480727us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.757157 (+   110us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:22.757274 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.757409 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.757563 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":98908,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"thread_start_us":768890,"threads_started":2}
I20260709 12:26:22.760203 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.761132 26199 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:22.761590 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.762231 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.762949 26199 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:26:22.765129 32503 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:22.761744 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 127702us
I20260709 12:26:22.767753 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.768666 31962 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:22.769342 32485 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.770512 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:22.619968 26196 rpcz_store.cc:276] 0709 12:26:20.906938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.601271 (+694333us) service_pool.cc:224] Handling call
0709 12:26:21.601621 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.601856 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:21.875116 (+273260us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.612010 (+736894us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.614989 (+  2979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.615148 (+   159us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.615248 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.615367 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.618885 (+  3518us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26637,"prepare.queue_time_us":392331,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":31203,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":378,"thread_start_us":970337,"threads_started":3,"wal-append.queue_time_us":435982}
I20260709 12:26:22.609706 26790 rpcz_store.cc:276] 0709 12:26:21.396534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.535787 (+139253us) service_pool.cc:224] Handling call
0709 12:26:21.536206 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.536466 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:22.030670 (+494204us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.598772 (+568102us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.604625 (+  5853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.604804 (+   179us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.604907 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.605040 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.605294 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":909,"prepare.queue_time_us":601462,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":96,"raft.queue_time_us":2240,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":328,"spinlock_wait_cycles":144813568,"thread_start_us":993479,"threads_started":2,"wal-append.queue_time_us":500082}
I20260709 12:26:22.746356 32415 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.780441 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.780550 32307 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.780638 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.798810 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:22.781663 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:22.782392 26046 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
W20260709 12:26:22.783726 32266 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.005s
I20260709 12:26:22.789273 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:22.755935 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:22.752978 32368 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.801972 26199 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:22.802942 26047 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:22.808383 26203 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:22.811792 26194 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:22.814519 26197 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:26:22.818481 32249 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.714s	user 0.005s	sys 0.015s
I20260709 12:26:22.819109 32249 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.820399 31962 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:22.820972 31962 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:22.821444 31962 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:22.822305 32323 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.828s	user 0.018s	sys 0.001s
I20260709 12:26:22.822816 32323 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.824007 32249 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.828536 32234 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.829452 32234 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.806102 32266 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:22.833359 32269 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.010s
I20260709 12:26:22.833841 32266 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:22.807978 32429 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:22.841009 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92001us
W20260709 12:26:22.841949 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 987) took 1498 ms (client timeout 1546 ms). Trace:
W20260709 12:26:22.842344 26195 rpcz_store.cc:269] 0709 12:26:21.343934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.356501 (+ 12567us) service_pool.cc:224] Handling call
0709 12:26:22.841558 (+1485057us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54002b4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:22.841774 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889182080}
I20260709 12:26:22.848948 26196 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:22.852885 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:22.855158 26757 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:22.856743 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1416) took 1438 ms. Trace:
I20260709 12:26:22.857221 26784 rpcz_store.cc:276] 0709 12:26:21.418790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.550146 (+131356us) service_pool.cc:224] Handling call
0709 12:26:21.550599 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.241755 (+691156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.856175 (+614420us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400565754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:22.856536 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572515200}
I20260709 12:26:22.858314 32249 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:22.861392 32113 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:22.862010 32113 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:22.862468 32113 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:22.865512 32370 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:22.865950 32266 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:22.866604 32266 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:22.866505 32307 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:22.868330 32364 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:22.868635 32481 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:22.867967 32370 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.878458 26202 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:22.881431 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:22.882283 26047 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:22.886551 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116202us
I20260709 12:26:22.869189 32481 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:22.833931 32269 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.898012 32269 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.912925 26049 raft_consensus.cc:2368] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17.
I20260709 12:26:22.888504 32234 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:22.878062 32415 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.890781 32266 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.928417 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:22.934841 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:22.923318 32269 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.929719 32429 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.937655 32323 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:22.956303 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:22.962381 32266 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:22.964663 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:22.966797 32430 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:22.967237 32430 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:22.967638 32430 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:22.966604 32040 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:21.858011 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:22.976693 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:22.979604 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:22.984736 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:22.986676 26128 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:22.980789 32368 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:22.994508 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:22.950197 32211 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:22.997390 32490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.024s	sys 0.041s
W20260709 12:26:23.004436 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53813us
I20260709 12:26:23.007459 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:23.002732 32490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.025s	sys 0.041s
I20260709 12:26:23.008941 32234 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:23.009577 32234 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.010092 32234 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.011122 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:22.998394 32368 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:22.980506 32429 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:23.012655 32367 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.774s	user 0.016s	sys 0.002s
I20260709 12:26:23.017400 26129 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.012823 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:22.990582 32269 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.028820 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:23.033109 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 271735us
I20260709 12:26:23.033061 32040 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:23.033602 32323 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.034406 26094 leader_election.cc:400] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:26:23.017762 32368 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:23.030768 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.034111 32040 consensus_queue.cc:1243] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:23.040757 32269 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:23.049830 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 185304us
I20260709 12:26:23.058095 26202 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:23.045130 32266 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:22.997887 32481 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.077610 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:23.077795 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:23.060163 26123 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:23.032322 32367 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:23.088131 26196 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.093889 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.062238 32415 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.103394 32533 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:23.099994 32490 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:23.105026 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85040.8us
I20260709 12:26:23.105188 32490 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:23.105749 32490 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:23.107122 32266 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:23.107698 32330 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:23.108347 32330 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.109843 32266 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:26:23.107625 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.113793 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:23.110271 32266 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:23.141886 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.142442 32429 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.142438 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:23.150153 32539 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 12:26:23.154906 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.166733 32323 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:23.167373 32323 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:23.168646 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:23.169718 26201 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:23.170740 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 136570us
I20260709 12:26:23.172641 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.176908 32266 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.178404 32266 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.119414 32533 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:23.168799 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.198081 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.197917 32533 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.198551 32553 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:23.199331 32553 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.201023 32234 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:23.201510 32553 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:23.168587 32542 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:26:23.203691 32234 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:23.152868 32485 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.210041 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.222534 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.223249 26790 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.181705 32415 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:23.231786 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.209334 32211 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.240533 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.243652 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.245909 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.214366 32249 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:23.219146 32330 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:23.255673 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 76055.3us
I20260709 12:26:23.258713 26050 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:23.256187 26121 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:23.246961 32307 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.262859 32367 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.283663 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.284734 26127 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:26:23.284825 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.285727 26122 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:23.285645 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.286406 26127 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.286755 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.287180 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.290132 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.291553 26123 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.280652 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.287868 26127 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.296348 32485 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:23.297756 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108376us
W20260709 12:26:23.299091 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 204464us
I20260709 12:26:23.209374 32420 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 12:26:23.302249 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.287861 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.280280 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.318282 26125 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.319175 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.280885 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:23.320683 32307 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.321393 32307 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.322263 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.323077 26127 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.323599 32307 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.324266 26127 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:23.301199 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.325656 26120 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:23.316393 26124 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.324369 26052 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:23.325670 26122 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:23.247364 32533 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.266757 32330 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.303551 32249 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.366081 32249 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.365792 32330 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.366336 26790 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:23.241683 32323 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.361333 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:23.356098 32420 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:23.406280 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87198.1us
I20260709 12:26:23.405763 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:23.413952 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68569.4us
I20260709 12:26:23.423454 32527 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:23.424319 32527 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:23.425307 32556 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:23.427376 26120 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 11
I20260709 12:26:23.427805 26120 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:26:23.430321 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.431026 26127 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:23.428148 26120 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.416090 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.419113 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:23.418282 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:23.368770 32265 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.433154 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:23.436407 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:23.444463 32113 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.581s	user 0.014s	sys 0.000s
I20260709 12:26:23.444656 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.445003 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:23.451925 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.453145 26052 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:23.454463 32368 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.454821 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.455868 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:23.445955 32490 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:23.460418 32265 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.463358 32265 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.445003 32113 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.464900 32485 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.436432 32527 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:23.468578 26790 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:23.469594 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:23.470825 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:23.471518 26790 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:23.472680 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.473034 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:23.473821 26783 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:23.493343 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.474287 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:23.474841 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.494691 26047 raft_consensus.cc:2368] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17.
I20260709 12:26:23.474436 26050 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:23.475244 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:23.491344 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.491349 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:23.496894 26790 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.502600 26785 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:23.489384 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:23.477598 26784 raft_consensus.cc:1217] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[12.9-16.11]   Dedup: 12.9->[14.10-16.11]
I20260709 12:26:23.502954 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:23.506203 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.492950 26789 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:23.505604 26784 pending_rounds.cc:85] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 9
I20260709 12:26:23.509279 26784 pending_rounds.cc:107] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:26:23.509677 26784 raft_consensus.cc:2889] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.511299 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.512745 26787 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:23.460363 32558 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:23.481580 32546 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:23.481551 32330 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.483168 32113 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:23.459910 32430 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:23.548053 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.540762 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.472168 32533 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:23.550165 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47341.2us
I20260709 12:26:23.557576 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:23.558208 26122 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:23.558148 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:23.551111 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:23.553520 26128 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:23.517208 32564 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:23.548445 31962 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.003s
I20260709 12:26:23.563318 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.548003 32367 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:23.567436 26048 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.567968 26756 leader_election.cc:400] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:23.570497 26756 leader_election.cc:403] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:26:23.547827 32490 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:23.560357 32546 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:23.564570 32553 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:23.583089 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.564059 32558 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:23.597227 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56526.7us
W20260709 12:26:23.644600 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84263.4us
I20260709 12:26:23.582098 32368 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.647121 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.592061 32367 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:23.653417 26122 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:26:23.650300 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72877.1us
I20260709 12:26:23.569154 32564 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:23.601027 32429 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:23.661659 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:23.663586 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 106146us
I20260709 12:26:23.656590 26047 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:23.568292 31962 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:23.699991 26049 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:23.702878 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51371.6us
I20260709 12:26:23.703979 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:23.708500 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 159489us
I20260709 12:26:23.715875 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:23.599128 32533 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:23.735721 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74333.3us
I20260709 12:26:23.744118 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:23.756750 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.778061 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:23.778468 32429 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:23.780082 32368 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.780807 32368 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.782749 32481 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:23.783555 32481 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:23.784365 32481 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:23.783795 32368 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.720278 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:23.791361 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81804us
I20260709 12:26:23.793162 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.795100 32527 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:23.799245 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 95373.7us
I20260709 12:26:23.803063 32430 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.805334 32485 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.806054 32485 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.816433 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.820847 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.829165 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.843917 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.845041 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.846061 26196 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:23.834404 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:23.847379 32481 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.845057 26198 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:23.849779 32481 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.840282 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.859709 26195 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:23.841558 26202 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:23.842684 26199 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:23.836031 32485 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:23.863870 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:23.710350 32307 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:23.875846 26787 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:23.897006 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.897302 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:23.898226 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.901296 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:23.903100 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:23.903493 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.904017 26125 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.893500 32235 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:23.878984 32571 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:23.879704 32368 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:23.904199 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:23.916601 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 220862us
I20260709 12:26:23.919973 26125 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:23.922432 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:23.906375 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 402934us
I20260709 12:26:23.885062 32265 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:23.867547 32430 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:23.908663 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.905550 26203 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:23.954326 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 125279us
I20260709 12:26:23.952803 26785 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:23.956643 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:23.962808 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:23.892539 32429 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:23.966676 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:23.917270 32568 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 12:26:23.971118 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.973783 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.981813 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:23.983837 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 322241us
I20260709 12:26:23.985195 26127 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:26:23.918701 31962 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:23.920055 32564 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:23.992915 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:23.920094 32533 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:24.008340 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:23.996119 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:23.962988 32551 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
W20260709 12:26:24.014500 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108719us
I20260709 12:26:23.910866 32483 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:26:24.043221 26123 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:26:24.014700 32266 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.836s	user 0.009s	sys 0.007s
I20260709 12:26:23.916685 32285 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.327s
I20260709 12:26:23.916599 32367 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.954996 32558 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:23.889742 32546 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.059197 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:24.038033 32429 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:24.055699 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:24.068373 32559 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.069041 26194 raft_consensus.cc:2368] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:24.069226 32527 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:24.070915 32106 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.092s
W20260709 12:26:24.073275 32570 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:26:24.074244 31962 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:24.075255 31962 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:24.075639 31962 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:24.075807 26783 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.003s
I20260709 12:26:24.076627 31962 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.078287 32235 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.077807 26783 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:24.079879 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:24.080180 31962 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.081041 32582 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:24.081537 32582 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:24.050649 32564 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.081986 32582 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.079581 32235 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.084975 32235 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:24.088393 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 143783us
I20260709 12:26:24.089493 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:24.091863 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:24.092226 32567 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.430s	user 0.000s	sys 0.001s
I20260709 12:26:24.092880 32564 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.097204 32533 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:24.088270 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1436) took 1029 ms. Trace:
I20260709 12:26:24.057883 26129 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:26:24.060166 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97992.8us
I20260709 12:26:24.106391 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.099684 26783 rpcz_store.cc:276] 0709 12:26:23.058435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.073104 (+ 14669us) service_pool.cc:224] Handling call
0709 12:26:24.087339 (+1014235us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54003101d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:24.087866 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871489280}
I20260709 12:26:24.113639 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:24.111903 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.055377 32266 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.125773 26201 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.127440 32588 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:24.119046 32553 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 12:26:24.128568 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76503.2us
I20260709 12:26:24.129429 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:24.137295 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 155312us
I20260709 12:26:24.104458 32231 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:24.116430 32106 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:24.144634 32231 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:24.150153 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58419.4us
I20260709 12:26:24.160957 32234 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:24.162523 32493 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:24.163041 32493 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:24.165035 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:24.164029 32493 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.174188 32493 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.179625 32429 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:24.179746 32231 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:24.118392 32430 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 12:26:24.197286 26048 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.627s	user 0.006s	sys 0.005s
I20260709 12:26:24.198196 26048 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:24.149359 32588 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.151175 32106 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:24.165556 32285 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:24.215174 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:24.222818 32578 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:26:24.223946 32581 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:26:24.206055 32429 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:24.214040 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:24.226778 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 111419us
I20260709 12:26:24.232180 26019 leader_election.cc:400] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:24.232767 26019 leader_election.cc:403] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:26:24.237200 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 107876us
I20260709 12:26:24.212721 32559 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:24.229539 32429 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:24.249639 32191 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:24.250170 32191 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:24.250571 32191 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:24.252498 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49366.3us
W20260709 12:26:24.254488 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 142667us
I20260709 12:26:24.258759 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:24.236752 32546 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:24.259603 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:24.261538 26124 raft_consensus.cc:2368] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:24.283238 32106 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.258159 32285 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:24.290382 32429 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.298200 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:24.298672 32564 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.300935 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:24.304751 26201 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:24.306454 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:24.296564 32265 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:24.308663 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:24.309672 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:24.302928 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.778s (SENT) [suppressed 9 similar messages]
W20260709 12:26:24.318372 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.288033 32582 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.315120 32191 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.303712 32588 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:26:24.333079 32106 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.344645 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.345957 32551 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:24.347918 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.348278 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 136547us
W20260709 12:26:24.348798 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46976.6us
I20260709 12:26:24.320539 32546 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:24.352201 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.351997 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:24.324177 32429 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.347838 32191 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:24.364686 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:24.368779 32575 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.354s	user 0.002s	sys 0.002s
I20260709 12:26:24.435621 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.436794 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:24.438465 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:24.437942 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:24.437918 26787 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:24.447079 26048 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:24.327289 32604 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:24.436717 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:24.460757 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:24.467516 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.470757 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 101844us
I20260709 12:26:24.469442 26124 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:24.469772 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:24.371320 32490 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:24.472837 26046 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
W20260709 12:26:24.473439 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.470206 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:24.466632 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:24.476853 26047 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:24.372396 32285 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:24.475333 26121 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:24.456712 32368 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.471783 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:24.440917 32553 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:24.473645 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:24.477416 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:24.466624 32415 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:24.489055 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 124525us
I20260709 12:26:24.442116 32582 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:24.497740 26120 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:24.471467 26756 leader_election.cc:400] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17.
I20260709 12:26:24.481612 26051 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:24.467439 32605 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:24.476794 32493 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.536863 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88757.3us
I20260709 12:26:24.512735 32364 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:26:24.548704 32582 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:24.562482 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:24.549976 32559 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:24.563835 32533 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:24.564136 32559 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:24.564302 32533 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:24.564630 32559 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:24.565284 32533 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.565553 32559 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.566751 32533 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.566887 32559 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:24.567651 32606 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:26:24.550760 32429 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:24.568828 32429 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:24.569260 32429 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.569749 32583 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:24.570191 32583 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:24.570611 32583 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.513156 32490 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:24.559855 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.571894 32490 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.573257 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:24.574432 26756 leader_election.cc:400] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17.
I20260709 12:26:24.574697 26756 leader_election.cc:403] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:24.515937 32553 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.576499 32430 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:24.576915 32430 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:24.577719 32430 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.560784 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:24.510456 32546 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:24.582175 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 279285us
I20260709 12:26:24.553560 32285 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.578109 32553 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:24.601871 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.617204 32588 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17.
I20260709 12:26:24.618302 32598 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:24.619071 32598 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.620500 32582 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:24.620947 32589 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:24.621469 32589 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:24.621806 32598 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:24.621891 32589 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:24.624202 26123 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:24.511065 32415 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:24.663765 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64970.8us
I20260709 12:26:24.581453 32605 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.669642 32605 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:24.670609 32586 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:24.672355 32323 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:24.672971 32323 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.675355 32323 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:24.679701 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:24.681100 32596 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:24.681600 32596 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:24.682060 32596 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:24.687733 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 252202us
I20260709 12:26:24.703667 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.626119 32546 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:24.720392 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:24.727036 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 125293us
I20260709 12:26:24.511235 32368 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.663157 32415 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.735852 26784 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:24.737432 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:24.741109 26126 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:24.737164 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:24.740190 32588 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.744351 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118747us
I20260709 12:26:24.745786 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:24.745960 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.747310 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:24.747700 32249 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:24.749449 25948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 288) took 35238 ms (client timeout 15000 ms). Trace:
I20260709 12:26:24.750468 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:24.751129 26788 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.744521 26786 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:24.754884 26757 leader_election.cc:400] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:26:24.755302 26757 leader_election.cc:403] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:24.756747 32265 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:26:24.749986 26790 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:26:24.760581 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57005us
W20260709 12:26:24.760915 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.761519 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.766984 26788 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:24.625756 32493 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.768669 26123 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:24.769018 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.769192 32249 raft_consensus.cc:2764] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:24.744138 32588 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.799078 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:24.800078 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80591us
I20260709 12:26:24.771871 32364 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:26:24.778856 32589 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.839788 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:24.832784 26784 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:24.807799 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:24.831426 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:24.844422 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:24.836120 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 67208.3us
I20260709 12:26:24.846076 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:24.820842 32558 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:24.856330 32429 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:24.857900 32588 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:24.858999 32429 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:24.822252 32613 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:24.829351 32605 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:24.878026 26783 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:24.881778 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.788822 32604 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:24.869560 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:24.850453 32559 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:24.896566 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140258us
W20260709 12:26:24.893771 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91316.2us
I20260709 12:26:24.841290 32235 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:24.923832 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:24.891163 32323 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:24.939334 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 191140us
W20260709 12:26:24.749886 25948 rpcz_store.cc:269] 0709 12:25:49.561491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.570601 (+  9110us) service_pool.cc:224] Handling call
0709 12:26:24.749157 (+35178556us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:17.149565 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:17.150250 (+   685us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:17.150305 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:26:17.150327 (+    22us) tablet.cc:662] Decoding operations
0709 12:26:17.177118 (+ 26791us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:17.177269 (+   151us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:17.177297 (+    28us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:26:17.192721 (+ 15424us) tablet.cc:701] Row locks acquired
0709 12:26:17.192779 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:26:17.193967 (+  1188us) write_op.cc:270] Start()
0709 12:26:17.194155 (+   188us) write_op.cc:276] Timestamp: P: 1783599977192917 usec, L: 0
0709 12:26:17.194194 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:26:17.198325 (+  4131us) log.cc:844] Serialized 7651 byte log entry
0709 12:26:18.083980 (+885655us) op_driver.cc:464] REPLICATION: finished
0709 12:26:18.505175 (+421195us) write_op.cc:301] APPLY: starting
0709 12:26:18.505276 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:18.869188 (+363912us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:18.884664 (+ 15476us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:19.665861 (+781197us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:19.884593 (+218732us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:26:19.929691 (+ 45098us) write_op.cc:312] APPLY: finished
0709 12:26:20.081665 (+151974us) log.cc:844] Serialized 172 byte log entry
0709 12:26:20.373446 (+291781us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:20.395155 (+ 21709us) write_op.cc:454] Released schema lock
0709 12:26:20.413163 (+ 18008us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:20.521000 (+107837us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:20.523208 (+  2208us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1221012,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":392538,"apply.run_cpu_time_us":117799,"apply.run_wall_time_us":1988218,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":171998,"prepare.run_cpu_time_us":57857,"prepare.run_wall_time_us":465951,"raft.queue_time_us":187065,"raft.run_cpu_time_us":16061,"raft.run_wall_time_us":170855,"replication_time_us":889551,"spinlock_wait_cycles":1408,"thread_start_us":1022869,"threads_started":5,"wal-append.queue_time_us":823734,"wal-append.run_cpu_time_us":27594,"wal-append.run_wall_time_us":2435319}]]}
I20260709 12:26:24.976785 32616 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.977562 32616 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.901381 32559 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:24.878552 32558 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.896796 32617 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:24.930141 32527 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.936687 32608 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:24.990267 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 145953us
I20260709 12:26:24.943271 32323 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.931704 32615 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:24.855778 32546 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:24.940340 32285 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:24.983822 32559 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:25.029093 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 143138us
I20260709 12:26:25.037357 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:25.049824 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 270043us
I20260709 12:26:24.963397 32571 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 12:26:25.065922 32551 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:25.066684 32571 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:25.052336 32106 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:25.067126 32571 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:25.021847 32368 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.054520 32617 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:25.057205 32635 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:25.058317 26020 leader_election.cc:400] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:25.017845 32527 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.060830 32604 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.063468 32558 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.022351 32323 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.064586 32249 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:25.064939 32481 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:25.052588 31968 raft_consensus.cc:2764] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:26:25.065861 32601 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:26:25.079962 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54293.8us
I20260709 12:26:25.082100 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:25.089020 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 157362us
I20260709 12:26:25.093748 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.109202 26051 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.096567 26052 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:25.068094 32571 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.099189 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:25.103274 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 140168us
W20260709 12:26:25.104791 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83325us
I20260709 12:26:25.079712 32285 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:25.114249 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:25.123265 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 161409us
I20260709 12:26:25.130326 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:25.124260 32231 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:25.133739 32558 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:25.134446 32527 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:25.128034 32285 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:25.140846 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 161265us
I20260709 12:26:25.135603 32527 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:25.154928 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.156971 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.158006 26045 raft_consensus.cc:2368] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:25.154050 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.157691 26047 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:25.159819 26054 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:25.154695 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.161118 26050 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:25.130365 32113 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:26:25.147683 32231 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:25.154699 32571 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.179180 26051 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:25.180830 32637 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.181464 32637 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:25.185026 32631 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:26:25.185499 32583 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.614s	user 0.012s	sys 0.000s
I20260709 12:26:25.185946 32583 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:25.189203 32633 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:25.157687 32558 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:25.191334 32558 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.159559 32370 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.193552 32231 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.192344 32323 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.197286 26051 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:25.204988 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:25.207559 26203 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:25.209271 32628 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:25.213547 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:25.218218 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:25.226742 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52363.8us
I20260709 12:26:25.228726 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.229488 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.229566 26048 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:25.216743 32617 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:25.222239 32615 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:25.221267 32191 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.252978 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:25.266443 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66464.1us
I20260709 12:26:25.210481 32628 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.268224 32625 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.272997 32323 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.278211 32589 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:25.289330 26203 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.291224 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:25.326467 32415 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.006s
I20260709 12:26:25.309787 26047 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:26:25.314052 32596 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.631s	user 0.014s	sys 0.000s
W20260709 12:26:25.317400 32636 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.094s	user 0.000s	sys 0.002s
I20260709 12:26:25.254254 32191 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.253304 32630 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.246760 32615 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:25.321290 32559 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.341418 26122 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:25.291419 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:25.334582 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.343362 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.343847 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:25.344530 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.344635 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:25.345261 26128 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.348395 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.325820 32616 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.330055 32583 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.359956 26125 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:25.362347 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.367178 26202 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.363236 26121 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:25.368299 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.338390 32596 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.375836 26787 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.378073 32628 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.337459 32625 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.366386 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:25.342923 32615 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:25.381955 32625 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:25.384199 32559 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:25.384420 32642 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
I20260709 12:26:25.384816 32106 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.385126 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.385356 32106 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.385841 26120 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:25.338374 32415 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.387503 32106 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:25.372850 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 132783us
I20260709 12:26:25.353367 32603 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:25.405130 26127 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:25.407919 32191 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:25.413153 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:25.417572 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:25.355731 32323 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:25.490888 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77959.4us
W20260709 12:26:25.490906 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77887.9us
W20260709 12:26:25.491962 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 74890.2us
I20260709 12:26:25.497831 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:25.495576 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:25.378302 32596 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:25.492472 32191 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:25.506913 32323 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.510056 32564 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:25.388798 32415 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.521136 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:25.527987 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 179300us
I20260709 12:26:25.532272 26120 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 17 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:26:25.496970 32626 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.541841 26047 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.546049 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.554388 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.555258 26053 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:25.555361 26048 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:25.521211 32630 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:25.554498 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.560003 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:25.567791 26054 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:25.500403 32616 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:26:25.577004 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.500999 32589 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:26:25.536614 32626 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.537563 32646 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:25.634588 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.635558 26046 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:25.652024 32527 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:25.652545 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.654019 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:25.631608 26094 connection.cc:256] RPC call timeout handler was delayed by -2.6358945409997432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:26:25.655721 26094 connection.cc:256] RPC call timeout handler was delayed by -2.1649398629997449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 12:26:25.656131 26094 connection.cc:256] RPC call timeout handler was delayed by -1.8957513059999656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
W20260709 12:26:25.656482 26094 connection.cc:256] RPC call timeout handler was delayed by -1.847889409000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 12:26:25.656869 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7906482509997659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 12:26:25.657244 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6845345629999429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 12:26:25.657557 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5027345630001037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 12:26:25.657863 26094 connection.cc:256] RPC call timeout handler was delayed by -1.4154345629999625s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 12:26:25.658193 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3695345629998883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
I20260709 12:26:25.527311 32191 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.654989 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.668668 26048 raft_consensus.cc:2410] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:25.633229 32626 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.632511 32646 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.675948 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:25.628984 32583 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:25.649291 32559 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:25.565680 32630 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:25.652575 32527 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:25.672950 32646 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.694319 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.686952 32632 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
W20260709 12:26:25.712930 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 144972us
I20260709 12:26:25.697397 32628 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:25.686612 32630 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:25.684492 32583 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.740661 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.741952 32583 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:25.743067 32653 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:25.743901 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.744727 26120 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:25.745817 32527 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.747563 32527 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:25.750322 32654 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.346s	user 0.006s	sys 0.000s
I20260709 12:26:25.750952 32481 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.751495 32481 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.753504 32637 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.753750 32481 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:25.755246 26046 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:25.757272 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:25.724682 32651 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:26:25.725099 32563 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:25.759786 32563 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.761973 32563 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.725785 32323 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.735581 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.736763 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.763810 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.764945 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.737057 32191 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:25.708118 32415 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.766149 32415 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.691388 32559 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.770143 32559 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:25.770871 32415 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.721966 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:25.772181 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54469.6us
I20260709 12:26:25.773774 32646 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:25.761749 32628 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:25.775437 32628 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:25.775799 32628 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:25.776773 32628 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.778193 32628 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.779343 26122 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.780722 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:25.758755 32493 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:25.781270 32571 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.003s
I20260709 12:26:25.782908 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.783514 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:25.732788 32650 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:25.774693 32249 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:25.784512 32249 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:25.784941 32249 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:25.786932 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.787758 26782 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:25.788705 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:25.790167 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.773741 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:25.797281 32370 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:25.798754 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:25.799849 32647 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:25.800879 26128 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:25.801383 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.801699 26127 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:26:25.802047 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.803041 26128 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:25.772431 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.804109 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.806205 32646 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:25.807859 32106 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:25.808532 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140187us
I20260709 12:26:25.816632 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:25.817494 32551 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:25.818126 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:25.818362 32551 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:25.810309 32649 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.435s	user 0.000s	sys 0.003s
I20260709 12:26:25.818720 32551 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.819234 32551 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.810942 32430 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:25.815815 32645 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.439s	user 0.002s	sys 0.000s
I20260709 12:26:25.755959 32615 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.822714 26048 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:25.802045 26047 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:25.832561 26048 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:26:25.833011 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.835448 26047 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:25.846303 26049 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [16.9-17.10]
I20260709 12:26:25.848997 26046 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:25.827775 32481 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:25.850191 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:25.850348 32481 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:25.850883 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:25.851358 32481 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:25.852633 32481 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.852957 32285 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:25.854019 32234 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:25.854477 32234 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:25.855106 32231 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:25.855743 32643 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:25.851799 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 736177us
W20260709 12:26:25.854147 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.855360 32234 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:25.864805 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150765us
I20260709 12:26:25.865600 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:25.869429 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:25.871376 32598 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:25.871618 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:25.871907 32234 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:25.872303 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.843885 32429 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.843703 32604 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.882417 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:25.886135 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:25.882416 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64364.8us
I20260709 12:26:25.887223 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:25.995219 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:25.997793 26049 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:25.895613 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.999971 26786 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:25.892714 26785 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
W20260709 12:26:26.007436 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.006471 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:25.890327 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.889875 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:25.889528 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:25.894170 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.014850 26202 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:25.893105 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:25.994532 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89150.9us
I20260709 12:26:26.001775 26785 pending_rounds.cc:85] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 12
I20260709 12:26:26.021085 26785 pending_rounds.cc:107] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:26:26.022037 26785 raft_consensus.cc:2889] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:26.022667 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 135512us
I20260709 12:26:26.023428 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:25.887223 32231 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.027621 26201 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:25.881508 32598 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:26.028337 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:25.888954 32637 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:25.889349 32493 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:26.055739 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.055889 32493 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.058343 32493 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:26.005635 32323 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:26:26.025321 32628 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:26.069550 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.072295 32558 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 12:26:26.073124 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.992569 32604 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.076792 32604 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:26.017968 32589 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.080665 32589 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.083305 32589 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:25.885643 32626 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.085647 32626 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.085767 32415 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:26.086609 32415 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.087339 32628 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:26.088299 32598 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.088658 32628 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.001880 32616 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:26:26.046296 32564 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.091686 32616 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:26.091566 32564 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.092105 32616 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:26.092970 32616 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.093585 32564 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:26.000737 32559 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:26:26.095561 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.096717 26048 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:26.097916 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:26.098723 26202 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.100028 26783 raft_consensus.cc:2376] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15. Re-sending same reply.
I20260709 12:26:26.103037 32637 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:26.114393 26126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.042s	sys 0.074s
W20260709 12:26:26.115052 26126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.042s	sys 0.075s
I20260709 12:26:26.095561 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.117544 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:26.118341 26124 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:26.120623 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 275060us
I20260709 12:26:25.875880 32113 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.173s
I20260709 12:26:26.125823 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:26.126515 26120 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:25.875249 32571 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:26.044480 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.130789 26790 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:26.126838 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:26.045367 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:25.992769 32649 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:26:26.045400 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.048857 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.050015 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.051054 26196 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:26.051144 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.051700 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.134667 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:26.052423 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:26.052639 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.053095 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:25.891215 32429 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.136258 26199 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:26.137884 32429 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.053879 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.138793 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.139308 26782 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:26.140175 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:26.140786 26783 raft_consensus.cc:2376] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:26:26.127604 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:26.143694 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.144466 26201 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:25.893667 32249 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.148952 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.152884 26052 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:26.153128 26048 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:26:26.155683 26048 pending_rounds.cc:85] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 7
I20260709 12:26:26.155963 26048 pending_rounds.cc:107] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:26:26.156388 26048 raft_consensus.cc:2889] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:26.155798 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 132435us
I20260709 12:26:26.137416 25945 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 15, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 15 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:26.159252 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:26.159847 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.159965 26045 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.114444 32616 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.164785 26202 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:26.165421 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:26.166276 26125 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:26.169338 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:26.170938 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.171689 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:26.126591 32638 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:26.079798 32630 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.142313 32558 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.193295 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:26.104099 32637 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.159010 32231 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:26.129271 32551 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.201006 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.196844 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:26.210116 26756 leader_election.cc:400] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16.
I20260709 12:26:26.211788 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:26.198709 32490 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.158114 32571 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:26.213135 32490 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.213928 32571 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.215692 32490 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:26.211422 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.219127 25945 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 13, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 13 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:26.220275 32641 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:26.221577 32641 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:26.222023 32641 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:26.221443 25945 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:26.222932 32641 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.223497 32415 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.224603 32490 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.225175 32490 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:26.226624 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.219030 32191 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:26.228780 32191 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:26.229957 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.229997 32191 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.231582 26196 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:26.232800 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.228052 32641 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:26.236951 32490 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:26.239056 26201 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:26.239471 32490 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.240034 32490 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.242280 32490 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:26.242597 26050 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:26.244503 32615 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:26.245329 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.267083 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:26.267287 26198 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:26.268206 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:26.268446 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:26.246127 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.246939 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.231505 26121 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:26.247442 32630 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:26.248106 26202 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:26.287474 32323 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:26.148362 26046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:26.317203 26046 raft_consensus.cc:606] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.169230 32330 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:26:26.224151 26195 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:26.256378 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.257520 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:26.333003 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75575us
I20260709 12:26:26.185961 32429 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 12:26:26.258986 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47678.3us
I20260709 12:26:26.336835 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.263979 26045 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:26.269351 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:26.274269 32533 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:26.275266 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.342429 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.279417 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.243098 25945 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:26.292922 26784 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:26.302290 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.270113 32648 raft_consensus.cc:2764] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
W20260709 12:26:26.303800 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 146382us
I20260709 12:26:26.305173 32415 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:26.202561 32558 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.319722 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:26.199805 32638 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.332331 26052 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:26.337006 26121 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.265031 26054 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 16: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:26.345695 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.310003 32626 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:26.349200 32113 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:26.303150 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 183986us
W20260709 12:26:26.348286 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68953.8us
I20260709 12:26:26.312112 32603 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.342594 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:26.261804 26122 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:26.359993 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:26.362327 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.349205 32533 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.364439 32670 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:26:26.365783 26756 leader_election.cc:400] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 14 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 14. Current term is 15.
I20260709 12:26:26.346812 26049 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
W20260709 12:26:26.439520 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 73904.5us
I20260709 12:26:26.373252 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:26.441430 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68318.9us
I20260709 12:26:26.338186 32639 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.431442 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71511.4us
I20260709 12:26:26.444756 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:26.445184 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:26.431931 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55296.5us
W20260709 12:26:26.365831 32665 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:26:26.430765 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.450389 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.435642 26126 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:26.451741 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.356958 32563 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.445719 26128 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:26.345121 32430 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:26.355258 32644 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.366254 32626 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:26.429818 32643 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:26.357252 32559 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:26.436928 32583 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:26.432710 32113 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:26.468612 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:26.436661 32605 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.454301 32490 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:26.495808 32605 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.496433 32490 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.496913 32490 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.376971 32415 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.500087 32605 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.502511 32415 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.458582 32640 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:26.503616 32640 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.504163 32640 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.462836 32249 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.374660 32628 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 12:26:26.508565 26124 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:26.473367 32626 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.434659 32603 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:26.508282 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75643.2us
I20260709 12:26:26.473047 32235 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.524144 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:26.456033 32367 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:26:26.484712 32643 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:26.482821 32647 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:26.493062 32583 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:26.530275 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89514.7us
I20260709 12:26:26.493551 32113 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.509752 32323 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:26.532624 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87518.7us
I20260709 12:26:26.455624 32639 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.548641 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:26.545063 26196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:26.467020 32563 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:26.489939 32669 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:26:26.549690 26196 raft_consensus.cc:606] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:26.556826 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45125us
I20260709 12:26:26.531260 32643 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.544734 32583 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.517666 32564 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:26.571823 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:26.558255 32676 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.001s
I20260709 12:26:26.583635 26789 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:26.582962 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2582) took 1232 ms. Trace:
I20260709 12:26:26.588776 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.590718 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:26.591387 26046 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.592312 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.594179 26050 raft_consensus.cc:2410] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:26.585927 26129 rpcz_store.cc:276] 0709 12:26:25.344147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.361241 (+ 17094us) service_pool.cc:224] Handling call
0709 12:26:25.361608 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.498832 (+137224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.498863 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.521517 (+ 22654us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.761724 (+240207us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:26:25.885170 (+123446us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:25.997354 (+112184us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.083828 (+ 86474us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800474d00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:26:26.084004 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.084078 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.575877 (+491799us) raft_consensus.cc:1685] finished
0709 12:26:26.575941 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.576835 (+   894us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137324,"prepare.queue_time_us":118593,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":9658,"raft.queue_time_us":219388,"raft.run_cpu_time_us":5212,"raft.run_wall_time_us":249919,"spinlock_wait_cycles":33112832,"thread_start_us":256179,"threads_started":1,"wal-append.queue_time_us":603349}
W20260709 12:26:26.596556 26167 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.778s (SENT)
I20260709 12:26:26.597122 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:26.560475 32551 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.572781 32546 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.602218 26045 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:26.612618 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:26.615675 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.619968 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:26.574646 32647 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 12:26:26.609017 32481 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.003s
I20260709 12:26:26.586225 32598 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:26.655174 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2106) took 1021 ms. Trace:
I20260709 12:26:26.656039 26051 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:26.616534 26200 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
W20260709 12:26:26.674091 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.682266 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 85732.6us
I20260709 12:26:26.703886 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:26.705643 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.699599 26054 rpcz_store.cc:276] 0709 12:26:25.633328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.735295 (+101967us) service_pool.cc:224] Handling call
0709 12:26:26.654873 (+919578us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54002d0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:26.655090 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027953920}
I20260709 12:26:26.709915 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.710729 26200 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:26.726907 26046 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:26.614868 32683 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:26.750504 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:26.750952 26050 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:26.751384 26203 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:26.740125 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:26.745011 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:26.701128 32608 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:26.782305 26053 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:26.786607 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.700899 32632 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:26.721964 32533 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:26.747054 32598 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:26.791332 32673 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 12:26:26.746330 32481 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
W20260709 12:26:26.876719 32672 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 12:26:26.790663 32638 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:26.791929 32615 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:26.866492 32683 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:26.872364 32546 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:26.909904 32674 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
W20260709 12:26:26.911118 32688 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:26:26.911754 32661 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:26.912115 32666 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
W20260709 12:26:26.912889 32679 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 12:26:26.912958 32682 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
W20260709 12:26:26.914229 32681 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
W20260709 12:26:26.915069 32678 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.332s	user 0.000s	sys 0.000s
I20260709 12:26:26.917013 32639 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:26.916422 25945 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 13 to 14, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:26.923187 32640 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.930179 32430 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.932725 32598 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:26.933310 32603 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:26.934016 32563 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:26.957756 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.874334 32632 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.007910 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1408) took 1142 ms. Trace:
I20260709 12:26:27.009142 32632 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.008378 26788 rpcz_store.cc:276] 0709 12:26:25.865523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.865665 (+   142us) service_pool.cc:224] Handling call
0709 12:26:25.891634 (+ 25969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.892073 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.892125 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.002326 (+110201us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.182362 (+180036us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:26.182582 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.182693 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.007684 (+824991us) raft_consensus.cc:1685] finished
0709 12:26:27.007705 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.007849 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2130,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":8660,"thread_start_us":113761,"threads_started":1,"wal-append.queue_time_us":403601}
I20260709 12:26:27.001937 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.004590 32430 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.010093 25945 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:26.940711 32615 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.996784 32490 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.012429 32430 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.012660 32615 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.016803 32490 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:26.975133 32626 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.018074 26787 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:26.998389 32694 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.022902 32694 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.025280 32694 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.025388 32615 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:27.016855 32641 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.782s	user 0.007s	sys 0.005s
I20260709 12:26:27.027773 32615 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.012210 25945 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 11 to 14, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 14 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:26:27.032642 32683 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:27.066138 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.004s
I20260709 12:26:27.066368 26195 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:27.066685 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.008s	sys 0.004s
I20260709 12:26:27.070833 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:26.989539 32546 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.072156 26047 raft_consensus.cc:606] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.073647 26047 raft_consensus.cc:614] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:27.063628 32626 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.067430 32693 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.031576 25945 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 13 to 15, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 15 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:27.078604 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:27.067481 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 525136us
I20260709 12:26:27.078706 26048 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: [15.8-16.9]
I20260709 12:26:27.070847 32697 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.087337 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.088269 26054 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:27.096812 26054 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:27.101348 32546 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.102775 25945 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 11 to 15, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 15 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:26:27.098295 32643 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.512s	user 0.004s	sys 0.004s
I20260709 12:26:27.106639 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:27.075700 32615 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.118758 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:26:27.099864 32616 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.929s	user 0.008s	sys 0.005s
I20260709 12:26:27.086606 32637 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:27.119611 32696 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.133821 32639 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:26:27.147913 32698 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.153276 32698 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.153627 32698 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.148519 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.157645 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:26:27.149710 26755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.005s	sys 0.015s
I20260709 12:26:27.162981 26046 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:26:27.163018 26755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.005s	sys 0.015s
I20260709 12:26:27.166605 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.116088 32641 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.151155 32639 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:27.171809 26052 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:27.171383 32639 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:27.150045 32589 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:27.162566 32113 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.004s
I20260709 12:26:27.083606 32693 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.152673 32637 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.149602 32696 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.181286 32696 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.168220 32692 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:27.170843 32231 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.179919 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.170002 32527 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:27.170204 32641 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.173907 32589 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:27.211639 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.203037 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.212848 26046 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.170442 32626 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:26:27.207749 32616 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.208912 32637 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.210392 32691 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.135360 32615 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.215620 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:27.183259 32583 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.000s
I20260709 12:26:27.171056 32647 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:26:27.132493 32697 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.179734 25945 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:27.117203 32643 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.184695 32692 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:27.242889 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.003s	sys 0.014s
I20260709 12:26:27.169843 32685 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:26:27.201331 32231 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.252482 26121 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:27.178673 32113 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.259827 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:27.214361 32641 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:27.263206 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.003s	sys 0.015s
I20260709 12:26:27.194491 32699 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.292325 32693 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.241591 32527 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:27.264678 32563 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:27.263857 32640 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.297319 32527 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.297998 32698 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.300457 32699 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:27.273754 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 661181us
I20260709 12:26:27.257822 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.289812 26121 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:27.306761 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.304423 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.306229 26046 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:27.307574 26047 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:27.309592 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.310197 26047 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.243117 32683 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.313361 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55627.1us
I20260709 12:26:27.254185 32615 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.316381 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.254185 32697 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.315909 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.265636 32692 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.314746 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.317911 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.304044 32231 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.319567 26046 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:27.319711 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.315953 26121 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.319103 26127 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:27.324337 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.320432 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.306960 32490 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:27.264698 32626 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:27.320895 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.321161 26789 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.328801 26120 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:27.318245 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.326823 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.327109 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.330855 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.329727 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.331463 26124 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:27.332929 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.264796 32616 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:26:27.316412 32693 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.337829 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.334774 32615 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.325551 32235 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:27.340618 32235 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:27.318435 32683 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.334849 32697 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:27.335367 32699 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.344370 32708 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:27.344852 32708 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:27.345860 32708 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.315400 32670 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:27.244567 32691 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.252477 32583 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.356153 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.374254 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:27.370469 26047 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:27.328801 32490 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.378043 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:27.379166 26787 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:27.379293 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.328440 32626 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:27.386864 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.387713 26126 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:27.387890 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:27.381018 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:27.388841 26787 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:27.389039 26047 raft_consensus.cc:606] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:27.346486 32644 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.880s	user 0.011s	sys 0.002s
I20260709 12:26:27.389830 26047 raft_consensus.cc:614] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:27.381562 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:27.352094 32671 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.889s	user 0.004s	sys 0.013s
I20260709 12:26:27.376468 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:27.390235 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:27.351635 32231 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:27.353305 32667 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.891s	user 0.010s	sys 0.008s
I20260709 12:26:27.347357 32708 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.352407 32709 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.337862 32605 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:26:27.428926 32231 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.407140 32693 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.429809 32643 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.433966 32693 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.423300 32709 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.422809 32711 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:27.426965 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3513) took 1741 ms. Trace:
I20260709 12:26:27.435113 26785 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:27.405440 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:27.409839 32696 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.436261 26790 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:27.386117 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:27.437423 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51398.5us
I20260709 12:26:27.438081 32696 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.438172 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2132) took 1412 ms. Trace:
I20260709 12:26:27.438419 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.347385 32698 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.439383 32533 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.381244 32490 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:27.440548 32626 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.435156 26123 rpcz_store.cc:276] 0709 12:26:25.685522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.685727 (+   205us) service_pool.cc:224] Handling call
0709 12:26:25.686166 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.693314 (+  7148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.693361 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.704168 (+ 10807us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.044653 (+340485us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:26.044709 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.044791 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.352531 (+1307740us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:27.426624 (+ 74093us) raft_consensus.cc:1685] finished
0709 12:26:27.426648 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.426899 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3067,"raft.queue_time_us":6050,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9242,"spinlock_wait_cycles":6894336,"thread_start_us":267968,"threads_started":1,"wal-append.queue_time_us":641235}
I20260709 12:26:27.438776 26194 rpcz_store.cc:276] 0709 12:26:26.025935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.026065 (+   130us) service_pool.cc:224] Handling call
0709 12:26:26.026397 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.049321 (+ 22924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.049354 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.049845 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.200801 (+150956us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:26.200861 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.200940 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.351306 (+1150366us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:27.437887 (+ 86581us) raft_consensus.cc:1685] finished
0709 12:26:27.437908 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.438115 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43311,"raft.queue_time_us":2989,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":5444,"spinlock_wait_cycles":7667456,"thread_start_us":150558,"threads_started":1,"wal-append.queue_time_us":295893}
I20260709 12:26:27.434453 32693 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.444253 32691 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.445150 32706 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:27.445402 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:27.445825 32706 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.446413 32533 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:27.446882 32533 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:27.447290 32533 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.425065 32113 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.435057 32711 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.449961 32711 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.347178 25945 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 3 to 18, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 18 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:27.408432 32704 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.438931 32696 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.406404 26124 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.389766 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:27.401042 32644 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.461275 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.462061 26126 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.464298 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.451709 32696 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.457966 32711 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.423270 32710 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.451719 25945 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:27.477015 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 122391us
I20260709 12:26:27.476696 32641 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:27.464893 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57339.6us
I20260709 12:26:27.474856 32696 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.477622 32711 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.491298 26120 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:27.491766 26120 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:27.500586 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:27.501309 26787 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.501680 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.502431 26125 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:27.503600 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.504197 26194 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:27.505121 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.505159 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.505905 26196 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.505961 26129 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:27.506973 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.508216 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:27.495013 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:27.511969 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:27.478770 32583 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.516809 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.478089 32706 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.518137 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:27.518735 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.524385 26790 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.522018 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:27.522192 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.526903 26199 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:26:27.513792 26120 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.532364 26120 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.542798 26202 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:27.549861 32696 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.517930 32191 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.554207 32191 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.525856 32691 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:27.556748 32691 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.531214 32698 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.557278 32693 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.557843 32698 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.558357 32698 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.557253 32691 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:27.569324 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1096) took 1792 ms (client timeout 2153 ms). Trace:
W20260709 12:26:27.570288 26122 rpcz_store.cc:269] 0709 12:26:25.777260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.780453 (+  3193us) service_pool.cc:224] Handling call
0709 12:26:27.569048 (+1788595us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:27.569227 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89927424}
I20260709 12:26:27.559602 32693 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.580981 32558 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.581364 32691 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.582648 32323 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:27.582273 25945 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:27.583262 32323 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:27.583717 32691 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.590970 32696 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.593313 32711 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.593755 32191 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.604188 32527 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:27.607568 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.604966 32527 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.610734 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.611747 26199 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.527945 32715 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.621636 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:27.623150 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:27.625499 26194 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:27.633690 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.630820 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.648989 32709 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.649117 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1018) took 1578 ms. Trace:
I20260709 12:26:27.596606 32323 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.654011 26196 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:27.657852 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.631479 32191 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.646152 32604 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:27.626327 32715 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.653641 26197 rpcz_store.cc:276] 0709 12:26:26.071211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.143852 (+ 72641us) service_pool.cc:224] Handling call
0709 12:26:26.166734 (+ 22882us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:26.228474 (+ 61740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.228508 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:26.229160 (+   652us) log.cc:844] Serialized 52 byte log entry
0709 12:26:26.464213 (+235053us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:26.541869 (+ 77656us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.542067 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.542147 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.934794 (+392647us) spinlock_profiling.cc:243] Waited 9.13 ms on lock 0x7b180034a080. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1
0709 12:26:27.567397 (+632603us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b180034a080. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:27.648833 (+ 81436us) raft_consensus.cc:1685] finished
0709 12:26:27.648863 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.649048 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74173,"prepare.queue_time_us":81633,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":65556,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":3567,"spinlock_wait_cycles":26407424,"thread_start_us":309693,"threads_started":2,"wal-append.queue_time_us":321848}
I20260709 12:26:27.665650 32191 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:27.678606 26784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1462) took 1895 ms (client timeout 2089 ms). Trace:
W20260709 12:26:27.679049 26784 rpcz_store.cc:269] 0709 12:26:25.783157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.783286 (+   129us) service_pool.cc:224] Handling call
0709 12:26:27.678397 (+1895111us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:27.678498 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1046084352}
I20260709 12:26:27.679780 32191 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.667407 32604 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:27.670619 26123 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:27.664201 32698 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.632033 32644 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.697392 32644 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.700506 32430 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.701131 32430 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.702371 32641 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.703754 32430 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.705340 32630 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:27.705544 26123 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:26:27.705801 32630 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:27.647889 26790 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:27.688282 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:27.688887 26199 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
W20260709 12:26:27.684130 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.709853 26784 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.611023 32533 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.710702 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.712523 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:27.713016 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 323312us
I20260709 12:26:27.713676 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.713793 26126 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:27.711606 32191 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.716684 32638 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:27.717306 32638 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:27.717887 32605 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:27.718259 32638 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.720355 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.721074 32598 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:27.721693 32533 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.722193 32713 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:27.722594 32713 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:27.723234 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.723457 32713 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.722793 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.725327 26199 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:27.726284 26789 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.673499 32715 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:27.694530 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 63817.5us
I20260709 12:26:27.695528 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:27.731616 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 108606us
I20260709 12:26:27.732640 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.611194 32692 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.685180 32527 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:27.731992 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 211245us
W20260709 12:26:27.736577 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80612.9us
I20260709 12:26:27.739995 32596 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:27.740495 32706 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:27.740794 32596 consensus_queue.cc:1243] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:27.741369 32644 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.742578 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.743726 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:27.711711 32490 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.718297 32605 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.745721 32605 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.749425 32690 raft_consensus.cc:993] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:26:27.749986 32690 raft_consensus.cc:1081] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:26:27.750015 32639 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.728943 32648 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:26:27.751266 32639 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.738463 32694 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:27.752050 26129 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:27.754096 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.754902 26128 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.757072 32235 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.758378 32235 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.761686 32615 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:27.761961 26121 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.762614 32615 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.763180 32615 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:27.766419 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.768061 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.775750 32589 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.777546 32589 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:27.784032 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.785004 26050 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.787621 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.788555 32694 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:27.789629 32694 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:27.790100 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57540.1us
I20260709 12:26:27.790892 32231 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:27.791390 26051 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.791669 32231 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:27.792840 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.793651 26124 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:27.793404 32231 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.793541 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.794518 26051 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.797148 32604 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.798624 32604 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:27.794533 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.804124 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.636477 25945 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 12, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:27.808384 32323 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.812558 32582 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:27.813221 32582 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:27.813501 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.814093 32692 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.814363 32582 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.816329 32582 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.816769 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.815076 26784 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:27.817477 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:27.818811 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.820272 26195 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:27.824419 32713 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.827847 32683 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:27.829023 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.719869 32630 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.737075 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.833177 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.880072 26050 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:27.880405 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 143390us
I20260709 12:26:27.807170 25945 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 8 to 17, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 17 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:27.840742 32626 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:27.841523 26199 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:27.737550 32698 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:27.893357 26197 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.845045 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.845652 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.846424 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.895109 26054 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:27.895737 26048 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.908593 26053 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:27.853655 32490 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:27.812525 26128 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:27.856957 32583 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.857769 32697 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:27.858474 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 117494us
I20260709 12:26:27.861822 32370 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:27.862571 32648 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:27.864845 26051 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:27.865839 32605 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.873840 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.936535 26047 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:27.874398 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 143038us
W20260709 12:26:27.877159 32551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.125s	user 0.005s	sys 0.009s
I20260709 12:26:27.937363 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.937407 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:27.938089 26053 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.837769 26120 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:27.882050 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.883499 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.886996 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:27.887144 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.939498 26196 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:27.939781 26123 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:27.847668 26052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be17917578e1415f98071364583240c1"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:27.940798 26052 raft_consensus.cc:606] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:27.896489 32630 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:27.897010 32429 consensus_queue.cc:1175] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 17 status { last_received { term: 17 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 17 index: 9 } }
I20260709 12:26:27.897922 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.947036 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.947458 26054 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:27.943084 26048 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:27.948798 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:27.948869 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:27.899713 32710 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:27.880900 32708 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:26:27.888056 32615 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.891050 32481 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:27.905109 26049 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.954335 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.905673 25945 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:26:27.956688 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.958468 32643 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:27.959131 32643 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:27.944329 32551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.199s	user 0.005s	sys 0.010s
I20260709 12:26:27.923127 26194 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.934571 26121 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:27.879956 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.972065 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2589) took 1728 ms. Trace:
W20260709 12:26:27.941138 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.935205 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.942432 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.943670 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.941180 26052 raft_consensus.cc:614] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:27.951216 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.972681 26121 rpcz_store.cc:276] 0709 12:26:26.243478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.465613 (+222135us) service_pool.cc:224] Handling call
0709 12:26:26.489479 (+ 23866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.252347 (+762868us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b54002a1aa0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:26:27.761731 (+509384us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b58008429d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:26:27.971799 (+210068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.971999 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152069760}
I20260709 12:26:27.975144 26199 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
W20260709 12:26:27.951267 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69340.9us
I20260709 12:26:27.951982 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:27.948060 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:27.976490 26195 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.976766 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.921039 32527 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:26:27.977900 32527 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:27.978552 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.922053 26050 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:27.981025 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:27.983903 32699 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.985077 32685 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:27.985718 32685 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.986603 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:27.987836 32683 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:27.962852 32669 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:27.988409 32683 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:27.963212 32608 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:27.961591 32481 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:27.890636 32715 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.964342 32113 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.021718 26049 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:27.964824 26047 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:27.965915 32615 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:27.966400 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.966881 26046 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20.
I20260709 12:26:27.967533 26786 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:27.968217 32708 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:27.969388 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:28.025002 26198 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.974382 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:27.931461 32648 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:27.984907 32699 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:27.963296 32640 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:27.991084 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:27.943372 32697 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:27.973688 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:27.991142 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:27.946513 32490 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.031332 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.041016 26199 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:28.033955 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.027535 26126 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:26:28.023944 32708 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:28.047430 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.048252 26788 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:27.988499 32669 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:27.997476 32249 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:26:28.028626 32608 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:28.037508 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 64016.7us
I20260709 12:26:28.022941 32583 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:28.023432 32370 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.039067 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.040781 32691 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.052986 32598 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:27.963908 32493 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:28.066145 26203 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:28.069505 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.036007 32713 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.042019 32234 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.032425 32367 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:28.054461 32669 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.089941 26195 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:28.058785 32685 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:28.030562 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.047482 32490 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.031395 32113 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:27.966903 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.075431 32323 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:26:28.136129 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.155586 26120 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.102596 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.084987 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:28.146255 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.117828 26123 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:26:28.088848 32729 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:26:28.075402 32644 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:28.110736 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:28.132625 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.153314 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.153687 26048 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:28.154294 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:28.158670 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 73814.7us
I20260709 12:26:28.163551 26126 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:28.165475 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:28.165908 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55317.5us
I20260709 12:26:28.166455 26049 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.169031 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.170529 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.171738 26201 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:28.131824 26121 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.159544 26124 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:28.075342 32546 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:26:28.086162 32691 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.074105 32583 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.129467 32370 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:28.134479 32638 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.151594 32694 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.183084 26791 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:26:28.153013 32640 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:28.155365 32709 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:28.192178 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.070149 32648 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.169557 26129 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:26:28.225332 26129 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:28.225864 26129 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:28.162334 32527 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.120568 32670 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:28.206338 26791 pending_rounds.cc:85] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 15
I20260709 12:26:28.235776 26791 pending_rounds.cc:107] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:26:28.236166 26791 raft_consensus.cc:2889] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:28.204205 32684 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:28.180162 32723 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:28.185478 32615 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:26:28.172704 32598 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:28.186648 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.181840 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:28.187211 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.099521 32708 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.185750 32589 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:28.216782 32113 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.098620 32683 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.218868 32691 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:28.220564 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.285776 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:28.286597 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.226912 26124 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:28.099722 32367 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.183560 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:28.293921 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 110432us
W20260709 12:26:28.294476 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 135229us
I20260709 12:26:28.295275 26121 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:28.295457 26198 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:28.243558 26120 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:28.194347 32605 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.181340 32721 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:28.221148 32709 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:28.099318 32481 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.306727 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:28.251598 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65049.1us
I20260709 12:26:28.206733 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:28.278996 32727 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:26:28.267489 32644 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:28.087984 32643 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.204761 32370 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.167608 32608 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.249218 32648 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.247727 32723 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:28.254081 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72332.5us
I20260709 12:26:28.322841 26124 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:28.357867 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.242167 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:28.222231 32694 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.234043 32640 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.267679 32706 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:28.245493 32684 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:28.366695 32684 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.367200 32640 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:28.367317 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 125294us
I20260709 12:26:28.367236 32694 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.368098 32684 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.251833 32697 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.314917 26783 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:28.336737 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:28.372406 26787 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.337492 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.374006 26785 raft_consensus.cc:2368] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:28.337715 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:28.334894 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 128241us
I20260709 12:26:28.250602 32670 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.339150 32234 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:28.339947 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:28.328387 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.349948 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.350224 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.385250 26786 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:28.385700 32709 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.386458 26054 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:28.387648 26786 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:28.387952 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:28.307899 32732 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:26:28.389730 32234 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:28.391078 32234 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.350805 26049 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:28.304559 32708 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.349589 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:28.364717 32644 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:28.396845 26789 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:28.397775 32644 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.398468 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:28.399333 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.400651 32490 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:28.401268 32490 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.401950 32367 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:28.403178 32370 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.403932 32640 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:28.408977 32583 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.409633 32583 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.375701 32604 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.410130 32583 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.343200 32605 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:28.399474 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:28.413028 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62904.6us
I20260709 12:26:28.417045 26018 leader_election.cc:400] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:28.417418 26018 leader_election.cc:403] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:28.418401 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.328305 32235 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:26:28.419143 32698 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:28.418468 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.302086 32589 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:28.330696 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.379639 26053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec84548f93742a096e72c1bd887809a"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:28.432963 26053 raft_consensus.cc:606] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:28.433333 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 211860us
I20260709 12:26:28.384385 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:28.434259 26782 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.328743 32113 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:28.419302 26049 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:28.302268 32323 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:28.426510 26756 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20.
I20260709 12:26:28.439105 26756 leader_election.cc:403] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:28.427058 26787 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:28.403203 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:28.431775 32604 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.441623 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:28.429854 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.430841 32367 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:28.370914 32723 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.445426 32429 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.445851 32704 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:28.446254 32704 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:28.446805 32669 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.447083 32704 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.369365 32710 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:28.448513 32710 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.449011 32710 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:28.449185 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:28.410148 32721 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:28.450114 32716 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20.
I20260709 12:26:28.451089 32716 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:28.381561 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:28.451676 26786 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:28.452584 32630 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:28.453290 32630 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:28.453811 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:28.454836 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.454293 32630 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.433517 26053 raft_consensus.cc:614] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:28.455765 32704 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:28.452826 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:28.456530 32716 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:28.457080 32716 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:28.458504 32716 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.403944 32370 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.459942 32370 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.460464 32640 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:28.460930 32640 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:28.461345 32640 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.462318 32370 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:28.461985 32697 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:28.463891 32235 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:28.464512 32235 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.464898 32235 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:28.465693 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.439338 32605 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:28.467979 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.450218 32721 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.477244 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.478410 26790 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:28.489394 32605 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.490851 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.492362 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.492991 26786 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:28.471599 26129 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.507071 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:28.431911 32615 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:28.508663 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.510645 26129 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.517096 32669 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:28.517488 32713 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.519317 32559 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:28.519821 32589 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.520852 32583 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:28.509289 32704 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:28.528777 26782 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:28.530925 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1419) took 1155 ms. Trace:
I20260709 12:26:28.532531 26791 rpcz_store.cc:276] 0709 12:26:27.375996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.376353 (+   357us) service_pool.cc:224] Handling call
0709 12:26:27.390200 (+ 13847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.236671 (+846471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.529012 (+292341us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:28.530845 (+  1833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816309120}
I20260709 12:26:28.513645 32527 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:28.466815 32113 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.535053 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:28.539209 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 371859us
I20260709 12:26:28.540503 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:28.541999 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:28.542930 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.544135 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.544981 26129 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.550892 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.551889 26126 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.510486 32723 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.536535 32527 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.570348 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.573135 26789 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:28.554472 32370 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:28.574059 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.590723 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.593415 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.594334 26120 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:28.568764 32527 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:28.606113 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64173.3us
I20260709 12:26:28.611270 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.612262 26046 raft_consensus.cc:2368] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18.
I20260709 12:26:28.613997 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.614775 26047 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:28.618711 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.622906 26126 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:28.623564 26123 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:28.633844 26129 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:28.638329 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.642215 26046 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:28.645467 26046 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:28.649822 26054 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:28.673946 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 332246us
I20260709 12:26:28.675165 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.639460 32713 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:28.632393 32704 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:28.674822 32742 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:26:28.688583 26046 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:28.665366 32429 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.692549 26047 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:28.692411 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:28.601259 32710 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.639276 32648 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:28.702593 26123 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:28.701843 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.711495 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.717298 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.694986 32493 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:28.719369 26785 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:28.679329 32640 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.721515 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.722698 26046 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.729475 26123 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:28.701540 32713 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:28.773193 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.695928 32704 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.780385 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1084) took 1290 ms. Trace:
I20260709 12:26:28.780884 26783 rpcz_store.cc:276] 0709 12:26:27.490439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.492582 (+  2143us) service_pool.cc:224] Handling call
0709 12:26:28.756429 (+1263847us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:28.780318 (+ 23889us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668035456}
I20260709 12:26:28.784286 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:28.784492 32608 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:28.784900 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:28.785475 26047 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:28.785739 32615 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:28.785863 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.786206 32615 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.786650 32615 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.787267 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:28.792630 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:28.793788 32743 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:28.794332 32743 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:28.794795 32615 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.795298 32743 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:28.798977 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.800746 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 292138us
I20260709 12:26:28.801646 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:28.803157 32713 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.803831 32615 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.805335 32753 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.808499 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:28.808427 32746 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:28.809733 32746 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:28.810107 32723 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:28.810195 32746 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.810746 32746 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.811882 32755 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:28.811880 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.815593 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:28.817684 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:28.818715 26784 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
W20260709 12:26:28.813403 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2455110400001104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
I20260709 12:26:28.820075 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.812577 32755 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:28.813336 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:28.853344 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:28.823069 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1034) took 1103 ms. Trace:
I20260709 12:26:28.854242 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:28.724382 32698 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:28.858421 26047 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:28.702220 32429 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:28.830844 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 161860us
W20260709 12:26:28.863919 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.832580 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.864908 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.854612 26203 rpcz_store.cc:276] 0709 12:26:27.719455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.803769 (+ 84314us) service_pool.cc:224] Handling call
0709 12:26:28.822836 (+1019067us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400335c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:28.822989 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600693120}
I20260709 12:26:28.735394 32323 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:28.867460 26054 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 12.
I20260709 12:26:28.862093 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:28.869234 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.871865 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:28.873559 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:28.733122 32493 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:28.884421 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 160550us
I20260709 12:26:28.868040 32608 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:28.860426 32759 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:28.871938 32604 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:28.886586 32615 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.866130 26046 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:28.871098 32737 raft_consensus.cc:2764] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:28.874526 32583 raft_consensus.cc:2764] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:28.874109 32738 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.925335 26200 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:28.884418 32598 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:28.926889 26198 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:28.932159 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78003.4us
I20260709 12:26:28.873461 32643 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:26:28.944540 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.946424 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:28.947669 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.948792 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.952908 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:28.953902 26121 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:28.954603 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:28.921823 32706 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:28.945525 32754 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:28.960167 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.963882 32323 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.959694 32694 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:28.969950 32694 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:28.981928 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:28.982909 26200 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:28.983305 26200 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:28.984373 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.979612   301 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.984961 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.985231   301 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:28.985914 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.987639   301 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:28.988801 32756 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:28.950407 32546 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:29.013814 32493 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.015914 32710 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:29.020915 32744 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:26:29.023733 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:28.926425 32615 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.051955 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:29.056777 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 123647us
I20260709 12:26:29.036980 26200 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.067744 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:29.078173 26203 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:28.973114 32582 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:29.088287 32761 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.148s	user 0.003s	sys 0.002s
I20260709 12:26:28.974321 32766 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:29.036499 32741 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:29.096719 32648 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.109115 32704 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.109882 32670 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.113333 26195 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 20. Ops: []
I20260709 12:26:29.116997 32640 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:28.923338 32755 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.082617 32598 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:29.138337 32113 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:29.124557 32721 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.000s
W20260709 12:26:29.124791 32235 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.000s
I20260709 12:26:29.160478 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:29.164623 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:29.121281 32737 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:29.169617 26128 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:29.121866 32766 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:29.178404 32235 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.098466 32582 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.183687 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:29.188405 32370 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.006s
I20260709 12:26:29.182372 32709 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.191350 32746 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.188978 32370 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:29.192883 32766 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:29.203344 32527 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.002s
I20260709 12:26:29.196564 26202 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:29.198563 32235 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:29.192898 32751 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:29.195897 32582 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.206202 32615 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.124382 32741 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:29.207226 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.207322 32746 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.209319 32741 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.126708 32648 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.212486 32755 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.203878 32527 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.252152 32648 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.186061 32669 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:26:29.260181 32694 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.260712 32738 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:29.180040 32721 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.263996 32721 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.239357 32615 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.270934 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 391795us
I20260709 12:26:29.272152 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:29.170218 32608 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:29.264926 32715 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:29.306785 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.293021 26790 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:29.295138 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:29.287123 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 393973us
I20260709 12:26:29.312888 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:29.327613 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:29.337373 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.357390 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44637.3us
I20260709 12:26:29.274982 32615 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:29.401275 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 131719us
W20260709 12:26:29.487228 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.816s (SENT) [suppressed 7 similar messages]
I20260709 12:26:29.378568 32527 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:29.286533 32551 raft_consensus.cc:2764] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:29.472321 32738 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.476533 26199 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:29.381011 32608 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.495011 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:29.494820 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:29.495684 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:29.497211 26094 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:29.500092 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:29.554783 26094 leader_election.cc:403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:29.506803 32644 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.508337 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:29.507719 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:29.539013 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:29.540019 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:29.561348 26053 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.561450 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 15.
I20260709 12:26:29.540498 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:29.564345 26051 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.541194 26049 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:29.543794 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:29.569408 26045 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:29.543962 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:29.548959 26050 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:29.547869 32429 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.506273 32766 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:29.555332 26124 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.554894 26129 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:29.559188 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62210.4us
I20260709 12:26:29.582458 26094 leader_election.cc:400] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18.
I20260709 12:26:29.582753 26094 leader_election.cc:403] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:29.570582 26052 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:29.555509 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:29.581441 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 73806.9us
I20260709 12:26:29.584661 26200 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:29.585089 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:29.585750 26050 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:29.586830 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:29.587486 26052 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
W20260709 12:26:29.588186 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82144.8us
I20260709 12:26:29.589726 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:29.594399 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:29.595268 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:29.598345 26052 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:29.599785 26045 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 7. (index mismatch)
I20260709 12:26:29.606945 26051 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:29.608415 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:29.608784 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:29.609212 26052 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.597569 26053 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:29.603760 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:29.616997 26049 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:29.615690 26045 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:29.630541 26195 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:29.631819 26194 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:29.633172 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.633322 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:29.632112 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.634758 26196 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.634202 26124 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:29.636011 26201 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:29.636380 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.637995 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.638746 26201 raft_consensus.cc:2368] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17.
W20260709 12:26:29.638075 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52340.2us
I20260709 12:26:29.553755 32694 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:29.650429 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.652421 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:29.662348 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:29.653322 26788 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:29.654107 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:29.654827 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:29.655186 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:29.663833 26129 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.657289 26125 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:29.658694 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:29.669765 26120 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 12.
I20260709 12:26:29.613308 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:29.590654 32670 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.676345 26196 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:29.677301 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:29.582876 32737 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:29.687426 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:29.628770 32704 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:29.677592 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:29.685674 26120 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:29.699255 26125 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:26:29.777138 26125 pending_rounds.cc:85] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 13
I20260709 12:26:29.777503 26125 pending_rounds.cc:107] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:29.777787 26125 raft_consensus.cc:2889] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:29.685804 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:29.780978 26120 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:29.572091 32738 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:29.766573 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58966.5us
W20260709 12:26:29.766902 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78431.4us
W20260709 12:26:29.784372 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.687824 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:29.613795 32708 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:26:29.676740 32743 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.698716 32323 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.665549 32429 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:29.793411 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105665us
I20260709 12:26:29.794091 26122 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:29.692457   302 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.583743 32644 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.702288 32741 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:29.694794 26129 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:29.676947   313 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.692655   315 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:29.659751 32685 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 12:26:29.677054 32638 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:29.783408   300 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:26:29.664562 32583 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
W20260709 12:26:29.797910 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 135670us
I20260709 12:26:29.819265 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:29.823135 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:29.708899 32670 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.817967 32708 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:29.849148 32708 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:29.850051 32708 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.848654 32615 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.708281 32694 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.786031 32704 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.793202 32743 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.796900 32323 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:29.866436 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 291728us
I20260709 12:26:29.803483 32429 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.804030   302 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:29.886564 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 271688us
I20260709 12:26:29.861934 32699 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.891798 32370 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18.
I20260709 12:26:29.872721   310 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:29.862907 32697 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:29.817705   315 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:29.879783 32708 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:29.888413   311 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.381s	user 0.001s	sys 0.001s
I20260709 12:26:29.906090 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1430) took 1431 ms. Trace:
I20260709 12:26:29.928326 26789 rpcz_store.cc:276] 0709 12:26:28.474556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.493232 (+ 18676us) service_pool.cc:224] Handling call
0709 12:26:28.493562 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:28.573610 (+ 80048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.905658 (+1332048us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b54005c35d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:29.905960 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182412672}
I20260709 12:26:29.813283   313 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:29.918503   309 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.360s	user 0.001s	sys 0.001s
I20260709 12:26:29.960400 26052 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:29.897696 32699 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.966909 32738 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.968364 32699 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.970062 32670 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:29.965696 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 375788us
W20260709 12:26:29.971719 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 835) took 1590 ms (client timeout 1638 ms). Trace:
I20260709 12:26:29.971779   348 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:29.972479 26048 rpcz_store.cc:269] 0709 12:26:28.381132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.381287 (+   155us) service_pool.cc:224] Handling call
0709 12:26:29.971552 (+1590265us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54001b49d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:29.971653 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1101571840}
I20260709 12:26:29.973016   348 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:29.976990 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:29.977171 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2130) took 1003 ms. Trace:
I20260709 12:26:29.977594 26046 rpcz_store.cc:276] 0709 12:26:28.973556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.973758 (+   202us) service_pool.cc:224] Handling call
0709 12:26:28.974247 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.475111 (+500864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.476213 (+  1102us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.477951 (+  1738us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.689881 (+211930us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:29.869921 (+180040us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.870409 (+   488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.870552 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.976944 (+106392us) raft_consensus.cc:1685] finished
0709 12:26:29.976981 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.977111 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":273160,"raft.queue_time_us":48764,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":17724,"thread_start_us":796199,"threads_started":3,"wal-append.queue_time_us":403325}
I20260709 12:26:29.978741   348 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:29.979409   313 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:29.982110 32694 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:29.942888   310 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:29.984416 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:29.998477 26200 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:30.000321 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2163) took 1027 ms. Trace:
W20260709 12:26:29.998878 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 213459us
I20260709 12:26:29.999225 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:30.018477 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:30.000960 26200 rpcz_store.cc:276] 0709 12:26:28.972865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.981526 (+  8661us) service_pool.cc:224] Handling call
0709 12:26:29.584301 (+602775us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b58004ec8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a927c 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c
0709 12:26:30.000136 (+415835us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527558528}
W20260709 12:26:29.964632   319 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
W20260709 12:26:30.026590 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 147788us
W20260709 12:26:30.027158 32759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.012s	sys 0.010s
I20260709 12:26:30.022929 32699 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:30.069950 32759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.012s	sys 0.011s
W20260709 12:26:30.077652 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 190291us
W20260709 12:26:30.125123 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91198.2us
I20260709 12:26:30.033674   335 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:30.079376 32644 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.130493 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.148049 32559 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:30.148773 32559 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:30.027415 32694 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:26:30.150611 32559 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.077276 32699 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:30.151580 32699 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:30.089666   319 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:26:30.156824   330 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:26:30.157757   346 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.128746   310 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.161720 32704 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.085453 32759 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:30.023181   348 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:26:30.070231   333 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:26:30.158406   346 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.149904 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.141003 26019 leader_election.cc:400] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17.
I20260709 12:26:30.085641   332 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:30.174894   332 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:30.175840   332 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.176393 32323 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.177554 32759 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.136536   334 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:30.167778 26019 leader_election.cc:403] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:30.169442   315 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.169978 32743 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.170441 32429 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.126446   306 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:30.174257   317 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.682s	user 0.001s	sys 0.001s
I20260709 12:26:30.202010   334 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:30.190573 32563 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:30.203476 32563 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.203126   334 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.203935 32563 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.204802 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1137) took 1227 ms. Trace:
I20260709 12:26:30.205224 26123 rpcz_store.cc:276] 0709 12:26:28.977210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.977401 (+   191us) service_pool.cc:224] Handling call
0709 12:26:28.977854 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:28.978288 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:28.978330 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:28.978953 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.378070 (+399117us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:29.378125 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.378221 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.173676 (+795455us) spinlock_profiling.cc:243] Waited 9.79 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:30.204472 (+ 30796us) raft_consensus.cc:1685] finished
0709 12:26:30.204501 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.204734 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2831,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":11442,"spinlock_wait_cycles":22511872,"thread_start_us":349329,"threads_started":1,"wal-append.queue_time_us":499411}
W20260709 12:26:30.210171   318 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.420s	user 0.001s	sys 0.003s
I20260709 12:26:30.195217   350 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:30.195952   306 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.196933   346 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.200409   353 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:30.200469 32753 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:30.201454   344 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 12:26:30.212675   354 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:30.212598 32704 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:30.214633   302 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:30.214839 32743 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17.
I20260709 12:26:30.212255 32699 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.182682 32559 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:30.187685 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54722.9us
W20260709 12:26:30.211977   323 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
W20260709 12:26:30.275121 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 248562us
I20260709 12:26:30.298278 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:30.287014 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.300877   343 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:30.301113 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:30.227646   350 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:30.214648   315 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.168174   335 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.316401 32683 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.301879   343 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:30.315074   350 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.348441   351 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:26:30.349483   341 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.314818 32743 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.359314 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.367540 26047 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:30.359252 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.320683 32697 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.364135 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 181354us
W20260709 12:26:30.365566 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93292.9us
I20260709 12:26:30.369663 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.371613 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:30.373497 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:30.373829 26783 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:30.374123 26785 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:30.371469 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.319931   315 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.379168 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.381472 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:30.382395 26782 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:30.383222 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:30.383802 26789 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:30.384910 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:30.388619   312 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:26:30.375419 32704 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.372998 32697 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:30.359478 32683 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.380450   353 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:30.375857 32640 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:30.400127   328 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.435s	user 0.001s	sys 0.001s
I20260709 12:26:30.384853 32630 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:30.386595 32759 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:30.417265 26785 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:30.419516 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:30.422819 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.424350 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.426883 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:30.441944 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.443459 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.448504 26789 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:30.448848 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.453495 26050 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:30.455175 26199 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:30.411697   343 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:30.455643 32490 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.459365   335 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:30.459676 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.434897 32759 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:30.464536 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:30.472226 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.474730 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.474826 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.475752 26199 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:30.455853 32323 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:30.478212 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:30.470594   333 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.470458 32766 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.534343   372 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:30.471771 32692 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.561456 32692 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.562152   372 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:30.562834   372 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:30.563154   372 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:30.563535   372 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:30.563813 32692 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:30.473034   353 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:30.565503 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.477340   312 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:30.470988 32490 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:30.557749 32723 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.482331 32323 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.477607   335 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:26:30.477447 32113 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 12:26:30.592867   301 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.598516   350 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:30.598870 32323 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.599151   301 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.600193   315 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.601404   301 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.602464 32563 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:30.604074 26203 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:30.605753 26202 raft_consensus.cc:2376] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:26:30.575870 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.599646 26052 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:30.606817 32563 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:30.607858 32759 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.565573   333 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.615397   333 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.595894 32723 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.595135 32490 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:30.622732 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1099) took 1033 ms. Trace:
I20260709 12:26:30.623229 26788 rpcz_store.cc:276] 0709 12:26:29.589633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.590084 (+   451us) service_pool.cc:224] Handling call
0709 12:26:29.590411 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.663164 (+ 72753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.622501 (+959337us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54004501d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:30.622675 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87555456}
I20260709 12:26:30.625051 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1143) took 1125 ms. Trace:
I20260709 12:26:30.633105 26198 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:30.634040 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.634918 26201 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:30.575013   353 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.632874 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.626940 26127 rpcz_store.cc:276] 0709 12:26:29.499728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.504095 (+  4367us) service_pool.cc:224] Handling call
0709 12:26:29.504424 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.807319 (+302895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.807371 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.807438 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:29.807482 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.624755 (+817273us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:30.624820 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.624981 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9638,"spinlock_wait_cycles":673478528}
I20260709 12:26:30.567907 32766 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.642174 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.644342 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.648473 26198 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:30.649369 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.649542 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:30.645087 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 167898us
I20260709 12:26:30.645345 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.645725 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.651647 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.655839 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:30.658576 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.658988 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.641132 32490 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:30.660037 26046 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:30.662129 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:30.660598 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.663852 26047 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.573077   312 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:30.671180 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:30.661525 32490 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.672071 32490 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.672449 32721 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:30.675021 32721 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:30.643577   301 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:26:30.675558 32721 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.655855   341 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:30.676028   301 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:30.658041   349 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:30.657616 32323 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:30.676120 32721 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.706216 32766 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.658144   354 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:26:30.676417   357 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:30.691147 32113 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.722370 32685 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:30.725391 32583 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:30.728907 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 909697us
I20260709 12:26:30.683811   350 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.704496   349 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.730935 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:30.730959 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45808.8us
I20260709 12:26:30.753408 26054 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:30.732692 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.666604 32694 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 12:26:30.740983 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69880.3us
W20260709 12:26:30.744681 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48805.5us
I20260709 12:26:30.694244   301 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:30.730049 26049 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:30.723940 32699 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.755692 26047 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:30.751057 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:30.788300 26049 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:30.750365 32583 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:30.792253 26051 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:30.734920   357 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.744633 32685 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:30.752679   349 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:30.755232 32638 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:30.833091 32323 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:30.882898 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 129375us
I20260709 12:26:30.783886 32113 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:30.884224 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 98083.8us
I20260709 12:26:30.832504   341 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.832931 32699 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:30.840731   339 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:30.840008   357 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.843451 32738 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.890316 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104648us
W20260709 12:26:30.894460 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 942415us
W20260709 12:26:30.901793 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150871us
I20260709 12:26:30.903708 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.909211 26201 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:30.910187 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:30.886209 32370 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.926615 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:30.934319 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:30.935060 32741 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:30.935441 32741 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:30.935807 32741 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.936513 32737 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:30.936946 32737 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:30.937294 32737 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:30.940102 32638 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:30.889916   345 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.948383 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.902180 32546 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.925892 32598 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:30.901672 32708 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:30.906659 32723 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.905319   353 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.881125 32583 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:30.972007 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105228us
I20260709 12:26:30.929567 32738 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.977269 32367 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:30.980559 32367 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.981599 32738 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.982025 32723 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:30.982883 32367 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:30.982620 32723 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.894485   301 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.985647   301 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.953404   376 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:30.902843 32699 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.006371   389 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.906847   339 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.010766 32113 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:30.977974 32598 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:30.964164 32739 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.011973   312 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:30.942672 32490 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:30.989794 32563 consensus_queue.cc:1175] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 19 status { last_received { term: 17 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.14-18.14]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:30.924511 32685 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.956597   345 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:31.059201 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 151749us
I20260709 12:26:31.074640 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.076581 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.080771 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:30.945647 32370 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.143303 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.144151 26126 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:30.979060 32638 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.902442   341 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:30.964845 32546 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.014001   337 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:31.141361 26121 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:31.142134 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:31.020689 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 113696us
I20260709 12:26:31.084666 32605 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.145958 26127 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:26:30.973820 32708 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.149837 26201 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:31.020385 32684 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.014858   372 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.190757 32370 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.191548 32490 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.211498 32490 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:31.211812 32708 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:31.199883 32648 consensus_queue.cc:1175] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 17 status { last_received { term: 16 index: 13 } last_committed_idx: 11 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:31.212208 32490 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:31.192217   341 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.199559 26122 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:31.096100 32367 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:31.207413 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65372.1us
I20260709 12:26:31.210541 32694 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.245847 26129 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:26:31.248680 26129 pending_rounds.cc:85] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 12
I20260709 12:26:31.249051 26129 pending_rounds.cc:107] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:26:31.249387 26129 raft_consensus.cc:2889] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.248196 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.245406 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:30.989993 32323 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.018937   376 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.253957 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.275275 26054 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:26:31.272480 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.093628 32739 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.024852   389 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.282876 26128 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:26:31.283370 26128 pending_rounds.cc:85] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:26:31.283744 26128 pending_rounds.cc:107] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:26:31.284098 26128 raft_consensus.cc:2889] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.151607   388 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:31.288589 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:31.293432 26054 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.294380 26094 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:31.168754   337 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.297154   337 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.297705   337 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.297948 32739 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.298245   337 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.306694   337 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.307677 26199 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:31.248075 32685 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.155220 32638 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:31.304116 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.335131 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.239845   333 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.271080   341 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.202538 32684 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.351967 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.286309 32708 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:31.350867 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.290803 32370 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:31.286583   391 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:31.358922 26122 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:31.291208 32738 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:31.203823 32605 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.365043 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.374294 32370 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.310405 32741 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.270776 32367 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.316654   388 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.379477   372 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.253610 32490 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.386549 26788 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:31.409672 26198 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:31.410094 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.317708 32737 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.418155 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.271679 32694 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:31.412139   380 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.391s	user 0.000s	sys 0.001s
W20260709 12:26:31.413733   384 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:26:31.446368 26124 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:31.448998 26122 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:26:31.454522 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 158810us
I20260709 12:26:31.455575 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:31.354287   341 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.450363 26054 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:31.285388 32598 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.316190   349 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.368018 32708 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.385181   391 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.353546   333 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.473184 32684 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.407752 32738 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.468657 32604 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.493553 26050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:31.469290 32699 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.473692   395 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.495378 26050 raft_consensus.cc:606] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:31.479507 32723 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.506744 32546 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.478792 32638 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.512780 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2567) took 1032 ms. Trace:
I20260709 12:26:31.513206 26200 rpcz_store.cc:276] 0709 12:26:30.480886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.481047 (+   161us) service_pool.cc:224] Handling call
0709 12:26:30.523422 (+ 42375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.641151 (+117729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.641193 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.641772 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 12:26:30.795601 (+153829us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:31.217440 (+421839us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.271562 (+ 54122us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.434339 (+162777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.434438 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.512547 (+ 78109us) raft_consensus.cc:1685] finished
0709 12:26:31.512573 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.512706 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86591,"prepare.queue_time_us":475597,"raft.queue_time_us":367357,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":34759,"spinlock_wait_cycles":9036032,"thread_start_us":747671,"threads_started":4,"wal-append.queue_time_us":310367}
W20260709 12:26:31.500478   409 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 12:26:31.523855 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 68378.3us
I20260709 12:26:31.504563   345 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.505465   339 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.533406 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.534047 26048 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 12:26:31.534178 26050 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:31.535677 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.538770 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.538770 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.539112 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.539757 26200 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:31.539938 26048 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:31.516702   403 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:31.547439 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.544773 32741 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.526618 32638 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:31.553913 26054 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:31.520687 32546 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.500954 32604 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.560447 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.561522 26050 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:31.561798 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.544615   412 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:31.549373 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.564811 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
W20260709 12:26:31.544867   407 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:26:31.555053 26054 raft_consensus.cc:606] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:31.508150   395 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.557595 32741 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.557684   349 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.573448   345 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:31.574378   341 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.574734   395 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:31.578982 32721 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.833s	user 0.009s	sys 0.002s
I20260709 12:26:31.548128 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.548173 26048 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:31.545446 32370 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.565315 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:31.567119 26054 raft_consensus.cc:614] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:31.570698 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.571460 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.571717 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.589578 26200 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:31.589756 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:31.562922   357 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 12:26:31.590883   303 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:31.553133   403 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.580407   306 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.591322   303 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.591749   303 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.592504   395 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.593065   395 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.595654   353 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.596230   353 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.598454   353 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.594807   403 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.600376   403 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:31.600939   403 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.602010   303 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.581211 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.582297 32738 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.554356 32638 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.583806 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.619884 26198 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:31.583951 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.590590   357 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.579368 32741 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.580986   333 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.564193 32546 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.605387 32604 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.607043 26126 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:26:31.569762   376 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.582912 32737 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.587232   412 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.609423 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.609634 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:31.610167   403 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.632309 26201 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:31.614374 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:31.615556 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50420.4us
I20260709 12:26:31.633368 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:31.613813   413 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:31.617494 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.618559 26121 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:31.620944 32638 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.625402 26126 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 13
I20260709 12:26:31.638773 26126 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:31.639045 26126 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.625876 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.628017 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:31.614768 32694 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.632309 26195 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:31.648128 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:31.634720 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.657989 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.658797 26124 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:31.617282 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.633785 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:31.627494   301 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.004s
I20260709 12:26:31.622484 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.660281 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:31.665304 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:31.620417 32685 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:31.640621 26127 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.641469 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.643783 32683 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.645033   303 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.645435 32741 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.645962 32708 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.637157   412 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.648362 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.682793 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:31.649163 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2573) took 1017 ms. Trace:
I20260709 12:26:31.683727 26194 rpcz_store.cc:276] 0709 12:26:30.631643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.632096 (+   453us) service_pool.cc:224] Handling call
0709 12:26:30.632422 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.709950 (+ 77528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.710029 (+    79us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.710098 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:30.710152 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.648933 (+938781us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b5400302354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:31.648978 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.649094 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177714560}
I20260709 12:26:31.650899 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2563) took 1207 ms. Trace:
I20260709 12:26:31.685446 26203 rpcz_store.cc:276] 0709 12:26:30.443056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.443173 (+   117us) service_pool.cc:224] Handling call
0709 12:26:31.650629 (+1207456us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540039ab54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:31.650817 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367160576}
I20260709 12:26:31.651242 32546 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.660051 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.581187   419 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:31.668800 32694 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:31.603681 32721 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.665660 26791 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:31.706664 32697 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.710582 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:31.713126 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:31.713181   357 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.714008 26199 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:31.716549 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.718241 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.719322 26195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:31.719838 26195 raft_consensus.cc:606] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:31.619787   353 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:31.675712 32709 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.678443 26201 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:31.667106 32738 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:31.666956   413 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:31.687672 26124 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:31.668013 32685 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.662439 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:31.693966   419 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:31.700290 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.701042 32694 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.721220 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.711886 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.667578 26198 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:26:31.727563 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.727676 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:31.727877 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:31.729575 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 67217.2us
I20260709 12:26:31.730135 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:31.752866 26201 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
W20260709 12:26:31.724116   382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.022s	sys 0.034s
I20260709 12:26:31.753679 26124 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:31.754316 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.755059 26128 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:31.756628 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:31.753196 26129 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:31.776650 26121 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 18 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:31.729024 32697 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.777866 32697 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.779299 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.729400 32738 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:31.780743 32738 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.780947   412 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.782071   419 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:31.781412 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.783226 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:31.783416   419 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.784160 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:31.785051 32638 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.786005 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:31.794135 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 163980us
I20260709 12:26:31.794247 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:31.789498 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.749492 32546 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:31.797317 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.788642 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:31.790477 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.729149   413 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.730736 32685 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.679345   420 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:31.792001 26121 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.777019 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.792747 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.802975 26046 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:31.793476 26201 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:31.787786   382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.024s	sys 0.037s
I20260709 12:26:31.754853 32694 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:31.775740   350 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:31.755359 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:31.795832 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:31.799002 26128 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:31.800482 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:31.801877 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:31.811924 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56670.8us
I20260709 12:26:31.804876 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.814364 26201 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:31.795477 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.769049 32683 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:31.814437 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.803633 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.815658 26129 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:31.817688 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.819221 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.797403 32546 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.825460 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:31.730687 32709 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:31.826362 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:31.827077 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.828562 26129 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 13.
I20260709 12:26:31.828850 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.829303 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.830552 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.800915   413 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.831447 26047 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:31.807243   420 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.835307 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:31.835848 26129 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:31.837245 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.838792 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:31.836701   420 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.842667 32741 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:31.843109 32741 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:31.843528 32741 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:31.845655   395 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.845866 32709 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:31.836912 32598 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:31.847275 32598 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.847657 32598 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:31.849423 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:31.850061 26791 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:31.822008   350 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:31.816624 32698 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.865618 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.823011 32683 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.853433 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:31.856432 26129 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 7. (term mismatch)
I20260709 12:26:31.873622 26129 pending_rounds.cc:85] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 6
I20260709 12:26:31.831641 32546 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.876526 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:31.856909 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.877877 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:31.873984 26129 pending_rounds.cc:107] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:26:31.878612 26129 raft_consensus.cc:2889] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.879043 26129 pending_rounds.cc:107] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:26:31.879371 26129 raft_consensus.cc:2889] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.838397   412 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.834474   425 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:31.892964 26127 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:31.816431   357 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:26:31.897652 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:31.833841 32551 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:26:31.885264   340 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.924506   340 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.924926   340 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.903551 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.867128 32698 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.947582 32698 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.915735 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.948393 26050 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:31.948221 32639 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:31.950008 32721 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:31.921262 26046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:31.951046 26046 raft_consensus.cc:606] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:31.951464 26046 raft_consensus.cc:614] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:31.949739   339 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:31.875520 32683 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:31.950445 32559 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:31.885555 32370 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:31.900761 26121 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:31.971535 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:31.972950 26046 raft_consensus.cc:2376] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19. Re-sending same reply.
I20260709 12:26:31.908834 32708 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:31.977849 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.978937 26046 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:31.988525 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:31.989267 26046 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:31.987012 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:31.996582 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:31.996582 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:32.001058 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 103497us
I20260709 12:26:32.002357 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.005146 26201 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:32.006511 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.008857 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.009982 26199 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 15: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:31.967301 32370 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.011234 26201 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:32.012862 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.013363 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 160624us
I20260709 12:26:31.920092 32694 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.016284 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.020849 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.017853 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.022709 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.023217 26199 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.967223   339 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:26:32.010883   405 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.470s	user 0.004s	sys 0.001s
W20260709 12:26:32.037456 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.063978 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:32.031574 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.013568   421 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:32.052744 32648 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:32.060194 26046 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:32.069869 26054 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
I20260709 12:26:32.019774 32638 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.019958 32370 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:32.069305   428 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:32.069319 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.111222 26050 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.095484 32598 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.084936   421 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:32.087998 32694 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.103765   340 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.104794 32741 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.090543   382 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.025413 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.066890 32704 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.108561 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.118834 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.115423   421 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.122534 32637 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.146579   420 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.156896 32637 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.158900   420 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.162654 32683 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.163551 32697 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.164376   345 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:32.164878   345 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:32.123075 32741 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.123219 32583 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:32.124930 32598 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.125764 26129 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:32.107177 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.181991 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:31.992297   350 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:32.139941 26019 leader_election.cc:336] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.816s (SENT)
I20260709 12:26:32.072429 32738 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.145573 26196 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:32.148075 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.149585 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.146548 32704 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.201874 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.203112 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:32.155769   430 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:32.156049   421 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.168955 26050 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:32.169941 26784 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:32.169996 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.170908 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.180575 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.186264 26787 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.186952 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:32.189029 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49151.7us
I20260709 12:26:32.218396 26201 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:32.218942 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:32.191684 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.180042 32639 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.196959 26199 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:32.166730 32638 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:32.200820 26785 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:32.165962 32683 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:32.227450 26124 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:32.226531 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.228055 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.228551 26199 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.214164 26196 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:32.175249   412 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.193572 32753 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.174238 32755 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.239400 32685 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.241732   340 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.248456 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:32.196602 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47123.3us
I20260709 12:26:32.249939 32738 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.250874 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:32.232069 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:32.252808 32546 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.253798 32638 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:32.254233 32638 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.254609 32638 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:32.255023 32546 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.256948   366 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:32.257552   366 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:32.206394 32741 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:32.258183 32741 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.258749 32741 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.258410   366 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.233556 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.261300 26128 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:32.261803 26046 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:32.165877   345 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.262667 26046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:32.263309 26046 raft_consensus.cc:606] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.263357   345 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.263732 26046 raft_consensus.cc:614] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:32.235036 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.265803 26050 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:32.230540 32639 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.238886 26786 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:32.239537 32755 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.224326 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.232009 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.226019 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.249982 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.272082 26783 raft_consensus.cc:2376] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17. Re-sending same reply.
I20260709 12:26:32.248803 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.273197 26784 raft_consensus.cc:2376] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20. Re-sending same reply.
I20260709 12:26:32.249276 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.274385 26791 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.278663   435 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.232506   359 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.241894 32685 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.287781 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.435367 26199 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
W20260709 12:26:32.423307 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 215400us
I20260709 12:26:32.436633 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.437054 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.437752 26196 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.370234 26791 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:32.438668 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.314704 26789 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:32.310817 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.440984 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.445106 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:32.434587 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 138683us
I20260709 12:26:32.447551 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:32.452452 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 141733us
I20260709 12:26:32.452131 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.295079 32738 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.459518 26123 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:26:32.460142 26123 pending_rounds.cc:85] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 12
I20260709 12:26:32.460528 26123 pending_rounds.cc:107] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:26:32.460843 26123 raft_consensus.cc:2889] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.346130 32598 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:32.465230 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.468475 26199 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.469009 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:32.471102 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3506) took 1805 ms. Trace:
I20260709 12:26:32.421444 32639 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.317112 32697 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.407481 32721 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:32.440021 32704 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.370103   382 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.479099 32583 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:32.500841 26199 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.282270 32753 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.487766 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.504263 26120 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.504591 26053 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.503794 26201 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:32.443961   403 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.441161   340 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.294948 32699 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.168s
I20260709 12:26:32.444330   357 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.307982   413 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.543126 32370 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.545058   413 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:32.546489   340 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.514856 32598 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.548837 32598 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.543141   357 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.550350 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3593) took 1255 ms. Trace:
I20260709 12:26:32.551188 32598 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.551908   357 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.451666   388 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.456848 32685 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:32.453333   435 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:32.455694   420 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.309257 32698 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:32.614352   404 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.135s	user 0.004s	sys 0.000s
I20260709 12:26:32.614589 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.615407 26050 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:32.615881 26790 raft_consensus.cc:2368] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18.
I20260709 12:26:32.616453 26050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:32.475683 32551 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.506276 26201 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:32.500409 32697 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.618466 26050 raft_consensus.cc:606] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:32.618886 26050 raft_consensus.cc:614] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:32.620180 32697 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.509268 32723 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:32.621393 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:32.621830 26050 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:32.622725   367 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:32.623319   367 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:32.624363   367 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.519380 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3501) took 2096 ms. Trace:
I20260709 12:26:32.471813 26045 rpcz_store.cc:276] 0709 12:26:30.649416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.649581 (+   165us) service_pool.cc:224] Handling call
0709 12:26:30.649950 (+   369us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:30.650893 (+   943us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:30.671168 (+ 20275us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.745384 (+ 74216us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:30.752829 (+  7445us) log.cc:844] Serialized 78 byte log entry
0709 12:26:30.752997 (+   168us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:30.755686 (+  2689us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.756106 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.756509 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.508352 (+751843us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b18003e2000. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:31.703348 (+194996us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b18003e2000. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:32.085999 (+382651us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18003e2000. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:32.107484 (+ 21485us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b18003e2000. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:26:32.453392 (+345908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.453475 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.453521 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:32.453535 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.454517 (+   982us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18826,"prepare.queue_time_us":1577700,"prepare.run_cpu_time_us":189,"prepare.run_wall_time_us":2354,"raft.queue_time_us":35771,"raft.run_cpu_time_us":704,"raft.run_wall_time_us":65050,"spinlock_wait_cycles":251468672,"thread_start_us":419523,"threads_started":2,"wal-append.queue_time_us":273762,"wal-append.run_cpu_time_us":3425,"wal-append.run_wall_time_us":1039275}
I20260709 12:26:32.522786 32753 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.527541 32367 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.532148   303 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:26:32.532472   439 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:26:32.534168 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.535477 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.632330 26048 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.309439   359 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.537088 32583 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:32.506378 32721 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.634100 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:32.494728   350 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.538949 32605 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.539891 32755 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.582681   395 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.564071   446 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.578698 26018 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.646721 26790 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:32.647245 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68640.1us
I20260709 12:26:32.557394 32639 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:32.561108   389 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.575073 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2629) took 1094 ms. Trace:
I20260709 12:26:32.564638   390 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:32.589542   437 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:32.567709 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.562388 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1162) took 1366 ms. Trace:
I20260709 12:26:32.577960 26127 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.568713   421 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.675755   421 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.514446   382 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.676267 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.677191 26783 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:32.678092   382 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.678378 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.678895 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.679461 26788 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:32.560423 26053 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:32.557685 32370 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.680631 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.577356   340 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.592183 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.628753   420 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:32.551718 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2561) took 2109 ms (client timeout 1816 ms). Trace:
W20260709 12:26:32.688045 26202 rpcz_store.cc:269] 0709 12:26:30.441839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.442366 (+   527us) service_pool.cc:224] Handling call
0709 12:26:32.551479 (+2109113us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:32.551596 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311903104}
I20260709 12:26:32.564781 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1160) took 1369 ms. Trace:
I20260709 12:26:32.564083 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.554740   343 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.552340 26197 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:32.550884 26123 rpcz_store.cc:276] 0709 12:26:31.295000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.871053 (+576053us) service_pool.cc:224] Handling call
0709 12:26:31.871336 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.461411 (+590075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.549985 (+ 88574us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540006fed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.550264 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351647104}
I20260709 12:26:32.497962 26196 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:32.583119   391 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:32.608740 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3598) took 1301 ms (client timeout 1555 ms). Trace:
I20260709 12:26:32.668462 32605 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:32.617389 26094 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
I20260709 12:26:32.521214 32637 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.472415 32738 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.626418 26052 rpcz_store.cc:276] 0709 12:26:30.423040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.425975 (+  2935us) service_pool.cc:224] Handling call
0709 12:26:30.426182 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.606706 (+180524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.519074 (+1912368us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400229054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.519304 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398489984}
I20260709 12:26:32.629832 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.556020 32638 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.711707 32583 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.712502 32738 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.708469 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.717275 26052 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:32.718264 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.719337 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.721531 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.722508 26047 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:32.615500 32698 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.650897 26048 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:32.652419 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:32.654587 26050 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:32.727355 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2636) took 1191 ms. Trace:
I20260709 12:26:32.727816 26121 rpcz_store.cc:276] 0709 12:26:31.535470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.448341 (+912871us) service_pool.cc:224] Handling call
0709 12:26:32.608987 (+160646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.609189 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.609211 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.609262 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:32.609291 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.609344 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.727190 (+117846us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54001c5550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:32.727296 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363633664}
I20260709 12:26:32.728495 26050 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.629483   303 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:32.670457 26124 rpcz_store.cc:276] 0709 12:26:31.480813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.776757 (+295944us) service_pool.cc:224] Handling call
0709 12:26:32.574767 (+798010us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54001f95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:32.574981 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970222976}
I20260709 12:26:32.577837   315 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:32.730528   315 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:32.731029   303 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.732720   303 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.675740 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.673151 26125 rpcz_store.cc:276] 0709 12:26:31.195887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.196068 (+   181us) service_pool.cc:224] Handling call
0709 12:26:31.196469 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.403066 (+206597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.403104 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.403147 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:31.403178 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.562063 (+1158885us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.562139 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.562318 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468998400}
I20260709 12:26:32.657651   395 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.738647 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3606) took 1130 ms. Trace:
I20260709 12:26:32.592413 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:32.702672 26122 rpcz_store.cc:269] 0709 12:26:31.307076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.362933 (+ 55857us) service_pool.cc:224] Handling call
0709 12:26:32.608477 (+1245544us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54001c5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:32.608643 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31484800}
I20260709 12:26:32.739655   395 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:32.739205 26125 rpcz_store.cc:276] 0709 12:26:31.608526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.737525 (+1128999us) service_pool.cc:224] Handling call
0709 12:26:32.737949 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.738223 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.738271 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.738326 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.738374 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.738443 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.738576 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.697535 26783 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:32.704191 26123 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.704879 26094 leader_election.cc:403] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:32.745484 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3600) took 1206 ms. Trace:
I20260709 12:26:32.745963 26123 rpcz_store.cc:276] 0709 12:26:31.539396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.703676 (+1164280us) service_pool.cc:224] Handling call
0709 12:26:32.703973 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.745145 (+ 41172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.745416 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.706983 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.706717 26126 rpcz_store.cc:276] 0709 12:26:31.195233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.195407 (+   174us) service_pool.cc:224] Handling call
0709 12:26:31.195787 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.639517 (+443730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.564545 (+925028us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54001f5ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.564721 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945009536}
I20260709 12:26:32.643827   350 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.731487   315 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.693364 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.740723 26122 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:32.610606 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.668370 32367 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:26:32.667604   446 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.703984   343 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.785526 26121 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:32.723054 32691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.024s	sys 0.019s
I20260709 12:26:32.712605 32638 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.927214 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2141) took 1371 ms. Trace:
W20260709 12:26:32.841949 32741 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.005s
I20260709 12:26:32.705194 32605 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.902312 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.812263   359 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:32.754525 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 137265us
I20260709 12:26:32.899199 26790 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:32.923506 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3595) took 1627 ms. Trace:
I20260709 12:26:32.843238 26050 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:32.837647 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2638) took 1301 ms. Trace:
I20260709 12:26:32.917723   418 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.930364 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3603) took 1337 ms. Trace:
I20260709 12:26:32.927690 26051 rpcz_store.cc:276] 0709 12:26:31.555679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.556011 (+   332us) service_pool.cc:224] Handling call
0709 12:26:31.556357 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.923122 (+366765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.923153 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.923200 (+    47us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:31.923235 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.926948 (+1003713us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.926989 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.927152 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842922240}
I20260709 12:26:32.941663 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:32.855962 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2570) took 2282 ms. Trace:
I20260709 12:26:32.942700 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:32.943408 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.838020   350 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.754052   391 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.920918 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3601) took 1381 ms. Trace:
I20260709 12:26:32.928747 26049 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:32.819578   390 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:32.938359 26127 rpcz_store.cc:276] 0709 12:26:31.536168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.450307 (+914139us) service_pool.cc:224] Handling call
0709 12:26:32.450803 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.673980 (+223177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.837299 (+163319us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002ca1d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.837543 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291950976}
I20260709 12:26:32.942924 26197 rpcz_store.cc:276] 0709 12:26:30.573464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.573677 (+   213us) service_pool.cc:224] Handling call
0709 12:26:32.552084 (+1978407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.702336 (+150252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.855674 (+153338us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b5400336150. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:32.855880 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4505112960}
I20260709 12:26:32.932554 26054 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:32.929105 32691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.028s	sys 0.019s
I20260709 12:26:32.934456 32605 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.944434 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.944736 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:32.944883 26129 rpcz_store.cc:276] 0709 12:26:31.295604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.880394 (+584790us) service_pool.cc:224] Handling call
0709 12:26:31.880871 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.181628 (+300757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.923262 (+741634us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54001d1fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:32.923447 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562767232}
I20260709 12:26:32.945516 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.948779 32583 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.945869 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.946765   418 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.954344 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.933022 32741 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.955525   418 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.956964 32741 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.953445 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:32.959110 32638 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.959709 32638 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.947587 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:32.928593   315 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.947445   340 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.950812 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.941684 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3608) took 1262 ms. Trace:
I20260709 12:26:32.781105 32738 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:32.961949   389 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.965000   389 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.967090   389 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.953439 32583 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.948581 26121 rpcz_store.cc:276] 0709 12:26:31.539680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.751479 (+1211799us) service_pool.cc:224] Handling call
0709 12:26:32.785238 (+ 33759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.919598 (+134360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.920855 (+  1257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.952487   390 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:32.953855 26786 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:32.964164 26122 rpcz_store.cc:276] 0709 12:26:31.679106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.740095 (+1060989us) service_pool.cc:224] Handling call
0709 12:26:32.740518 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.941349 (+200831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.941605 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.970229 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:32.970819 26787 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.970026 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:32.962507   421 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:32.972321   421 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:32.970624   340 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:32.975453   417 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.412s	user 0.007s	sys 0.000s
I20260709 12:26:32.977445 32583 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.978600   340 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.980440   421 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:32.983933 26787 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:32.984442 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:32.998812 26790 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:33.000115 32721 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.001535   446 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:32.964030 32741 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:33.006374 32741 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.973517   389 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:26:33.011379   421 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:33.011803   421 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:33.012104   390 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:33.012740   421 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:33.014098   451 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.298s	user 0.000s	sys 0.003s
I20260709 12:26:33.010352 32741 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.951097   391 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.014065   421 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:33.017524   391 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.021427   420 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:32.943856 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.071192 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:33.071489 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:33.073037 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:33.068046 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:33.067590 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:33.074193 26046 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:33.075726 26047 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:26:32.947288 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:33.095817 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 125709us
I20260709 12:26:33.097517 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.102986 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:33.106648 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.114485 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:33.116845 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:33.118208 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1164) took 1578 ms. Trace:
I20260709 12:26:32.939992 26124 rpcz_store.cc:276] 0709 12:26:31.593317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.730748 (+1137431us) service_pool.cc:224] Handling call
0709 12:26:32.731065 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.731291 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:32.731562 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.930040 (+198478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.930089 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:32.930132 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.930197 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.930310 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":947,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":75,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4527,"thread_start_us":197056,"threads_started":1}
I20260709 12:26:33.118837 26120 rpcz_store.cc:276] 0709 12:26:31.539086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.505059 (+965973us) service_pool.cc:224] Handling call
0709 12:26:32.505301 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.548871 (+ 43570us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:32.549133 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.704121 (+154988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.704220 (+    99us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:32.704453 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.117380 (+412927us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:33.117490 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.117907 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":388,"prepare.queue_time_us":7094,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1977,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8086,"spinlock_wait_cycles":99611520,"thread_start_us":154217,"threads_started":1,"wal-append.queue_time_us":255184}
I20260709 12:26:33.122699 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2634) took 1587 ms. Trace:
I20260709 12:26:33.123272 26128 rpcz_store.cc:276] 0709 12:26:31.534812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.818918 (+284106us) service_pool.cc:224] Handling call
0709 12:26:33.122108 (+1303190us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:33.122450 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014359424}
I20260709 12:26:33.144614 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:33.146574 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.163033 25946 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:33.168044 25946 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:33.172274 25946 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 15, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 15 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:33.175792 26128 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:33.176033 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2582) took 1567 ms (client timeout 1943 ms). Trace:
I20260709 12:26:33.176401 26196 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:33.176594 26198 rpcz_store.cc:269] 0709 12:26:31.608114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.625618 (+ 17504us) service_pool.cc:224] Handling call
0709 12:26:33.175458 (+1549840us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b54002b3754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:33.175752 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61470080}
I20260709 12:26:33.177259 25946 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:33.178382 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.179626 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.179760 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.180598 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:33.180053 25946 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:33.183815 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.002960 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.002943 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.189988 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.191041 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.191663 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.192277 26785 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:33.193526 26784 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:26:33.195309 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:33.194594 25946 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 13, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 13 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:33.197001 25946 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 13 to 15, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 15 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:33.200582 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.180457 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.178619 26194 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:26:33.201129 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.202433 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.005455 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:33.203725 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.190008 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.193403 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.204640 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.205659 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.208527 32546 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:33.208737 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.209551 32551 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:33.211104 32699 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:33.212438 32708 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:33.212539 32706 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:33.216013 32370 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:33.253975 32370 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:33.254060 32706 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:33.254523 32370 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:33.256937 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.258100 26784 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:33.258754   343 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.262979 32546 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:33.263355 32563 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.264928 32563 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:33.264797   411 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:33.265393 32563 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:33.265246 32546 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.266876 32563 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.216851 32638 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.216972 32493 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.217921 32721 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:33.218192 32697 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.222435 26194 pending_rounds.cc:85] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 13
I20260709 12:26:33.284770 26194 pending_rounds.cc:107] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:26:33.285450 26194 raft_consensus.cc:2889] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.218223 32583 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:33.225248 32683 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:26:33.239895   315 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:33.244813   351 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:33.245584 32670 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:33.250241 26789 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:33.250545 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.301782   351 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.305662 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:33.306084 32583 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:33.306468 26782 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15. Re-sending same reply.
I20260709 12:26:33.307812 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.308710 26124 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:33.301858 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.270354 32563 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.273569 26046 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:33.270848 26199 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:33.321264   315 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:33.321621 32493 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:26:33.234406 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 86203.1us
I20260709 12:26:33.256007 32706 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.306020   351 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.285450 26120 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:33.307530 32583 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.329684 32493 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:33.342989 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 236594us
I20260709 12:26:33.321265   354 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:33.351226   354 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:33.353000 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.352764   354 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.353979 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:33.353240 32638 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.354908   354 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.354548 32493 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.355010 26124 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:33.355768 32708 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:33.356361 32493 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.356992 32637 raft_consensus.cc:2764] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:26:33.325531   315 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:33.368335 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:33.369850 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:33.372125 32559 raft_consensus.cc:2764] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:26:33.355186 32638 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.372435 32367 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.372130   343 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:33.373917   343 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:33.374348   343 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:33.374564 32638 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:33.371345   315 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.378050 26128 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:26:33.388618 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.389870 26124 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
W20260709 12:26:33.390959 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 218774us
I20260709 12:26:33.391808 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:33.408556 32559 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:33.412578 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:33.413053   315 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.413837 32638 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:33.412611 32697 consensus_queue.cc:1175] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 19 status { last_received { term: 19 index: 13 } last_committed_idx: 12 last_received_current_leader { term: 19 index: 13 } }
I20260709 12:26:33.414462 32638 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:33.416092 32638 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.421082 32367 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.422600   420 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:33.423059   420 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:33.424566   420 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:33.426440 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45665.7us
I20260709 12:26:33.429625 26046 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: [14.11-18.13]
I20260709 12:26:33.433827   446 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:33.434763   446 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.436736 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:33.437489 26128 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.436754 25946 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 3 to 12, term changed from 14 to 16, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:33.439911 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1070) took 1436 ms. Trace:
I20260709 12:26:33.440367 26194 rpcz_store.cc:276] 0709 12:26:32.003681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.007999 (+  4318us) service_pool.cc:224] Handling call
0709 12:26:33.177736 (+1169737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.286369 (+108633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.439676 (+153307us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002b3750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:33.439838 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2688892288}
I20260709 12:26:33.440507 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.463070 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:33.470733 26120 raft_consensus.cc:2376] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19. Re-sending same reply.
W20260709 12:26:33.485608 26786 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.014s
I20260709 12:26:33.485831 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:33.486270 26786 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:33.490281 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:33.489133 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 97662.5us
I20260709 12:26:33.494208 32493 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:33.494944 32493 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.495443 32493 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.500658 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:33.501442 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 237464us
I20260709 12:26:33.502540 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.505836   446 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.505937   378 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:33.508423 32721 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:33.509003 32721 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:33.510448 32605 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.009s
I20260709 12:26:33.510255 32721 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.510910 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:33.500916 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71053.4us
W20260709 12:26:33.513973 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 305673us
I20260709 12:26:33.494467 26124 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:33.524247 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:33.528820 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.531199 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:33.528584 32583 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:33.537005 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.538061 26124 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:33.538164 26050 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:33.462919 32370 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:33.545688 32741 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.001s
I20260709 12:26:33.546317 32741 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:33.547485 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.554041 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.570057 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.569743 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.510948 32605 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:33.584643 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 63730.5us
I20260709 12:26:33.587558 26120 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:33.569137 32559 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.577773   378 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:26:33.592803 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.567169 32637 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:26:33.598457 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:33.585777 32583 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:33.600662 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.603724 26120 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:33.606397 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64767.1us
I20260709 12:26:33.611852 26124 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:33.617801 26052 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:33.620085 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:33.628610 26194 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.628942 32741 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.630424 26202 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: [14.12-17.13]
I20260709 12:26:33.637633 26199 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: [17.12-18.13]
I20260709 12:26:33.641603 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:33.614342 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:33.642742 26120 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:33.645015   343 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.646157 32559 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:33.647445 32559 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:33.647795   305 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.648818   306 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:33.649233   306 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:33.654790 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.656448   303 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:33.606992 32370 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.671746 26046 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:33.673069   410 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:33.673604   466 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:26:33.673673   410 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:33.675232   410 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:33.676440 32738 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.701s	user 0.004s	sys 0.014s
I20260709 12:26:33.677042 32738 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15.
W20260709 12:26:33.677256 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 112123us
W20260709 12:26:33.677055 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 86715.8us
I20260709 12:26:33.681413 32605 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.660773 32615 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:33.686964 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.688750 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.692410 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.692672 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.693825 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.696223 26120 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:33.696583 26124 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:33.700438 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:33.707543 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.708825 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.709369   351 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.710841 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.711956 32738 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.712829 32738 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:33.707394 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.722090 26122 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:33.723831 26046 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.12-22.12]
W20260709 12:26:33.728603 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66549.4us
I20260709 12:26:33.732045 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.737566 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.742935 32583 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:33.743462 32583 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.743875 32583 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:33.744843 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:33.745997   470 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:26:33.746951   303 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:33.747118 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.747581   303 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:33.748257 32637 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.748816   303 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.749321 25946 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 14 to 17, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 17 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:33.750756 32637 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:33.751756 26202 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:33.751030 25946 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 14, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:33.754761 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.748137 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:33.740002 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.762694 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.763834 26124 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:33.766518   420 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:33.754747   469 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:33.769119 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:33.779361 26128 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:33.768837   410 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:33.786257 26199 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:26:33.788734 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48999.7us
I20260709 12:26:33.792158 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.793159 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.782871 32370 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:33.761093 32493 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:33.805011 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56967.1us
I20260709 12:26:33.786000 32738 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:26:33.821094 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81174.9us
I20260709 12:26:33.823279 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.824487 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:33.825001 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:33.840723 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 86090us
I20260709 12:26:33.801136 32563 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:26:33.843989 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 147160us
I20260709 12:26:33.844817 26127 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 13: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:33.843760 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3526) took 1177 ms. Trace:
I20260709 12:26:33.848805 26202 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:33.851004 26128 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 13.
I20260709 12:26:33.849722 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:33.848708 26052 rpcz_store.cc:276] 0709 12:26:32.658874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.741850 (+ 82976us) service_pool.cc:224] Handling call
0709 12:26:32.742227 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.618462 (+876235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.802243 (+183781us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:33.836635 (+ 34392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974752000}
I20260709 12:26:33.847113 32759 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:33.855028 32759 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:33.855307   464 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:33.857357 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.858624 32759 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.859834 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.839954 32370 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:33.860615 26126 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:33.861728 32493 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.861238 32370 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:33.851536 26129 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:33.873060 32563 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:33.876995 32563 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.878142 32583 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.881565 32699 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:33.882314 32699 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:33.884276 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.888588 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:33.889510 26121 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:33.890252 32706 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:26:33.891464 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.908977 26194 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:33.921588 32721 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:33.921967 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:33.943192 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:33.923251   354 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:33.933423 32583 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.955717 26046 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.916432 32367 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:33.962088 32493 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:33.963973 32493 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:33.964445 32493 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:33.966130 32493 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:33.967815 32493 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:33.969537 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.970230 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.971316 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.971946 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.972024 26201 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:33.973419 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:33.971971 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.982707 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.983531 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:33.983906 26196 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:33.984202 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:33.985240 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:33.953272 32708 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:33.986792 32708 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:33.987845 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:33.989470 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 67612us
I20260709 12:26:33.956720 32615 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:34.001091 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:33.945439   305 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:26:34.007294   354 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:34.010596 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:34.003850   351 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:33.987584 32741 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.021632 26128 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:34.022138 26121 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.041692 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:34.043519 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:34.023382   354 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:34.046833 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:34.038203 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.051806 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:34.055038 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.055882 26129 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:34.056588 26789 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.065068 26783 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:34.033419 32741 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.065651 26783 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:34.081115 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.090320 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.106596 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.089402 32546 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.115044 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 101228us
I20260709 12:26:34.105473 26783 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.119457 26046 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:34.121140 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.123193 26123 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:34.128568 26783 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.114710 32699 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.129930 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.135293 26128 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:34.138316 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.144810 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:34.146219 26127 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:34.117511 32546 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.162451 32546 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.151746 32741 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.107142 32708 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:34.152138   475 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:26:34.153343   476 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:34.172978 26789 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:34.174957 32639 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:34.176419 32638 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:34.176967 32638 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:34.176776 32639 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.177408 32638 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.181046 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.171444 32583 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:34.185443 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57727.4us
I20260709 12:26:34.187124   306 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.188208   306 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.195554 32741 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.174187   420 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:34.201179 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:34.206202 25946 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 16, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 16 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:34.174485 32699 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:34.205392 32741 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:34.209237 26124 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:34.208729 26122 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:34.213485 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:34.220382 32699 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.223174 32699 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.201206   420 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:34.228633   420 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.235318 26121 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:34.243173   343 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:34.246604 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.256217 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.258445 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.259337 32605 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:34.259938 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.250732 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.249069 25946 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.261237   351 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:34.262584 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.263325 26121 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:34.264856 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.265611 26121 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:34.266476 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.253854 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.267136 32638 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.268169 32563 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:34.254295 26783 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:26:34.268757 32563 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:34.269021 26783 pending_rounds.cc:85] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 12
I20260709 12:26:34.215641 32639 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:34.269587 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.269722 32563 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.270413 32639 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:34.270650 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.271049 32563 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.256539 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.256819   343 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.269590 26783 pending_rounds.cc:107] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:26:34.273132 26783 raft_consensus.cc:2889] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.273900 26122 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:34.274791 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:34.276252 26203 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:34.275897 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:34.277621 26121 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:34.277920 26194 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:34.260046 32605 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:34.266557 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 289724us
W20260709 12:26:34.283300 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.252858 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:34.261981 25946 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 6 to 16. New cstate: current_term: 16 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:34.285974 26125 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:26:34.285642 25946 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 14 to 17, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 17 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:34.286813 26046 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:26:34.286988 26121 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:34.287230 26046 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 11
I20260709 12:26:34.287632 26046 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:26:34.287894 26046 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.294874   354 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.280776 26126 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:34.261940 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:34.256493 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.248627 26129 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:34.251797 32759 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.280575 26128 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:34.277912 26122 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:34.287276 25946 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.305124 26120 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 12:26:34.306902 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50463.7us
I20260709 12:26:34.307444   351 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:34.305927 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.308533   343 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:34.302471   410 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.000s
I20260709 12:26:34.326356 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.308691 32370 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:34.324900 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:34.336484 26789 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:34.336963 26789 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:34.337618 26202 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:34.343106 26196 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:34.343427 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:34.324349   351 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:34.336172 32367 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:34.355269 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49468.8us
W20260709 12:26:34.356428 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 151888us
I20260709 12:26:34.345292 26789 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.331840 32583 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:26:34.348701   351 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.356468 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.349691 32370 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.374648 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49834.6us
I20260709 12:26:34.377890 26120 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:34.378499 26122 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:34.379037 26202 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:34.380119 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:34.383045 26122 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:26:34.389350 26046 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:34.357825 32741 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:34.391324 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140711us
I20260709 12:26:34.387243 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:34.363646   305 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:34.365481 32546 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:34.395105 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68811.3us
I20260709 12:26:34.397477 32546 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.386456 32370 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.402997 32546 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:34.404615 32370 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.405664 32370 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:34.406044 32370 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.406380 32370 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:34.408329 26054 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:34.373497 32637 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:34.398059 32741 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:34.412714 32741 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.397598 32615 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:34.392198   343 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.412756 26789 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.421403 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:34.424118 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49827us
I20260709 12:26:34.425414 26129 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:34.431458 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47042.1us
I20260709 12:26:34.440214 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:34.441324 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:34.440266 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83921.6us
I20260709 12:26:34.427311   305 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:34.447108   305 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.448127 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:34.449051 32639 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:34.449491 32639 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:34.449944 32639 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:34.449864 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.452636 26127 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:26:34.453068 26127 pending_rounds.cc:85] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 9
I20260709 12:26:34.453323 26127 pending_rounds.cc:107] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:26:34.453578 26127 raft_consensus.cc:2889] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.444962 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:34.457291 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:34.480405 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57296.1us
I20260709 12:26:34.438872 32638 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:34.460796   490 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:34.492323 26167 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.699s (SENT)
I20260709 12:26:34.456499 32738 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:34.498030 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100794us
W20260709 12:26:34.498605 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55450.1us
I20260709 12:26:34.513378   410 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:26:34.481633   488 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:26:34.485816 32637 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:34.514338 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:34.521063 26789 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:34.517163 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.502060 32697 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:34.526064 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76264.9us
W20260709 12:26:34.524175 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.943s (SENT) [suppressed 33 similar messages]
W20260709 12:26:34.527534 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.530612   495 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:34.532558   367 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.094s
I20260709 12:26:34.519505 32637 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.522629 32638 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:34.545958   351 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:34.546787   494 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:34.538813   315 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:34.547730   315 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:34.548135   315 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:34.548815   492 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:26:34.525302 32697 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:34.531097 32639 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.556857 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:34.557704 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.553539 32697 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:34.559118   489 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:26:34.543640 32637 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.548784   351 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:34.551335   497 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.526198 32738 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.540653 32546 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.554890   490 raft_consensus.cc:2764] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:34.588311   490 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:34.555078 32638 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.589836 32639 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.590484 32639 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:34.590876 32639 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:34.591274 32638 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:34.601078 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.552898   378 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:34.576905   343 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:34.604521 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 119276us
I20260709 12:26:34.565263   495 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:26:34.591760 32639 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.610911   420 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.598881   497 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.613685 32546 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.613358 32697 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:34.636526 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84758.7us
I20260709 12:26:34.606667   343 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:34.706024 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70903.8us
W20260709 12:26:34.711454 26167 leader_election.cc:336] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.702s (SENT)
I20260709 12:26:34.711961 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:34.605198   410 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
W20260709 12:26:34.713233 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 274688us
W20260709 12:26:34.704672 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.715248 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.740128 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 202667us
I20260709 12:26:34.742120   378 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:34.742494 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.742652   497 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.744217   503 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:26:34.712620   343 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:34.755283 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 146423us
I20260709 12:26:34.757284 26789 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:26:34.799853 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.800868 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 89448.1us
I20260709 12:26:34.802755 32697 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.804646   498 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:34.822834 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:34.824285 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 223624us
I20260709 12:26:34.830706 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.836932 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:34.848937 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:34.826306   485 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.880942 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:34.880137 26194 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:34.837939   512 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:34.843605   378 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:34.895888 26195 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:26:34.896513 26195 pending_rounds.cc:85] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 11
I20260709 12:26:34.896814 26195 pending_rounds.cc:107] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:26:34.897117 26195 raft_consensus.cc:2889] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.898272 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:34.906311 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 77967.8us
I20260709 12:26:34.907712 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:34.907074 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 144875us
W20260709 12:26:34.916318   507 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:26:34.917716   515 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.918350   515 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.920073   343 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:34.920282 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:34.920974   506 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:26:34.921293 26203 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:34.922778   514 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:34.922616   343 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.936551   519 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:34.937433   519 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:34.938813   519 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:34.932204 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 175677us
I20260709 12:26:34.920361   488 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:34.953470 26018 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.049s (SENT)
I20260709 12:26:34.954209 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:34.932252   420 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:34.970171   515 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:34.970824   515 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:34.971441   515 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.972651   420 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:34.984601   513 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.114s	user 0.002s	sys 0.000s
I20260709 12:26:34.984889   515 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:34.905784   512 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:34.987799 26203 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
W20260709 12:26:34.989411 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81797.7us
I20260709 12:26:34.990964 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:34.993470 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.891794   485 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:35.001623 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.951648   488 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:35.003084 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.004422 26203 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:34.993170   515 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:26:34.997026   420 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:34.990129   410 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:35.008100 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 209297us
I20260709 12:26:35.011323 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:34.994334   512 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.012049 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:34.992218 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50792.3us
I20260709 12:26:35.033090 26050 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:35.013391 32706 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:35.013876 26048 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:35.014253 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.014449 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:35.034358 26047 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.015080 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.016608 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.017304 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:35.035418 26045 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.019598 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:35.020188 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.020892 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:35.022006 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.018554 32639 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.024514 26200 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:26:35.044613 26200 pending_rounds.cc:85] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 15
I20260709 12:26:35.045012 26200 pending_rounds.cc:107] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:26:35.045288 26200 raft_consensus.cc:2889] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.024828 26783 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:26:35.046351 26049 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:35.047649   517 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.048002   495 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:35.048314   517 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.049026 26199 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:35.049414 26199 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:35.050205 26199 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.050727   517 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.051230 26199 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:35.052434 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.053131 26051 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 13.
I20260709 12:26:35.053865 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.054036 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.054319 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.055051 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.055506 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:35.025187   420 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:35.026176 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.027758   515 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.020212 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:35.005322 32370 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.599s	user 0.014s	sys 0.000s
I20260709 12:26:35.023722 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:35.038121   512 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:35.038518   490 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:35.039619   516 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:35.040127   511 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:35.036793 26120 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:35.001968 32708 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.006254   488 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:35.056082 26051 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:35.063189 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.109062 26045 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:26:35.140708 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77761.1us
I20260709 12:26:35.141523 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:35.143199 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.142305 32708 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.150621 26045 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:35.150480 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:35.158383 26045 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:26:35.113794 26047 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:26:35.110466 26046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.025s	sys 0.067s
W20260709 12:26:35.159997 26046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.025s	sys 0.067s
I20260709 12:26:35.131035 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:35.131886 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 91945.2us
I20260709 12:26:35.163663 26199 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:26:35.164220 26199 pending_rounds.cc:85] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 13
I20260709 12:26:35.164626 26199 pending_rounds.cc:107] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:26:35.164997 26199 raft_consensus.cc:2889] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.148653 32638 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:35.146261   354 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.126662 32370 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.115864   515 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.194245 26045 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:35.155820   517 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:35.166797 32546 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:35.156545 32741 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:26:35.210196 26049 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
W20260709 12:26:35.216033 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59080.8us
I20260709 12:26:35.197819   517 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.198130 32706 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:35.239773 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.241015 26018 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.323s (SENT)
I20260709 12:26:35.218478 32546 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:35.223122   517 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:35.255612 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94388.5us
I20260709 12:26:35.236306   303 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.205101   529 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
W20260709 12:26:35.257426 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.012s
W20260709 12:26:35.261040 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.004s	sys 0.012s
I20260709 12:26:35.244297   511 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:35.261478 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 555956us
I20260709 12:26:35.255700   516 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:35.232844 32706 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.275782 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.214274   420 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.280043 26196 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:35.283823 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:35.284579 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.283722 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:35.285456 26194 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:35.265064   485 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:35.301369 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 448988us
I20260709 12:26:35.263379   511 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:35.302419 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:35.261705   303 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.270946   516 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:35.306418 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 85352.5us
I20260709 12:26:35.311367 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:35.317402 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.319531 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 206882us
I20260709 12:26:35.321154 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:35.323038 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61090.5us
W20260709 12:26:35.326747 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.329432 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.302211   518 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.124s
I20260709 12:26:35.335718 32699 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:35.336329   540 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:35.336920   540 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:35.338297   540 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.339636   303 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:35.340188   530 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:35.340728 32563 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:35.341171 32563 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:35.341897   528 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:26:35.342685 32563 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:35.344520   524 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:26:35.348304 32741 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:35.349902 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:35.349999 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:35.336328 32699 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:35.344417   543 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:35.354197 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:35.363255 26047 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:35.367656 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 252228us
W20260709 12:26:35.368304 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.369810 26167 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.518s (SENT)
I20260709 12:26:35.352639   303 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:35.331229 32370 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.374094 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.362857   543 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.376839   543 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.380347   536 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.381111 32699 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:35.382117   531 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:26:35.382447 32699 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.370162 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:35.339787   540 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.341174   535 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.362035 32563 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:35.363535   515 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.411166 26051 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:35.423780   532 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:35.424391 26167 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.009s (SENT)
I20260709 12:26:35.424731   532 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:35.426000   532 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:35.393867   315 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.845s	user 0.013s	sys 0.000s
W20260709 12:26:35.429256 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.402642   305 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:35.433531 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:35.395684 32741 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:35.441627 26199 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.330999 32546 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.446274 26200 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:35.448534 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.454416 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.462785 32605 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:35.463472 32605 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.380985   536 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.394546   520 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:26:35.432452   315 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:35.471091 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 77670.6us
W20260709 12:26:35.475050 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 79495.5us
I20260709 12:26:35.483700   552 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:35.484609 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.444476 32741 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:35.466910 32637 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.885s	user 0.009s	sys 0.004s
I20260709 12:26:35.385584   516 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.478822 26047 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:35.412266   485 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:26:35.386200   511 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.484423   552 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:35.504531 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.405395   535 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.543190   542 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.140s
W20260709 12:26:35.630519   546 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.159s	user 0.000s	sys 0.006s
I20260709 12:26:35.481282   519 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:35.480520   520 consensus_queue.cc:1243] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:35.518291 32637 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.629858 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.503162 32741 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:35.519290 32367 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.632558 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 202069us
I20260709 12:26:35.478294 26199 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
W20260709 12:26:35.659394 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.679131   543 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.680289   519 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.682139   536 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.630357   488 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.689867 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.665844 32370 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:35.720757   519 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.672437   490 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 12:26:35.710637 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80854.8us
I20260709 12:26:35.726014 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.726771   535 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.727396   535 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:35.684073   517 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.704362   542 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:35.718278 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 243805us
I20260709 12:26:35.719381   538 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:35.692888 32367 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.729537   517 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.724771   543 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:35.717588 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:35.723500 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.733548   488 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.727294 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.708765   552 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:35.748984 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.752732 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.743086 32370 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:35.755571 25946 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:35.756363 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.735744 32367 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.757151 25946 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 10 to 14, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 14 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:35.758908 25946 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 17 to 19, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 19 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:35.762451 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:35.772444   488 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:35.774153   553 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:35.775372 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:35.782418   517 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:35.782879 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:35.783402 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:35.782472   510 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:35.785245 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.785506   510 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:35.786412 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.786669 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.787665 26201 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 20 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:35.788368   510 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:35.788663 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.788790 32697 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:35.789273 26202 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:35.789431 32697 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.783198   517 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:35.791620 32697 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.792631   517 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.774065 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.793430   517 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:35.794094 32708 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.794318   517 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.794974 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.795780   351 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:35.795019 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:35.799733 32738 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.800092   351 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:35.800580   351 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.800917   351 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.800990 32706 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.803676 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.811975 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:35.804409 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.816210 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.844501 26124 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:35.802770 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:35.848178 26125 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:35.802506   542 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:35.853032 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.853367   542 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.853941 26126 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:35.854811 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:35.855609   542 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.855876 32708 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:35.856699 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.848290 32738 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:35.858167 32738 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:35.859030 32605 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.859158 32738 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.860373 32493 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:35.853039 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.860965 32493 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.861378 26125 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:35.862660   511 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.863411 32741 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.865661   502 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:26:35.869494   517 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.851208   495 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:35.852123 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:35.870652   495 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:35.871593   495 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.872560   498 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:35.873191   498 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:35.874814 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72133.4us
I20260709 12:26:35.875865   315 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:35.876643   534 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:35.878257 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.878888   533 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:35.879316   533 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:35.880067 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:35.878914 26120 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
W20260709 12:26:35.884390 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 253807us
I20260709 12:26:35.888399 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:35.888999 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 192064us
W20260709 12:26:35.892508 26018 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.374s (SENT)
I20260709 12:26:35.899771 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:35.900496 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:35.901407 26052 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.892082 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:35.900640 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.889211 26053 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:35.905066 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:35.907030 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:35.907459 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.908146 26789 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:35.902990 26049 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:35.863317 32493 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:35.922940 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:35.923216 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:35.932739 32370 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.933835 32370 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.937594   537 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.938413 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.938305   537 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:35.939371 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.940033 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.940356 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.940683 26783 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19. Re-sending same reply.
I20260709 12:26:35.942109 26786 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:35.942042 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.943852 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.944012   537 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:35.945014 26791 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:35.964515 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.021s	sys 0.028s
W20260709 12:26:35.965304 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.021s	sys 0.028s
I20260709 12:26:35.965375 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:35.966460 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.968711 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.966421 26120 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:35.964545 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43832.1us
I20260709 12:26:35.848510   552 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
W20260709 12:26:35.986972 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 92577.9us
W20260709 12:26:35.995518 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.899458   502 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.998687 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75573.6us
I20260709 12:26:35.899679   517 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.997841   542 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:35.882418   503 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
W20260709 12:26:36.004567 26167 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.912s (SENT)
I20260709 12:26:36.010216 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:35.877709   498 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.010375 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:35.997694 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:35.984768   525 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.019989 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.021476 26786 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:26:36.035895 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.011610   519 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:36.042512 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44976.3us
I20260709 12:26:36.044307 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.016463 32708 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:36.048897 26045 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:36.049633 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:36.049229 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.046284 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.021384   525 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:36.052021 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:36.053434 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.044190   536 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:35.880089   533 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.057654 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:36.090953 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 198451us
I20260709 12:26:36.046486   519 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:36.061218 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.056957 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.099260 26047 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:36.015359   517 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.057662   548 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.054862   563 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.019054   502 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.061456   525 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.057421   542 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.057108 32367 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:26:36.100873 26018 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.189s (SENT)
I20260709 12:26:36.101104 32546 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.122490 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.097369   519 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:36.112088 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57475.9us
I20260709 12:26:36.125450 32546 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:36.129652 32546 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.113991 32759 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:36.137383 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.124930   542 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.126552   543 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.125952   521 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.126296   520 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.148797 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56563.5us
I20260709 12:26:36.112150   533 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:36.153390   560 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:26:36.153465 32605 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:36.154330 32605 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.154961 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.156991 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.146428   542 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.158409 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.136559 32759 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.147833 32367 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.148218 32697 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:36.166898 32697 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.138986   536 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.168295 32759 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:36.171329 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.172789 32367 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:36.181090 32367 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.163774   503 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:36.173712   502 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.159946   521 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.150803   543 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.190900 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:36.176263 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.172811 26791 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:36.150043 32615 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:36.193037 26122 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:36.195693 26120 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:36.165730 32741 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.197117 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:36.199481 26121 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.195777 26052 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:36.205286 26786 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:36.206349 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.197268 32615 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.208784 26122 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:36.210057 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.151710   520 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:36.200425   562 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:26:36.214329 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.214211 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.217373 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.218120 26786 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.213667 26052 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:36.225550 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.226837 26122 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.226235 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.214067   538 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.189636   503 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.232810   303 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.214002   343 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:36.224462   568 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:26:36.242360 26128 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
W20260709 12:26:36.243175 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.243611 26789 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:36.247758 26789 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:36.247751 26122 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:36.245371 26786 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:36.254148 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:36.254724   564 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:26:36.233460   303 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.255576   512 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:36.256016   512 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:36.251607 26121 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:36.259618 26127 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:36.259259 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.257037   512 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.261602 32615 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.261831   512 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.262144 32637 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.268405 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.269039 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.269189 26200 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:36.269774 26195 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:36.269891 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.259145 26789 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.285280 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:36.286100 26203 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
W20260709 12:26:36.246381   569 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:26:36.288631 26789 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.257362   303 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.292634 26121 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:36.294451 26046 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:26:36.298342 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:36.318485 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.319195 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.320376 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.325031 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.326370 26047 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:36.247308   532 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:36.336762 26050 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:36.338064 26121 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:26:36.327188 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:36.328425 26122 raft_consensus.cc:1217] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 13.7->[15.8-18.10]   Dedup: 15.8->[16.9-18.10]
I20260709 12:26:36.339146 26122 pending_rounds.cc:85] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 8
I20260709 12:26:36.339437 26122 pending_rounds.cc:107] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:26:36.339344 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:36.339769 26122 raft_consensus.cc:2889] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:36.342068 26046 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:36.347494 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.348420 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:36.340580 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.340222 26121 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:36.346545 26050 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.243660   351 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.353287 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:36.357029 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.357434   543 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:36.358439 26756 connection.cc:256] RPC call timeout handler was delayed by -1.505192062000333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
W20260709 12:26:36.358875 26756 connection.cc:256] RPC call timeout handler was delayed by -1.0731920620000892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 12:26:36.359012   503 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.360395   512 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.360936   512 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.365190   520 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.363102   512 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.366372   521 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.349387   532 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.368603   351 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.368959   532 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.370500 26125 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:36.359961 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.371644 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.372426   537 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.377750 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:36.378628 26051 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:36.379729 26054 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:36.299736   517 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:36.341463   571 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
I20260709 12:26:36.384859 26756 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:36.387017 26125 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:36.387894 26786 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:36.389535 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.390323 26786 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:36.391798 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.385357 26050 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:36.392565 26786 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:36.393807 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.393445 32367 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.252628   343 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.396998 32367 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.398183   343 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.398614   527 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:36.399534   527 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:36.399901   527 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:36.400898   527 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.402153   527 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.244820   538 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.405225 26789 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:26:36.405616 26789 pending_rounds.cc:85] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 13
I20260709 12:26:36.405833 26789 pending_rounds.cc:107] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:26:36.405951   538 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.385560 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.408281 26789 raft_consensus.cc:2889] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:36.410496   542 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.411849 26123 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:36.412904 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.414589 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.414616 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.415903 26789 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:36.418998 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.419193 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.419759 26200 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:36.419844 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.420250 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.420655 26783 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:36.420846 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.420879 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:36.421597 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.421726 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.422547 26129 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.423032 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.423760 26203 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:36.426496 32697 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.428398 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.429235 26784 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 16: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:36.429774   573 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.201s	user 0.002s	sys 0.000s
I20260709 12:26:36.430056   538 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:36.430783 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.430735   538 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.431572 26785 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:36.432631 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.433012   538 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.434923 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.436043 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.437175 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.437367   516 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.438120 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:36.439126 26050 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:36.439018 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.439586 26789 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:36.439929 26785 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:36.440454 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.440714 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.441071 26789 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:36.441252 26785 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:36.441915 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.430029 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.442509 26789 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:36.443656 26784 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:26:36.438225 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.444267 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.446717 26050 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.450834 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.446714 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:36.452278 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.452489 26198 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.454066 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.455183 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.460860 26047 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:36.463819   420 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:36.464608 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.469323 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.471486 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.472434 26203 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:36.473146 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.475337 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.476676 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.479782 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.495288 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:36.495785 26167 leader_election.cc:336] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.505s (SENT)
I20260709 12:26:36.488561 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.502480 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.502966 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
W20260709 12:26:36.486545 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.478360 26756 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:36.504561 26756 leader_election.cc:403] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:36.479254 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:36.514004 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.514727 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.515296 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:36.487035   511 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.003s
I20260709 12:26:36.516659 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.518563 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.519366 32367 raft_consensus.cc:2764] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:36.520110 32367 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.520532   536 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:36.516594 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.519832 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.525166 26791 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:36.523523 26790 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:26:36.530472 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.925201 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:36.531505 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.531407 25946 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:36.532404 26784 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:36.507889 32638 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:36.533697 32638 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:36.534148 32638 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.533454 25946 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 14, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 14 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:36.534404 32370 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.511022   537 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18.
I20260709 12:26:36.489327   420 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:36.535379 25946 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:26:36.497768   527 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:36.531315 26788 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:36.555408 26198 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:36.557317 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.560302   542 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:36.619735 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.624691 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.542095   554 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:36.626139 26790 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:36.627693 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:36.624512 26788 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:36.640650 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.654122 26053 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:36.655488 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.631536 26782 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
W20260709 12:26:36.668298 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 148516us
I20260709 12:26:36.668960 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.632157 32367 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:36.672012 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.626412   535 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:36.625734   411 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:36.530309   525 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.678004 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:36.642310   519 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.507s	user 0.014s	sys 0.007s
I20260709 12:26:36.682723 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.682556 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.687323 26790 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:36.688517 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:36.685941 26788 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:36.703715 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:36.681836 32546 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.522s	user 0.005s	sys 0.005s
I20260709 12:26:36.711542   411 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.714291   411 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.715179 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:36.716168 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:36.723487 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.738464   508 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 2.024s	user 0.008s	sys 0.009s
W20260709 12:26:36.740157   580 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:26:36.710831 32546 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.743366 26127 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.743417 26790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:53642
I20260709 12:26:36.744592 26790 raft_consensus.cc:606] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.744997 26790 raft_consensus.cc:614] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:36.766568 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:36.768517   517 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:36.769804 32741 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:36.770232 32741 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.770584 32741 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.773288 32605 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.709484   554 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 12:26:36.682462   535 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.776289 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.777633 26757 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.778342   535 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.781549 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:36.783568   558 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:36.784366   558 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:36.785416 25945 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 192) took 39736 ms (client timeout 15000 ms). Trace:
I20260709 12:26:36.785777   558 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.788447   543 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:36.785924 25945 rpcz_store.cc:269] 0709 12:25:57.048879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.116805 (+ 67926us) service_pool.cc:224] Handling call
0709 12:26:36.785152 (+39668347us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:28.788689 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:28.789336 (+   647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:28.789388 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:26:28.789406 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:28.790821 (+  1415us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:28.790945 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:28.790976 (+    31us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:26:28.867292 (+ 76316us) tablet.cc:701] Row locks acquired
0709 12:26:28.880491 (+ 13199us) write_op.cc:260] PREPARE: finished
0709 12:26:28.921326 (+ 40835us) write_op.cc:270] Start()
0709 12:26:28.952596 (+ 31270us) write_op.cc:276] Timestamp: P: 1783599988891630 usec, L: 0
0709 12:26:28.966599 (+ 14003us) op_driver.cc:348] REPLICATION: starting
0709 12:26:29.093177 (+126578us) log.cc:844] Serialized 6345 byte log entry
0709 12:26:30.207961 (+1114784us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:30.574747 (+366786us) op_driver.cc:464] REPLICATION: finished
0709 12:26:30.671614 (+ 96867us) write_op.cc:301] APPLY: starting
0709 12:26:30.671735 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:31.577555 (+905820us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:31.577605 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:31.785653 (+208048us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:31.786025 (+   372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:26:31.786096 (+    71us) write_op.cc:312] APPLY: finished
0709 12:26:31.786528 (+   432us) log.cc:844] Serialized 148 byte log entry
0709 12:26:31.894876 (+108348us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:31.895146 (+   270us) write_op.cc:454] Released schema lock
0709 12:26:31.895282 (+   136us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:31.896001 (+   719us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:31.896040 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":443533,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":59845,"apply.run_cpu_time_us":56236,"apply.run_wall_time_us":1224307,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71103,"num_ops":16,"prepare.queue_time_us":419363,"prepare.run_cpu_time_us":37339,"prepare.run_wall_time_us":684400,"raft.queue_time_us":84638,"raft.run_cpu_time_us":13867,"raft.run_wall_time_us":181086,"replication_time_us":1590631,"spinlock_wait_cycles":16780032,"thread_start_us":844506,"threads_started":5,"wal-append.queue_time_us":626672,"wal-append.run_cpu_time_us":15361,"wal-append.run_wall_time_us":2873300}]]}
I20260709 12:26:36.790195   537 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:36.689714   519 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:36.793895 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 77888.6us
I20260709 12:26:36.794909 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.748283   411 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:36.716140   410 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:26:36.802927 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.810127 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.792685   554 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:36.761902 32546 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.801765   521 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.818128 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.812496   538 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.820973   411 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:36.812543   532 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:36.805683   533 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:26:36.823513   533 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:36.823907   533 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:36.818832   305 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:26:36.825055   305 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:36.824880   533 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.825654   305 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:36.826496   533 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.826543   305 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:36.828637   583 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:26:36.829784   411 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:36.830797 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.830925   411 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.786437 32741 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.827940   305 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.814965   554 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:36.836606 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:36.851454 26786 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:26:36.813916   537 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:26:36.836004   520 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.874925 26125 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.851436   536 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:36.834867   521 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:36.893803   521 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:36.895622   521 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:36.891729   589 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.149s	user 0.000s	sys 0.004s
I20260709 12:26:36.896088   521 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.862078   520 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:36.910526   554 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.899215 32638 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:36.913755 32367 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:36.913954 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 236057us
I20260709 12:26:36.918607 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:36.919495 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 82957.2us
W20260709 12:26:36.921090 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 144961us
W20260709 12:26:36.916023 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 100281us
I20260709 12:26:36.920377 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.916715   520 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.921696 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:36.901396   525 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.916469   521 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.932904 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.934937 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.936353 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.921020   519 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:36.958536 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.927858   509 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:36.927456   343 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.959111 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:36.928685 32741 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:36.965986 26018 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.946321   498 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.966400 32605 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:26:36.971621   527 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.948402   420 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.947889 32367 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:36.971335   498 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:36.960754   515 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:36.983683 32638 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:36.972028 32706 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:26:36.972183   527 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:36.960873   525 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:36.998242 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3639) took 1009 ms. Trace:
I20260709 12:26:36.976923   420 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:37.000981 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:37.036165 26790 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:37.037431 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.003072 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:36.983129   367 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:37.018136 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:37.009723   343 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:37.020941 26120 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:37.041987 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.042687 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:37.027709 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:37.028416 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:36.961252   554 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.044674 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:37.020555 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.032446 26127 rpcz_store.cc:276] 0709 12:26:35.988655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.286316 (+297661us) service_pool.cc:224] Handling call
0709 12:26:36.286625 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.743867 (+457242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.997915 (+254048us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:36.998166 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1049942272}
I20260709 12:26:36.985865 32367 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:37.038301 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.039080 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.044907 26124 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:37.032393 26045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:26:37.054884 26045 raft_consensus.cc:606] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.055433 26045 raft_consensus.cc:614] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:37.039130   420 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.032426   527 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.031363   515 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:37.041277   509 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:37.065994 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.073427 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:37.073845 26790 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:37.014097   548 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:26:37.041440 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:37.082532 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.050036 32638 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.051810 32367 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:37.061048   516 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.091442 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.099043 26129 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
W20260709 12:26:37.093961 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54981us
I20260709 12:26:37.085736 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.101761 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.079586   343 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:37.089321   511 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:26:37.084517   532 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.115298 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:37.117931 26129 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.093474   512 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:37.108090 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 187814us
I20260709 12:26:37.080833   525 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:37.095449 32370 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:37.105526   509 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:37.121740 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.105365   538 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:37.108326   537 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:37.138145   525 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:37.138767   525 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:37.144582   515 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.129062 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.130158   532 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:37.137027 26129 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:37.155009 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.124529   519 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.120662   512 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.162971 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:37.167606 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:37.172482 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69303.7us
I20260709 12:26:37.128494   509 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:37.173579 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.154095   367 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:26:37.224722   498 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.225375   521 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.226634 32638 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:37.227767   521 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.231256 26129 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:37.232340   579 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:37.240077   599 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:26:37.169747   512 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.244848 32697 raft_consensus.cc:2764] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:37.152643   604 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:37.249073   604 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:37.249555   604 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:37.249619 26782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1464) took 1310 ms (client timeout 1702 ms). Trace:
W20260709 12:26:37.250140 26782 rpcz_store.cc:269] 0709 12:26:35.938978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.939118 (+   140us) service_pool.cc:224] Handling call
0709 12:26:37.249398 (+1310280us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b5400660154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:37.249526 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590284032}
I20260709 12:26:37.221328   509 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:37.173098   503 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:37.253146   503 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:37.253687   503 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:37.244836   598 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:26:37.139981   525 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:37.256944   597 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:26:37.258782 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78675.7us
W20260709 12:26:37.260021 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 186709us
I20260709 12:26:37.164991   517 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:37.154783 32370 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:37.268185 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 114309us
I20260709 12:26:37.167338   602 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:37.280197 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:37.361995 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:37.362808 26782 raft_consensus.cc:2376] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18. Re-sending same reply.
W20260709 12:26:37.373052 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93844.4us
I20260709 12:26:37.276902   609 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.174028   519 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:37.401743   608 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:26:37.403275   607 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 12:26:37.413166   611 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:37.415242   525 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.276625   532 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.113s
I20260709 12:26:37.417254   532 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:37.417788   532 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:37.418937   532 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.425179 32563 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:37.426002 32563 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.429553   610 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:37.430323   610 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:37.364848   542 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.024s	sys 0.024s
I20260709 12:26:37.435027 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:37.272541   520 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:37.442780 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61981.3us
I20260709 12:26:37.439733 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:37.396064   609 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.450511 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.451400 26126 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:37.452989   609 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:37.457717 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:37.441186   542 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.024s	sys 0.024s
I20260709 12:26:37.466924   305 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:37.281962   602 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.468626 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.470089 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.471902 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.470808 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:37.390549   521 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:26:37.455881 32697 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:37.448989   536 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.492482 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.476878 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.484858 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.487119 26196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:37.487517 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.495926 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.498503 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:37.504629 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.007s	sys 0.047s
W20260709 12:26:37.507398 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.008s	sys 0.047s
I20260709 12:26:37.484097   603 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:37.510649 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 586907us
I20260709 12:26:37.467782   305 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.458630   411 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:26:37.526402 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:37.522516   558 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.492614   532 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.521574 26129 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:37.528736 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.536118 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.496134   612 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.492987   536 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:37.519097 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:37.518126   614 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:37.561517 26196 raft_consensus.cc:606] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.562036 26196 raft_consensus.cc:614] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:37.516098   548 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:37.516464   603 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.557274 26789 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.493335 32563 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.558583 32367 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.469403 32370 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.524488   519 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:37.560264   612 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:37.537760   610 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:37.537396   617 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.581821 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1202) took 1125 ms. Trace:
W20260709 12:26:37.573311   554 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.001s
I20260709 12:26:37.528203   420 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.566217   548 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.563480   536 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:37.613991 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3640) took 1625 ms (client timeout 1798 ms). Trace:
I20260709 12:26:37.541623   558 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.619246 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:37.624337 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:37.629648 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:37.633287 26050 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.593643 26125 rpcz_store.cc:276] 0709 12:26:36.449462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.453301 (+  3839us) service_pool.cc:224] Handling call
0709 12:26:36.453595 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.880506 (+426911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.574427 (+693921us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:37.574755 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968237440}
I20260709 12:26:37.574697   603 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:37.631150 26123 rpcz_store.cc:269] 0709 12:26:35.988936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.158133 (+169197us) service_pool.cc:224] Handling call
0709 12:26:37.613803 (+1455670us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002c9cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:37.613930 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581125504}
I20260709 12:26:37.654033 26046 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:37.656991 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.657689 26198 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:37.669441 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:37.643987 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:37.674932 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:37.616539   548 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:37.571293   614 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.653231   579 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:37.593736   612 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.684048 26789 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:37.658804 32563 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:37.669883 32697 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:37.680588 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 151914us
W20260709 12:26:37.693509 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 165794us
I20260709 12:26:37.608816   617 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.706640   305 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:37.684803   520 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.717033 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:37.715595 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:37.731630 26789 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:37.729569 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:37.730497 26051 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:26:37.773583 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44107.4us
I20260709 12:26:37.774369 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:37.774920 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.775658 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.776540 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.711681   579 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:37.736775   615 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:26:37.733458 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 98625.1us
I20260709 12:26:37.713039 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.710657   616 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:37.783106 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70174.4us
I20260709 12:26:37.775527 26197 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
W20260709 12:26:37.778868 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61991.5us
I20260709 12:26:37.781385 26201 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:37.783824 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:37.776330 26194 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:37.785403 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.791828 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:37.791987 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.792600 26051 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:37.793728 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:37.785019   305 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:37.797125 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:37.712150 32563 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:37.808547   626 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:26:37.809020   367 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:26:37.822634   614 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:37.823590   512 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.002s
I20260709 12:26:37.785753 32759 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.796073   610 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:26:37.828204   305 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:37.875319 32563 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.875761   305 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.876679   552 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:37.880409 32367 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.882612 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.883833 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.802735 32697 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.884872   558 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:37.885138 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.886287 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.809067   615 consensus_queue.cc:1243] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:37.832365 26046 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:26:37.856204   527 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.001s
I20260709 12:26:37.853260 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.903457 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:37.858978 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:37.848119   616 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:37.887946   613 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:26:37.780036   602 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:37.851544   604 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.004s
W20260709 12:26:37.872398 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75347.4us
W20260709 12:26:37.872217 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 88445.6us
I20260709 12:26:37.888200   614 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:37.873488   512 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.873939   367 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:37.874758   610 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:37.830361 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:37.883903 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:38.013338 26049 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:38.013653 26123 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:37.836750 26050 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:26:37.869071   503 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.615s	user 0.014s	sys 0.000s
I20260709 12:26:37.870177 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:37.908545 26046 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:37.905376 26051 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:37.909520 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:37.908331 26048 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.855779   511 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.801602   634 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.122s
I20260709 12:26:37.895665   519 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.891968 32759 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:37.987023   516 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.881s	user 0.011s	sys 0.003s
I20260709 12:26:37.919229   604 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:37.905593   527 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:37.989866   533 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.163s	user 0.014s	sys 0.002s
I20260709 12:26:38.083761 26053 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:37.990350   512 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:37.920207   509 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.007s
I20260709 12:26:38.085192   509 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.995512 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:37.996657 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.087056 26197 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:38.086977 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:38.000823 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52111us
I20260709 12:26:38.088116 26200 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:37.920142   420 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:37.901368   615 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:37.916224 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:38.098688 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 182535us
I20260709 12:26:38.028442   610 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:38.039922 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.042721 26201 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:37.884876   603 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:37.924945   552 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 12:26:38.062330 26127 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:38.048172 26123 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.027316   498 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.079244 32759 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.081990   527 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.040300   602 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:38.068755   516 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:38.060114   511 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:38.047374   503 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.147850 26199 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:38.014878   367 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:38.181646 26203 raft_consensus.cc:2368] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:38.043432   632 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.185966 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 145999us
I20260709 12:26:38.065073   519 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.209368 26050 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:38.186372   527 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:38.210016   305 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:38.210229   527 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.210952   305 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.198233   648 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.001787   579 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.212663   648 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.215310   648 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.117985   615 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.209664   512 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:38.193636 32708 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:38.217037 32708 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.216553   512 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.217512 32708 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.218184   615 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.219084   512 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.201223   509 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.201697   603 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.197268   612 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.224803 26123 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:38.182830 32697 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.232530 26125 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:38.190683   511 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.098018   517 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:26:38.231987 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:38.249053   552 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:38.249568   552 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:38.225432   632 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.250527   552 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.253538   604 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.185525   498 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.256176   498 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.256862   578 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:26:38.251972   552 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:38.245080   548 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.004s
I20260709 12:26:38.260138   578 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:38.260813   578 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:38.261094   609 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.261006   548 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.261807   609 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.262147   578 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.266584 26046 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:38.268600 26048 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:38.269600 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.270094 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:38.270437 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 479383us
I20260709 12:26:38.272792 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.274245 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.277961 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:38.278978 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1132) took 1864 ms (client timeout 1778 ms). Trace:
I20260709 12:26:38.279315   305 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.280601   503 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.281339 32370 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:38.282300 26786 rpcz_store.cc:269] 0709 12:26:36.414164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.414295 (+   131us) service_pool.cc:224] Handling call
0709 12:26:38.278759 (+1864464us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:38.278909 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002111360}
W20260709 12:26:38.244948   536 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260709 12:26:38.304996 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:38.305428   367 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.305702 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:38.306407 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:38.306833 26787 raft_consensus.cc:2368] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:38.306770   367 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.309005   645 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:38.308434 32370 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.309542   635 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.309063 26785 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:26:38.310698 26785 pending_rounds.cc:85] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 13
I20260709 12:26:38.311038 26785 pending_rounds.cc:107] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:26:38.311297 26785 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:38.284981 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 185311us
I20260709 12:26:38.290609 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:38.282215   516 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.344955 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:38.309702   645 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.233175   602 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.360853 26018 leader_election.cc:400] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:38.361645 26018 leader_election.cc:403] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:38.314467 32370 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.357827   645 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:38.376992   640 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:26:38.380869   662 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.381035   609 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.381640   662 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.381198   537 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:38.383452   537 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:38.384743   537 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.386081   537 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.386315 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.387037   665 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:38.387518 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.387714   657 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:38.388381   657 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.389671 26788 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:38.390125 26788 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:38.390614   657 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.390972 32706 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.391503 32706 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.392735   615 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:38.393283   615 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:38.394162   631 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.585s	user 0.002s	sys 0.001s
I20260709 12:26:38.394658   602 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.342140   635 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.395840   617 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.397588   635 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.316452   536 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.398517 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.399298 26199 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:38.399847   536 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.401384   615 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.244021 32697 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.405948 32697 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.310218   498 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:26:38.309767   511 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.408341 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:38.409056 26050 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.409734 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.390833 26788 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:38.411204 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57270.1us
I20260709 12:26:38.354784   517 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.424412 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.428122 26051 raft_consensus.cc:2410] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 12:26:38.345242 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54708.6us
I20260709 12:26:38.450809 26787 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:38.452032 26051 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:38.418566 32697 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.359545   516 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.469108 26051 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:38.472548 26050 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:38.463078   579 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:38.454469   639 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.246296   610 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.475229 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:38.460820   624 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.479763 26051 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:38.488885 32706 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.492895 26786 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:38.492995 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:38.387646   665 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.482352   517 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.500578 26051 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:38.485639   635 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:38.492421   624 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.505853   578 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.487339   579 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:38.635962 26791 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1483) took 2287 ms (client timeout 2232 ms). Trace:
I20260709 12:26:38.498895   502 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:26:38.509927   499 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:38.668197   499 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:38.668458   502 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:38.669021   502 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:38.669549   499 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.670858   499 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.489305   664 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.490715   639 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.524868   578 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:38.672973   578 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:38.673480 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.673775 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.674233 26203 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:38.674633   639 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.675321 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:38.636546 26791 rpcz_store.cc:269] 0709 12:26:36.245424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.252393 (+  6969us) service_pool.cc:224] Handling call
0709 12:26:38.522205 (+2269812us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400639f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:38.522372 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621566208}
I20260709 12:26:38.677110 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:38.637204   617 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.680796   617 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.683473   617 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.655300   305 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.684513   305 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.687012   305 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.546267   610 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.636209   615 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.692813   615 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.695708   615 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.698114 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.699060 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:38.661818   511 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:38.663121 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.663259   642 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.701129   663 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.332s	user 0.009s	sys 0.000s
I20260709 12:26:38.701696   642 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.527966   635 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.706322   635 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.709256   662 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.657153   516 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.713505   642 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:38.638057   512 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.719751 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44736 (request call id 38) took 1251 ms. Trace:
I20260709 12:26:38.720348 26196 rpcz_store.cc:276] 0709 12:26:37.468276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.468417 (+   141us) service_pool.cc:224] Handling call
0709 12:26:38.719413 (+1250996us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400163d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:38.719591 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152953600}
I20260709 12:26:38.513013   517 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.747488 26791 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:38.719303   512 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.723846 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:38.762006 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:38.769779 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:38.770852 26791 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:38.738515   665 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.791575   517 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:38.792341   645 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.795567   645 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:38.767854   537 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:38.814862 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 440211us
I20260709 12:26:38.815810 26791 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
W20260709 12:26:38.779354 32708 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
I20260709 12:26:38.823936 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:38.815876 32370 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.824792 26782 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:38.825665 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.828434 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.776790   642 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:38.819430 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:38.830783 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:38.833328 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.833725 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.836531 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:38.862260 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:38.822662 32738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.019s	sys 0.062s
I20260709 12:26:38.796459   664 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.821645   537 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.858944 26196 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
W20260709 12:26:38.870640 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51332.3us
I20260709 12:26:38.828910   624 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:38.866927 32738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.019s	sys 0.063s
I20260709 12:26:38.873454   411 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:38.876405 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:38.872519   537 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:38.829530 32708 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:38.882848   367 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.005s
I20260709 12:26:38.866412   642 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:38.930719 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 433499us
I20260709 12:26:38.884467   305 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:38.872078   664 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:38.934710 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:38.937155 26789 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:38.938647 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:38.939720 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:38.941821 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:38.942601 26194 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:26:38.946211 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.946679 26201 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:38.947856 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2253) took 1171 ms. Trace:
I20260709 12:26:38.948400 26195 rpcz_store.cc:276] 0709 12:26:37.776011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.776137 (+   126us) service_pool.cc:224] Handling call
0709 12:26:37.801911 (+ 25774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.874501 (+ 72590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.874537 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:37.874585 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:37.874621 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.947417 (+1072796us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b54002b5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:38.947482 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.947645 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166429056}
I20260709 12:26:38.949388 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:38.888012   411 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:38.952646 26200 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:38.928059   552 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.005s
I20260709 12:26:38.953706 26198 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:38.954985 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.959481 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:38.961187 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.962122 26203 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:38.960628 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.963178 26048 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:38.963897 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:38.966661 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.967218 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:38.967782 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:38.974113   411 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:38.978734 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.958591   367 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:38.980016 26200 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:38.960628   503 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:38.951866   662 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:38.968806   672 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:38.978935 32721 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:38.983614 32697 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.001s
I20260709 12:26:39.005407 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:38.982713   367 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.007402 26194 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:38.986829   512 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:39.019181   651 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.649s	user 0.008s	sys 0.005s
I20260709 12:26:39.021886   665 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:39.024995   660 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.655s	user 0.012s	sys 0.000s
I20260709 12:26:39.029850   642 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.030822 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3588) took 1216 ms. Trace:
I20260709 12:26:39.032027   632 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.009482 32738 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.013195   672 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.204443 32605 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.205346 32605 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.071650   641 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.209136   641 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.213008 32708 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.213850   517 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:39.214545 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.214668   517 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.217757   525 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.218613   525 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.220837   665 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.223125   665 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.225400 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1495) took 1963 ms. Trace:
I20260709 12:26:39.071926   420 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.228998   420 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.231299   645 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:39.232021   645 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.234683   645 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.033676   578 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:39.035772   627 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.140s	user 0.004s	sys 0.006s
I20260709 12:26:39.036034 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:39.296088 26045 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.037004 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.039150 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.297731 26787 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
W20260709 12:26:39.042403   647 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.674s	user 0.003s	sys 0.015s
I20260709 12:26:39.043414   610 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:39.039234   664 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.049362 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:38.986663   552 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:38.995189   503 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.056447   367 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.013458 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:39.062440 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:39.079449 26050 rpcz_store.cc:276] 0709 12:26:37.814725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.480261 (+665536us) service_pool.cc:224] Handling call
0709 12:26:38.481306 (+  1045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.481759 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.481800 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.497870 (+ 16070us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.957081 (+459211us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:38.957130 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.957199 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.030517 (+ 73318us) raft_consensus.cc:1685] finished
0709 12:26:39.030550 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.030738 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1913,"raft.queue_time_us":159,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5183,"thread_start_us":195917,"threads_started":1,"wal-append.queue_time_us":423427}
I20260709 12:26:39.014982 32697 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:39.190793 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.160732   515 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:39.196623 32738 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.225989 26783 rpcz_store.cc:276] 0709 12:26:37.262090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.265070 (+  2980us) service_pool.cc:224] Handling call
0709 12:26:37.265420 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:37.270689 (+  5269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.270734 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:37.365718 (+ 94984us) log.cc:844] Serialized 52 byte log entry
0709 12:26:37.777587 (+411869us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:37.804010 (+ 26423us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.841339 (+ 37329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.841449 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.018376 (+1176927us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:39.224949 (+206573us) raft_consensus.cc:1685] finished
0709 12:26:39.225074 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.225275 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49992,"prepare.queue_time_us":37589,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":84474,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":8664,"spinlock_wait_cycles":14998272,"thread_start_us":344550,"threads_started":1,"wal-append.queue_time_us":983567}
I20260709 12:26:39.291543   632 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.292395   606 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 2.010s	user 0.003s	sys 0.012s
I20260709 12:26:39.293236   512 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:39.312690   642 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:39.298422 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:39.398794 26046 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:39.199839   672 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.312930 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:39.318763 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 269394us
I20260709 12:26:39.400794 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1503) took 1090 ms. Trace:
I20260709 12:26:39.161758   343 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.212019   641 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.330435   305 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:39.417994   305 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.419720 32738 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.423511 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2216) took 1155 ms. Trace:
I20260709 12:26:39.400381   512 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.424088 26051 rpcz_store.cc:276] 0709 12:26:38.268228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.513750 (+245522us) service_pool.cc:224] Handling call
0709 12:26:38.516274 (+  2524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.517096 (+   822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.517170 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.857321 (+340151us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.072028 (+214707us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:39.072104 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.072205 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.423255 (+351050us) raft_consensus.cc:1685] finished
0709 12:26:39.423279 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.423447 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":192969,"threads_started":1,"wal-append.queue_time_us":549309}
I20260709 12:26:39.426160 26051 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:39.426666   512 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:39.385279 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.058115 32721 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.427474   512 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:39.393810 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:39.293699   650 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.106s	user 0.007s	sys 0.009s
I20260709 12:26:39.323364   558 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.401471 26787 rpcz_store.cc:276] 0709 12:26:38.310662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.312053 (+1001391us) service_pool.cc:224] Handling call
0709 12:26:39.400681 (+ 88628us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:39.401098   627 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:26:39.405915 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.392549   515 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.392604 32605 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.415797   343 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.423429   672 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.426826 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 889) took 1152 ms. Trace:
I20260709 12:26:39.326460   552 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.246930   662 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.364483   537 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.454389 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:39.456802 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:39.456961 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:39.457518 26783 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:39.457957 26046 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:39.458609 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 894) took 1082 ms. Trace:
I20260709 12:26:39.329452   610 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.396984   632 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.443506 26051 rpcz_store.cc:276] 0709 12:26:38.274859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.425638 (+1150779us) service_pool.cc:224] Handling call
0709 12:26:39.425975 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.426564 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.426764 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.426529   509 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.444607 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.463413 26787 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:26:39.458916 26046 rpcz_store.cc:276] 0709 12:26:38.376052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.455774 (+1079722us) service_pool.cc:224] Handling call
0709 12:26:39.457804 (+  2030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.458335 (+   531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.458543 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.405033   642 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.467242 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1582) took 1105 ms. Trace:
I20260709 12:26:39.467490 26054 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:39.467865 26787 rpcz_store.cc:276] 0709 12:26:38.361791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.444215 (+1082424us) service_pool.cc:224] Handling call
0709 12:26:39.467154 (+ 22939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.469060 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:39.469825 26787 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:39.428145   512 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.437428 26045 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:39.472856 26789 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:39.471961 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 472813us
I20260709 12:26:39.480377 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1493) took 2218 ms. Trace:
I20260709 12:26:39.480198 32708 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:39.480849 26790 rpcz_store.cc:276] 0709 12:26:37.261550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.261720 (+   170us) service_pool.cc:224] Handling call
0709 12:26:37.262159 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.273798 (+ 11639us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:37.378121 (+104323us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.657303 (+279182us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.657949 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.658122 (+   173us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:37.658388 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.769107 (+110719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.769225 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.035194 (+1265969us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:39.480074 (+444880us) raft_consensus.cc:1685] finished
0709 12:26:39.480104 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.480289 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53010,"prepare.queue_time_us":127229,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":314003,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":41737,"spinlock_wait_cycles":16748288,"thread_start_us":258924,"threads_started":2,"wal-append.queue_time_us":386596}
I20260709 12:26:39.482379   515 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.484221   641 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:39.487198 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:39.492564   661 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.124s	user 0.002s	sys 0.024s
I20260709 12:26:39.492910   578 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.493582   578 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.494270   512 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:39.488507 32738 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.484091 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1579) took 1161 ms. Trace:
I20260709 12:26:39.494905 32738 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.497279 32738 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:39.484292   499 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.813s	user 0.016s	sys 0.000s
I20260709 12:26:39.498087   420 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.498383   499 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:39.498878   499 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.495057 26790 rpcz_store.cc:276] 0709 12:26:38.322358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.483112 (+1160754us) service_pool.cc:224] Handling call
0709 12:26:39.483504 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.483711 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.483740 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.483792 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:39.483827 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.483889 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.484025 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.501492   579 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.459801   665 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:39.502264 26120 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:39.503754   610 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.505499 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.505359 26045 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:39.506058   642 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.507375 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.494103   578 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.508916 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.513916 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.514024 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:39.511360 26129 raft_consensus.cc:2410] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:39.516079 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.520285 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.502213 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1580) took 1146 ms. Trace:
I20260709 12:26:39.523025 26790 rpcz_store.cc:276] 0709 12:26:38.355876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.500138 (+1144262us) service_pool.cc:224] Handling call
0709 12:26:39.501677 (+  1539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.501881 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.501906 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.501952 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:39.501991 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.502048 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.502162 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.527491 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.528367 26127 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:39.528573 26790 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:39.525722 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2709) took 2088 ms. Trace:
I20260709 12:26:39.455602   672 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.525099 26783 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:39.532423 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.532533 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.532829 26783 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:39.538677 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1584) took 1050 ms. Trace:
I20260709 12:26:39.566958 26783 rpcz_store.cc:276] 0709 12:26:38.488619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.532382 (+1043763us) service_pool.cc:224] Handling call
0709 12:26:39.532666 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.538281 (+  5615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.538598 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.539348 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.567185 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2201) took 1949 ms. Trace:
I20260709 12:26:39.568411 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:39.568719 26053 rpcz_store.cc:276] 0709 12:26:37.617794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.618889 (+  1095us) service_pool.cc:224] Handling call
0709 12:26:39.566941 (+1948052us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:39.567090 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038329216}
I20260709 12:26:39.473606   525 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.529697 26121 rpcz_store.cc:276] 0709 12:26:37.437478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.437622 (+   144us) service_pool.cc:224] Handling call
0709 12:26:37.438029 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.466721 (+ 28692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.466768 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.467418 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.776142 (+308724us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:37.776334 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.776481 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.292907 (+1516426us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:39.511890 (+218983us) raft_consensus.cc:1685] finished
0709 12:26:39.511911 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.525627 (+ 13716us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22566,"raft.queue_time_us":29681,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8221,"spinlock_wait_cycles":11506560,"thread_start_us":305905,"threads_started":1,"wal-append.queue_time_us":573378}
I20260709 12:26:39.570784 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.571863 32370 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.573685   578 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.573906 26121 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:39.575039   638 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.743s	user 0.009s	sys 0.024s
I20260709 12:26:39.507391   678 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:39.576613   678 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.577013   642 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.577747   642 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.579108   678 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.580124 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.460523   662 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.560053 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.560570 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.561308   367 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.582659   662 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.562119   343 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.583189   367 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.471930   509 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.585564   509 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.564637 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.560068 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.557081 26046 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:39.586429 26120 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:39.580449   642 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.544018 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.536787 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1581) took 1179 ms. Trace:
I20260709 12:26:39.587600 26122 raft_consensus.cc:2410] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:26:39.587255 26790 rpcz_store.cc:276] 0709 12:26:38.356963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.524135 (+1167172us) service_pool.cc:224] Handling call
0709 12:26:39.528408 (+  4273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.528947 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.536699 (+  7752us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.563457   499 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:39.589108 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1586) took 1091 ms. Trace:
I20260709 12:26:39.589697 26045 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:39.589566 26790 rpcz_store.cc:276] 0709 12:26:38.498259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.588158 (+1089899us) service_pool.cc:224] Handling call
0709 12:26:39.588498 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.588734 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.588772 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.588831 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:39.588875 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.588944 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.589055 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.591112 26783 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:39.591697 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1585) took 1100 ms. Trace:
I20260709 12:26:39.591713 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1587) took 1077 ms. Trace:
I20260709 12:26:39.592413   662 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.592202 26790 rpcz_store.cc:276] 0709 12:26:38.514613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.590962 (+1076349us) service_pool.cc:224] Handling call
0709 12:26:39.591269 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.591432 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.591455 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.591491 (+    36us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:39.591518 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.591560 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.591646 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.593078   662 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.594435   632 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.594837 26790 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:39.595170   517 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.512593   498 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:39.595907   498 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.596189   662 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:39.596943 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1494) took 2335 ms. Trace:
I20260709 12:26:39.592156 26783 rpcz_store.cc:276] 0709 12:26:38.491756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.580374 (+1088618us) service_pool.cc:224] Handling call
0709 12:26:39.580680 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.591487 (+ 10807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.591652 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.597294 26784 rpcz_store.cc:276] 0709 12:26:37.261833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.261944 (+   111us) service_pool.cc:224] Handling call
0709 12:26:37.274662 (+ 12718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.521578 (+246916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.521609 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.522138 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.750806 (+228668us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:37.750859 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.750952 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.574431 (+1823479us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:39.596646 (+ 22215us) raft_consensus.cc:1685] finished
0709 12:26:39.596667 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.596891 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103354,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7061,"spinlock_wait_cycles":11237632,"thread_start_us":438101,"threads_started":2,"wal-append.queue_time_us":295508}
I20260709 12:26:39.493616   502 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.600356 32741 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.600847   502 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.600916 32741 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.603118 32741 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.609373   552 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.611725   672 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.614104 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:39.626267 26787 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:39.627942 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.631268 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.632021 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.628868 26122 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
W20260709 12:26:39.633508 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.637796 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.632725 26129 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:39.640772 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.640818 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:39.644434 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.646378 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:39.647768 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.645310 26129 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:39.649576 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:39.650152 26197 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:39.592935   503 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:39.650938 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 377318us
I20260709 12:26:39.653604 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.657823 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.658512 26194 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:39.657845 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.654458 26129 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:39.659377 26203 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:39.596347   498 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:39.664729 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.666447 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:39.670255 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.633419   517 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:39.671166 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:39.672628 32370 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:39.667172 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:39.673290 32370 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:39.674400 32370 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.640661   642 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:39.684535 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.660708   662 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:39.686610 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.690341 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.690979 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1208) took 2011 ms. Trace:
I20260709 12:26:39.691052 26053 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:39.691468 26125 rpcz_store.cc:276] 0709 12:26:37.679754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.679885 (+   131us) service_pool.cc:224] Handling call
0709 12:26:37.682331 (+  2446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.238090 (+555759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.690728 (+1452638us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:39.690920 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264829312}
I20260709 12:26:39.692358   645 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:39.692776   645 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.693121   645 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:39.694700   517 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:39.641055   552 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:39.695946   552 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:39.696587   552 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:39.695708   517 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.698169 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:39.697685   552 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:39.698781 26782 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.699409   552 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:39.699848   509 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:39.667829   503 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:39.700416   509 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.701730 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:39.699568   655 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.339s	user 0.014s	sys 0.014s
I20260709 12:26:39.702752 26790 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:39.704145 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.705375 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1148) took 3270 ms. Trace:
I20260709 12:26:39.705456   662 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:39.706524   509 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.716207 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.716311 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.717253 26053 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:39.703841 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:39.718182 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.709467 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.710335 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.712260 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.713454 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.708007 26785 rpcz_store.cc:276] 0709 12:26:36.434704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.441800 (+  7096us) service_pool.cc:224] Handling call
0709 12:26:38.301157 (+1859357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.311943 (+ 10786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.701483 (+1389540us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400426650. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:39.705305 (+  3822us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4231013248}
I20260709 12:26:39.683810   537 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:39.723346 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.722151 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:39.722913 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 660534us
I20260709 12:26:39.723322 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.725283 26785 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:39.725346 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.724592 26790 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:39.722604 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.726649 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:39.734233 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.735574 26197 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:39.736166 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:39.736761 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.588s (SENT) [suppressed 18 similar messages]
I20260709 12:26:39.738050 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:39.738116 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 287664us
I20260709 12:26:39.738940 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:39.740186 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.745092 26203 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:39.733394 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:39.747867 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.750265 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.752060 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:39.762662 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.723766 32741 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:39.780938 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44862.1us
W20260709 12:26:39.736008   411 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.016s	sys 0.029s
W20260709 12:26:39.743649   654 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.355s	user 0.009s	sys 0.052s
W20260709 12:26:39.792374 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59111.8us
I20260709 12:26:39.743168   499 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.708928   642 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:39.805020   411 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.016s	sys 0.030s
W20260709 12:26:39.819065   654 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.430s	user 0.009s	sys 0.053s
I20260709 12:26:39.787853 32741 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:39.885870 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.886092   525 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:39.888389 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:39.743434 32638 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:39.793965   662 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.893983 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:39.893328 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 105019us
I20260709 12:26:39.896737 26782 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:39.917284 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.897640 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:39.926723 26129 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.926852 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.910419 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:39.911245 26121 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:39.911978 32741 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.900936   558 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.895403   641 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:39.921794 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2218) took 1652 ms. Trace:
I20260709 12:26:39.934211 26050 rpcz_store.cc:276] 0709 12:26:38.269441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.384632 (+1115191us) service_pool.cc:224] Handling call
0709 12:26:39.403376 (+ 18744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.404498 (+  1122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.404527 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.420583 (+ 16056us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.730050 (+309467us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:39.730113 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.730214 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.921578 (+191364us) raft_consensus.cc:1685] finished
0709 12:26:39.921599 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.921733 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":712,"raft.queue_time_us":45711,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":3915,"spinlock_wait_cycles":52480,"thread_start_us":308977,"threads_started":1,"wal-append.queue_time_us":499194}
I20260709 12:26:39.932523 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.924628 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2198) took 2292 ms. Trace:
I20260709 12:26:39.894795   343 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:39.936666 26052 rpcz_store.cc:276] 0709 12:26:37.616758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.618706 (+  1948us) service_pool.cc:224] Handling call
0709 12:26:37.658491 (+ 39785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.678104 (+ 19613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.678155 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.678736 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.814300 (+135564us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:37.814363 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.814469 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.698882 (+1884413us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:39.908932 (+210050us) raft_consensus.cc:1685] finished
0709 12:26:39.908955 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.909157 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13012,"raft.queue_time_us":73484,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":24724,"spinlock_wait_cycles":16660864,"thread_start_us":105626,"threads_started":1,"wal-append.queue_time_us":538435}
I20260709 12:26:39.918819 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.951241 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.937088   641 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:39.912449   657 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.917683   610 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:26:39.941723 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:39.940169   411 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:26:39.949150 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:39.950459 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.951439 26196 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:39.938827 26199 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:39.951534 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:39.961982 26197 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:39.909054   537 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:39.964684 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54508.4us
I20260709 12:26:39.966167 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:39.962913 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.951704 26129 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:39.966897 26787 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:39.931085   527 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:39.974803 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70190.1us
I20260709 12:26:39.972646 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:39.979550 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:39.980458 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:39.980923 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.952950   641 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.981700 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:39.983342 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:39.979440 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:39.984323 26785 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.952874 32638 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:39.964375   688 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.923544   498 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:39.967639   577 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:39.975135 32708 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:39.975396   678 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.974918   684 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.019619 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.025358 26129 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:40.030596 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:39.985313   579 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:39.955266   343 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.014586   634 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:39.974746   537 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.044209 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:39.962450   657 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:40.046730 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63476.9us
W20260709 12:26:40.042361 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56519.1us
I20260709 12:26:39.976346   662 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 12:26:40.041808 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61477.9us
I20260709 12:26:40.015111   527 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:40.096771 26050 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:40.018285   577 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.045395 32370 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:40.039474 26127 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:40.019088   632 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:40.062927   516 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.032931   684 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.064738 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.065908   537 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.066330 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.066550   686 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:40.067273 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.039707   579 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:40.092214 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136192us
I20260709 12:26:40.093067 26129 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:40.136312 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.138096 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.105139   642 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.109815 32370 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:40.131690   502 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.004s
I20260709 12:26:40.031351   678 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.126240 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.131222 26121 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:40.132174 26120 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.018919   688 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.041061   512 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:40.107378   577 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.138545   579 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.109716   527 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.111263 26785 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:40.143235 32370 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.116254   516 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.150050 26126 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:40.177211 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.133951   552 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:40.160970 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.140542 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:40.141294 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92136.5us
I20260709 12:26:40.133256   632 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.137670   558 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:40.147955   634 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:40.162400   499 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:40.108634 32638 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.189769 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:40.187011 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:40.185737 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:40.190289 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105627us
I20260709 12:26:40.139755   645 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.190658 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.191362 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.198361 26198 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.198350 26787 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:40.199544 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.200176 26787 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:40.186712 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.187269 26129 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:40.168781   683 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:40.200500 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:40.201015 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.201661 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:40.205191 26202 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:40.154258 32741 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.179291   516 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.217075 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.217795 26790 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:40.218042 26198 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:40.180197   662 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.220631   662 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.221508   527 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.225533   684 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:40.226740   704 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:40.229938   632 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.189359   525 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:26:40.234045   668 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.832s	user 0.003s	sys 0.023s
I20260709 12:26:40.235812   657 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.239243   678 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.240675 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.241600 26785 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:40.206445 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.243559 26129 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:40.203437 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.212571   641 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.261689   525 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:40.262733   525 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.263505 26785 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:40.263594   641 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.215504   343 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.212010 26120 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:40.265867 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.266259 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.233279   634 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.267560 26126 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:40.268534 26125 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:40.271174 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.255345 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.205879 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.273620 26787 raft_consensus.cc:2368] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21.
I20260709 12:26:40.246836 32370 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.278007 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.278304 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:40.198930 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:40.261372 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.281616 26129 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:40.289667 26787 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:40.304356 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.305090 26047 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:40.305606   645 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.306255 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.307132 26047 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 14: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:40.307714   688 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:40.305176 26121 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 12:26:40.309681 26124 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:40.307989 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.311710 26122 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:40.311971 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:40.313898 32370 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.188781   552 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:40.279934   700 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:26:40.279770   699 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:40.323457 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.324887 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.326036 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.326608 26047 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:40.327437 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:40.329361 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.333480   548 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.335589   552 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.242537   683 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:40.309782   343 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:40.339823 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:40.354557 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 236179us
I20260709 12:26:40.359297   343 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:40.315215   709 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:26:40.360610   343 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.359422 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.320788 32708 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:40.331123   516 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.311352   657 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:26:40.378629 32708 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.382699 26125 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:40.356875   548 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:26:40.413797 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1500) took 2105 ms. Trace:
I20260709 12:26:40.414590 26786 rpcz_store.cc:276] 0709 12:26:38.308191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.312759 (+  4568us) service_pool.cc:224] Handling call
0709 12:26:38.314131 (+  1372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.493537 (+179406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.389235 (+1895698us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:40.413709 (+ 24474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399034752}
I20260709 12:26:40.435760 32741 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:40.454715 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 901) took 1691 ms (client timeout 2241 ms). Trace:
I20260709 12:26:40.456969 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2227) took 1442 ms. Trace:
W20260709 12:26:40.458218 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 902) took 1392 ms (client timeout 1543 ms). Trace:
I20260709 12:26:40.455276 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2213) took 2661 ms. Trace:
I20260709 12:26:40.425753   683 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:40.460860   658 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 2.092s	user 0.015s	sys 0.036s
W20260709 12:26:40.458576 26054 rpcz_store.cc:269] 0709 12:26:39.065936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.190426 (+124490us) service_pool.cc:224] Handling call
0709 12:26:40.458049 (+1267623us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b5400171e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:40.458166 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182651264}
I20260709 12:26:40.462554 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.457386 26045 rpcz_store.cc:276] 0709 12:26:39.014231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.521140 (+506909us) service_pool.cc:224] Handling call
0709 12:26:39.589378 (+ 68238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.590143 (+   765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.456784 (+866641us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b5400359dd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:40.456913 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156068736}
W20260709 12:26:40.455186 26048 rpcz_store.cc:269] 0709 12:26:38.763107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.384914 (+621807us) service_pool.cc:224] Handling call
0709 12:26:40.454506 (+1069592us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:40.454641 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66970496}
I20260709 12:26:40.467422   519 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.455132   509 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:40.461740 26049 rpcz_store.cc:276] 0709 12:26:37.793919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.221439 (+427520us) service_pool.cc:224] Handling call
0709 12:26:38.221721 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.221936 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:38.312794 (+ 90858us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.480787 (+167993us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.485151 (+1004364us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b1800188bc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:39.565618 (+ 80467us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b1800188bc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:39.570383 (+  4765us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b1800188bc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:26:40.337160 (+766777us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b1800188bc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287
0709 12:26:40.426409 (+ 89249us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.454781 (+ 28372us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.454860 (+    79us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:40.454927 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.455040 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.455195 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243211,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":11157,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":5615,"spinlock_wait_cycles":52969728,"thread_start_us":645404,"threads_started":5,"wal-append.queue_time_us":454341,"wal-append.run_cpu_time_us":1882,"wal-append.run_wall_time_us":1024373}
I20260709 12:26:40.468492   509 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.469590 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.471159 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.471251 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.463333 26054 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:40.473578 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.473809 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.472371 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.475035 26049 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:26:40.481385 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.488094 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.489272 26048 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:40.475556 26045 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:40.474692 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.490939 26054 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:26:40.492358 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.492866 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.493535 26049 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:40.493826 26048 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:40.494925 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.495884 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:40.496631   516 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:40.497345 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.498219 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:40.503408   662 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.507337 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.511521 26045 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:40.508647   724 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:26:40.508057 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.525063 26783 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:40.514297 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:40.515146 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2224) took 1549 ms. Trace:
I20260709 12:26:40.528535 26046 rpcz_store.cc:276] 0709 12:26:38.965875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.476263 (+510388us) service_pool.cc:224] Handling call
0709 12:26:39.476646 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.586527 (+109881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.514839 (+928312us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:40.515074 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136143104}
I20260709 12:26:40.517190   579 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.532018 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:40.532747 26045 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:40.546694 26053 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:40.554463 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.557574 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.559312 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.563216 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.565259 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.565304 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.567011 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.569576 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.530455 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.519529 26054 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:40.572669 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.566809 26125 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18. Re-sending same reply.
I20260709 12:26:40.576119 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.577491 26054 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:40.577795 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:40.586678 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:40.574945   519 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.622287 26054 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:40.571113   657 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:40.519608   577 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.572078   525 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:40.584082   709 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:40.611698 32546 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.615423   673 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:40.615644   672 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:40.642712 26054 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:40.641209   709 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.575315 32708 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:26:40.649729 26054 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:40.631101   548 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:40.665149 26789 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1588) took 1874 ms (client timeout 1895 ms). Trace:
I20260709 12:26:40.638018   657 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:40.665782 26789 rpcz_store.cc:269] 0709 12:26:38.790947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.938396 (+147449us) service_pool.cc:224] Handling call
0709 12:26:40.664879 (+1726483us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b5400639f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:40.665068 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226670976}
I20260709 12:26:40.640444   525 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:40.581192   729 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:40.648641   673 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.688356   673 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:40.689081   579 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.689787   548 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:40.683868   657 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.690820   548 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.686715   525 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:40.681908   527 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:40.698951   527 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.700028   657 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.701557   527 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.702662   657 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.707392   681 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:40.707875   681 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:40.646713   343 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.650049   709 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.712754   343 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.710814   673 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.715538   343 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.629525   519 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.719655   707 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:40.720191   707 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:40.707685   525 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.722662   709 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.648382   672 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:40.732933 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 113731us
I20260709 12:26:40.734237 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2288) took 1012 ms. Trace:
I20260709 12:26:40.630348   716 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:26:40.651932 32546 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:40.730741   672 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:40.769104 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 568633us
I20260709 12:26:40.734756 26195 rpcz_store.cc:276] 0709 12:26:39.722105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.722273 (+   168us) service_pool.cc:224] Handling call
0709 12:26:39.722565 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.722775 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:39.861772 (+138997us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.179657 (+317885us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.180024 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.723854 (+543830us) spinlock_profiling.cc:243] Waited 984 us on lock 0x7b180042b5c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:26:40.724035 (+   181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.724085 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:40.724131 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.724196 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.734155 (+  9959us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10285,"prepare.queue_time_us":223892,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":3439,"raft.queue_time_us":668780,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":55762,"spinlock_wait_cycles":12484608,"thread_start_us":419865,"threads_started":3,"wal-append.queue_time_us":114943}
I20260709 12:26:40.770051 26757 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:40.721310   707 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:40.775249 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 576321us
I20260709 12:26:40.777077 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:40.783282   498 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.784252   532 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:40.765147   715 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:40.784891   532 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:40.787204   577 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.796917 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.799819 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1594) took 1164 ms. Trace:
I20260709 12:26:40.803251 26128 raft_consensus.cc:2368] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22.
W20260709 12:26:40.803489 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 216874us
I20260709 12:26:40.800225 26784 rpcz_store.cc:276] 0709 12:26:39.635712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.635826 (+   114us) service_pool.cc:224] Handling call
0709 12:26:39.658123 (+ 22297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.658347 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:39.899567 (+241220us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.497033 (+597466us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18001c95a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:40.717506 (+220473us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b18001c95a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287
0709 12:26:40.719379 (+  1873us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b18001c95a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b7827958f 00007f5b712b8571 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd
0709 12:26:40.765039 (+ 45660us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.765364 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.799475 (+ 34111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.799529 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:40.799573 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.799645 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.799766 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30190,"prepare.queue_time_us":338832,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":65214,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7709,"spinlock_wait_cycles":90065792,"thread_start_us":490359,"threads_started":4,"wal-append.queue_time_us":239937}
I20260709 12:26:40.803804 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3679) took 1285 ms. Trace:
I20260709 12:26:40.805430 26128 rpcz_store.cc:276] 0709 12:26:39.518218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.518392 (+   174us) service_pool.cc:224] Handling call
0709 12:26:40.796682 (+1278290us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:26:40.803741 (+  7059us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122416896}
I20260709 12:26:40.806553 26123 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:40.778966   711 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:40.809314 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.809975 26121 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.812608 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:40.815042   516 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:40.815631   516 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.815994 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.816684 26054 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:26:40.817911 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.818317   642 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:40.818928   642 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.820887 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:40.822029 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:40.822979 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:40.818440 26054 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.824906 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:40.796506   532 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:40.825773   684 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.789089   715 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:40.832456 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.833181 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.853444 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.854008 26046 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.856436 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.857296 26048 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:40.825778   512 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:40.889720 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66809.9us
I20260709 12:26:40.817917   516 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.894821 26121 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:40.895699 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:40.823969   686 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:40.831940   684 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.852512   715 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:40.905575 26048 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:40.907832 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:40.908358 26048 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.909150 26046 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:40.900981 26054 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:40.866014   716 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.911515 26046 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:40.915711   642 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.917102   716 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.919430   716 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.923743 26121 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:40.916214   707 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:40.933353 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2285) took 1315 ms. Trace:
I20260709 12:26:40.938200 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:40.906893   684 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.900583   525 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.935684 26201 rpcz_store.cc:276] 0709 12:26:39.617487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.619113 (+  1626us) service_pool.cc:224] Handling call
0709 12:26:39.619566 (+   453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:39.620050 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.620092 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:39.620832 (+   740us) log.cc:844] Serialized 52 byte log entry
0709 12:26:40.519571 (+898739us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18003d9900. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:26:40.728021 (+208450us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:40.728062 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.728128 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.921037 (+192909us) raft_consensus.cc:1685] finished
0709 12:26:40.921059 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.933258 (+ 12199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":28253,"spinlock_wait_cycles":8091392,"thread_start_us":208242,"threads_started":1,"wal-append.queue_time_us":1167966}
W20260709 12:26:40.956929   742 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:26:40.957938   738 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:26:40.945382   713 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:40.960886 32546 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.910840   686 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.970757   686 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:40.978456   681 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:40.980712 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:40.981400 26784 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:40.982344 26790 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:40.983780 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.984593 26783 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:40.985594 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:40.985865 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:40.986650 26789 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:40.945190   736 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:26:40.988103 32721 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:40.989063 26784 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:40.989800   673 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:40.990290   715 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:40.945511   740 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:26:40.934856   747 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.993372   747 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.003865   420 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:41.018628   420 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:41.019498   420 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.020542   420 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:41.020748   502 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:26:41.029285 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1231) took 1319 ms (client timeout 1636 ms). Trace:
W20260709 12:26:41.029814 26122 rpcz_store.cc:269] 0709 12:26:39.709674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.711197 (+  1523us) service_pool.cc:224] Handling call
0709 12:26:41.029064 (+1317867us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:41.029215 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1296055552}
I20260709 12:26:41.001055   684 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:41.017089   750 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:26:40.996188   747 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.008090   519 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.017261   673 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:41.054541 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 867557us
I20260709 12:26:41.005414   532 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.059952   532 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.035673   672 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.074340 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:41.074723 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.075271 26121 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:26:41.075199 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 298221us
I20260709 12:26:41.076143 26128 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:41.076642 26201 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:41.076076 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.077739 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.057273   673 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.080013 26198 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:41.082067 26126 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:41.086020 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.086452 26202 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:41.086712 26194 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:41.087471 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:41.087790   722 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.794s	user 0.000s	sys 0.004s
I20260709 12:26:41.089466 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.090063 26199 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.092410 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.094445 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2720) took 1033 ms. Trace:
I20260709 12:26:41.094841 26124 rpcz_store.cc:276] 0709 12:26:40.061080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.065411 (+  4331us) service_pool.cc:224] Handling call
0709 12:26:40.065802 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.314768 (+248966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.094236 (+779468us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54001c4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:41.094392 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560338816}
I20260709 12:26:41.095192 26202 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:41.095713 26194 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:41.096457   684 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.116380   503 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:41.117154 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.117497   503 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.118726 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.121173   552 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.121636   552 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:41.121743   662 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:41.122078   552 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.122156   662 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:41.122716   662 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.096975   715 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.129681   673 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.130832   662 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.133198   662 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.134817   552 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.135869 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.137117   552 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.137382 26120 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:41.138409   673 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:41.139343   673 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:41.139617 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2723) took 1077 ms. Trace:
I20260709 12:26:41.129695   709 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.140348   673 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.143426 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:41.144047 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.144173 26049 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:41.145118 26788 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.145308   672 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.145334 32638 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:41.145370 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:41.137110 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3683) took 1610 ms (client timeout 1947 ms). Trace:
W20260709 12:26:41.147478 26127 rpcz_store.cc:269] 0709 12:26:39.526330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.559689 (+ 33359us) service_pool.cc:224] Handling call
0709 12:26:41.136892 (+1577203us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b540037c154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:41.137046 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053684608}
I20260709 12:26:41.148041 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:41.140018 26120 rpcz_store.cc:276] 0709 12:26:40.062266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.266730 (+204464us) service_pool.cc:224] Handling call
0709 12:26:41.137191 (+870461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.137848 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.139409 (+  1561us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b540037c150. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:41.139548 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001165184}
I20260709 12:26:41.149113   502 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:41.151376   711 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.151870   711 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.152225   711 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.154829   509 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:41.156960 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 261318us
I20260709 12:26:41.157836 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.159724 26757 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:41.160929   516 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.161482   516 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:41.162096   516 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.169687 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2238) took 1280 ms. Trace:
I20260709 12:26:41.170279 26047 rpcz_store.cc:276] 0709 12:26:39.888763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.339520 (+450757us) service_pool.cc:224] Handling call
0709 12:26:41.169377 (+829857us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400228b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:41.169607 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654600064}
I20260709 12:26:41.178861 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.180889   537 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:41.189842 26788 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:41.190779 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1576) took 3476 ms. Trace:
I20260709 12:26:41.201515   665 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.202059   665 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.202883   537 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:41.203393   537 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.200305 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:41.199064 26199 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:41.206208 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.206928 26194 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:41.209138 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:41.209794 26789 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.211843 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1163) took 1021 ms. Trace:
I20260709 12:26:41.212400 26782 rpcz_store.cc:276] 0709 12:26:40.190522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.190744 (+   222us) service_pool.cc:224] Handling call
0709 12:26:40.201713 (+ 10969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.768358 (+566645us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:40.768389 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.773453 (+  5064us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.136099 (+362646us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b1800476ec0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:26:41.175906 (+ 39807us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:41.175966 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.176082 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.211499 (+ 35417us) raft_consensus.cc:1685] finished
0709 12:26:41.211527 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.211774 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":591535,"raft.queue_time_us":567521,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3855,"spinlock_wait_cycles":10983680,"thread_start_us":39561,"threads_started":1,"wal-append.queue_time_us":434099}
I20260709 12:26:41.206961 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:41.219388 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.221019 26789 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:41.191182 26788 rpcz_store.cc:276] 0709 12:26:37.714768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.742623 (+ 27855us) service_pool.cc:224] Handling call
0709 12:26:37.742926 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.389540 (+646614us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400571820. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:26:39.915668 (+1526128us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b580067dcd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:26:41.144748 (+1229080us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b58006f14d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:26:41.190415 (+ 45667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.190705 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2994874752}
W20260709 12:26:41.217512   538 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:26:41.218338   644 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.224587   760 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:41.225210   548 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:41.226905   646 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.227373   646 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:41.227777   646 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:41.233489 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 54694.2us
W20260709 12:26:41.234176 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 158276us
W20260709 12:26:41.235721 26050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2236) took 1347 ms (client timeout 1616 ms). Trace:
I20260709 12:26:41.229631 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:41.236166 26050 rpcz_store.cc:269] 0709 12:26:39.888203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.008938 (+120735us) service_pool.cc:224] Handling call
0709 12:26:41.235521 (+1226583us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:41.235661 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147151744}
I20260709 12:26:41.240898 26194 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:41.235037 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:41.249399 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.245400 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:41.254503 26094 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:26:41.271154 26094 leader_election.cc:403] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:26:41.256187 26787 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:41.271670 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.232079 32638 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.254433 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.275174 26791 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:41.254629 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:41.271580 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.232376   760 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:41.229378   762 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:41.244185   707 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:41.282078 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.236402   503 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.282783 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.294050 26791 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:41.295454 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.297271 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:41.298097 26790 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.284121 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.286576 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.300431 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.277904   657 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.287566 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:41.288525 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59146.8us
I20260709 12:26:41.290558 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.293131 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:41.306444 26018 leader_election.cc:400] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21.
I20260709 12:26:41.306919 26018 leader_election.cc:403] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:41.274425   420 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.301726 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.252997   343 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:26:41.253445   634 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:41.282284   624 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.253748   509 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:41.348428 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:41.289090   527 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.288515   533 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:41.335575 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:41.304343   673 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:26:41.348845   623 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.350842   634 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:41.366225   694 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21.
W20260709 12:26:41.366791   768 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:26:41.368592 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:41.369024 26125 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:26:41.338539 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.304888   657 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.350287 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:41.374200 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.325263   548 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:41.300484   645 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:41.344754   685 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:41.336802 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:41.407609 26127 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:41.348483   420 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.346006 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.371500 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:41.272956   759 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 12:26:41.302976   711 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.409219 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.346241   532 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:41.353812   624 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:41.376645   548 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.364070   646 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.373939   665 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:41.416627 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118870us
I20260709 12:26:41.366277   634 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:41.416677 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70735.3us
I20260709 12:26:41.440254 26790 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:41.441192 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.350103   503 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:26:41.378065   657 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.437544 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:41.447156   646 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.430987   624 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.451948 26789 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:41.374105   498 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:26:41.424413   645 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:41.424120   632 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:41.436389   548 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.436092   685 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.368328   760 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.484784   420 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.484364   516 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.465337   503 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:41.432355   537 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:41.489941 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81044.5us
I20260709 12:26:41.465595 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:41.445068   759 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:41.503167 26788 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
W20260709 12:26:41.499665   766 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
W20260709 12:26:41.505322 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50980.7us
I20260709 12:26:41.505520 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.506194 26790 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:41.506067 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:41.509083 26019 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.588s (SENT)
I20260709 12:26:41.509559 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.470721   632 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:41.511626   673 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.512190   673 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.519743   673 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.522092   673 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:41.522908   673 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:41.524291   673 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.527293 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:41.529197 26788 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:41.527284   517 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:41.533667 26168 connection.cc:256] RPC call timeout handler was delayed by -1.9042333779998444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 12:26:41.534277 26168 connection.cc:256] RPC call timeout handler was delayed by -1.5082333780001136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 12:26:41.534557 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.534579 26168 connection.cc:256] RPC call timeout handler was delayed by -1.1260301680003977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
W20260709 12:26:41.535040 26168 connection.cc:256] RPC call timeout handler was delayed by -1.0009301680001954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
I20260709 12:26:41.541527 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1120) took 1355 ms. Trace:
I20260709 12:26:41.542124 26200 rpcz_store.cc:276] 0709 12:26:40.185782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.185944 (+   162us) service_pool.cc:224] Handling call
0709 12:26:40.186272 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.188181 (+  1909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.188218 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.188730 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.297585 (+108855us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:40.442736 (+145151us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.002489 (+559753us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b48000b1b58. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:26:41.540996 (+538507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.541106 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.541163 (+    57us) raft_consensus.cc:1685] finished
0709 12:26:41.541281 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.541450 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1099495,"prepare.queue_time_us":135929,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1639193,"raft.run_cpu_time_us":3113,"raft.run_wall_time_us":272645,"spinlock_wait_cycles":6592896,"thread_start_us":235278,"threads_started":2,"wal-append.queue_time_us":93939,"wal-append.run_cpu_time_us":4191,"wal-append.run_wall_time_us":1170269}
W20260709 12:26:41.544751 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 344541us
I20260709 12:26:41.549131 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:41.549885   645 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.550613   558 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:41.551385   558 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:41.533560   632 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:41.552865 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:41.552529   558 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.554267 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:41.555073 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.19775e+06us
I20260709 12:26:41.555855 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.557696 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.557808 32721 raft_consensus.cc:2764] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:41.558925 32721 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:41.560599   757 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.560949 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.561196   411 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:41.561792   757 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.549368   711 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:41.563443   770 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:41.564543 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.564777 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.565644   499 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:41.566238   499 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.568758 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.569067   499 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:41.569330   769 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:26:41.570024 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.572516 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.578284 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.580490 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.582137 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.582613 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.583464 26194 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:41.583529   711 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.585647 26194 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:41.586169 26194 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:41.587109 26194 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.589151 26202 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:41.589687 26194 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:41.591426 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57814.7us
W20260709 12:26:41.592120 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58268.7us
W20260709 12:26:41.595651 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 86603.3us
I20260709 12:26:41.503394   503 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:41.609129 26169 leader_election.cc:400] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22.
I20260709 12:26:41.604202 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:41.510612   641 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:41.608999 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:41.619647 26125 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:41.624511 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:41.625260 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:41.626423 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:41.626870 26194 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:26:41.625278 26129 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:41.627389 26194 pending_rounds.cc:85] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 14
I20260709 12:26:41.619021 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:41.627897 26194 pending_rounds.cc:107] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 19.15
W20260709 12:26:41.628718 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.629201 26194 raft_consensus.cc:2889] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:41.628720 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:41.604683   634 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.597250   688 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:41.635463   688 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:41.635903   688 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:41.636355   719 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:26:41.637225   503 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.638417   719 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:41.638831   719 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.639169   719 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.640380   711 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:41.641150   711 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:41.641474 26122 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:41.642109   548 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.642392   711 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.603803   420 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.644855   624 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.647040   624 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:41.648088   706 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:26:41.649354   706 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:41.649725   706 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:41.650722   706 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.654734   706 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.634681   499 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:26:41.660420   499 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:41.660637   761 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.660943   499 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:41.661213   761 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.662261   499 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.651176 32721 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.656306   715 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:41.656786 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.657286   694 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:41.659117   773 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:26:41.654632   657 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:41.667807 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48914.3us
I20260709 12:26:41.668426   641 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:41.669019   672 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:41.670019 26124 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:41.662773   537 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:41.675745 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.664304 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:41.688647 26791 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:41.702476 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.703564 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.703764 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.704416 26198 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:41.707446 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 114413us
I20260709 12:26:41.687758   775 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:41.707854 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51145.1us
I20260709 12:26:41.717281 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.711987 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.674252   757 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:41.720598 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.718284 26195 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:41.729732 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.760671 26203 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.761260 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.762632 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.690156   694 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:41.724117 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.763621 26195 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.764112 26194 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:41.707437   499 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.671942   632 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.715667 26791 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:41.734748 26199 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.709121   610 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:41.736711   634 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.737001   657 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.709312   672 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:41.739876 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:41.740597 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.723341 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.801920 26782 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:41.734146   757 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.766881   776 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.742208 26784 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:26:41.716322 32721 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:41.738547   785 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.733312   715 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:41.730867 26198 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:41.816959 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.817711 26198 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.752669   548 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.794190 26203 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:41.724933   775 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.798940 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:41.824923 26194 raft_consensus.cc:2376] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22. Re-sending same reply.
I20260709 12:26:41.799549 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:41.741070 26789 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:41.802739 26199 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:26:41.804827 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64313.1us
W20260709 12:26:41.806185 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 141978us
I20260709 12:26:41.798203   420 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:26:41.812973 32721 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.827580 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.830675 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:41.845213 26203 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:41.808212   662 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.852281 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.854318 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.854453 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:41.846501 32721 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:41.857301 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:41.855633 32370 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:41.806867   760 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:26:41.864259 26198 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
W20260709 12:26:41.876292 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 109885us
W20260709 12:26:41.859552 26019 leader_election.cc:336] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.831s (SENT)
I20260709 12:26:41.879642 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:41.884632 32370 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:41.885262 32370 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.885670 32370 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:41.851796   548 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:41.800820   641 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:41.941268 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81746.2us
W20260709 12:26:41.890909 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 167708us
I20260709 12:26:41.943360 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:41.910869 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.849167   771 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:26:41.938198 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58451.1us
I20260709 12:26:41.951359 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:41.876500   645 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:41.880796   503 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:41.822867   785 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:41.966542 32370 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.955495   672 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:41.971576 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82862.5us
W20260709 12:26:41.972157 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 131622us
I20260709 12:26:41.849213   683 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 12:26:41.851723   673 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 12:26:41.977387   645 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:41.977752   786 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:26:41.977634 32370 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:41.978201   715 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:41.979127   757 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.942515   788 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:41.980141   783 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:26:41.980728 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.937297   548 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:41.845124   694 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:41.948524   657 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:41.980887   782 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 12:26:42.030262 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 82881.1us
I20260709 12:26:41.939648   680 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 12:26:41.975618   761 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:41.986665   503 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:42.045562 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94317.1us
I20260709 12:26:42.051052 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:42.058436 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:42.060606 32370 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:42.061041 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:42.061975 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:42.061043 32370 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:42.062728 32370 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.066277 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:42.066771   784 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:26:42.067164 26122 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 16: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:42.068400   792 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:42.069242   793 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
W20260709 12:26:42.070870 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.071687   517 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:42.072885   777 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:42.074214   502 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:42.076699   785 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:42.077809   796 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:42.078872 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:42.078688   645 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:42.063619 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:42.095850 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:42.059362   694 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:42.112354 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.113850 26195 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:42.116199 32370 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.058400   632 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:42.093844   788 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:42.153496 32370 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:42.156198 32370 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:42.156906 32370 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:42.164330   781 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 12:26:42.164839   789 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:26:42.165690 32721 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:42.178047 26019 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.168s (SENT)
I20260709 12:26:42.178607 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:42.121562   673 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:42.186811 32370 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:42.205067   804 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:42.205189   800 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:26:42.206838 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 128046us
I20260709 12:26:42.208868 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:42.209472 26200 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:42.125569   577 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:42.168073 32721 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:42.221385 26019 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.301s (SENT)
I20260709 12:26:42.176697   715 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:42.209642   797 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 12:26:42.224866 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.226940 26047 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:42.227373 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.235231 26048 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:42.227806 26046 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 20. Ops: []
I20260709 12:26:42.235873 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.228294 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.237653 26045 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:42.229177 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.230247 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.238677 26054 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:42.226943 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.241055 26053 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:42.242493 26049 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:42.248718 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.249742 26051 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:42.247896 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.251482 26045 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:42.135182   792 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:26:42.263729 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.210141   632 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:42.265543 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.211815   577 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.212334   517 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:42.267838 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.269134 26052 raft_consensus.cc:2368] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19.
I20260709 12:26:42.211804   775 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:42.213045   673 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:42.275940 26125 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:26:42.319344 26019 leader_election.cc:336] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.573s (SENT)
W20260709 12:26:42.370661 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51401us
W20260709 12:26:42.371959 26019 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.576s (SENT)
I20260709 12:26:42.275700   788 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:42.362648   577 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:42.377851   798 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:26:42.380568 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.361941 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:42.373939   673 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:42.294130   688 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.002s
I20260709 12:26:42.417304 26195 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:42.417946 26199 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:42.293146 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:42.419754 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:42.398303 26051 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:42.368887 26125 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:26:42.409960 26019 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.601s (SENT)
I20260709 12:26:42.267015   803 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:42.234050   715 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:42.421430 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:42.406623 26050 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:42.269567   777 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:42.414448   796 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:42.437747 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:42.271056   793 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:42.434918 26126 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:26:42.422899 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:42.438566 26194 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:42.441381 26122 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:42.415860 32638 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:42.427822   803 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:42.454747 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.440366   761 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.240729   503 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:42.438004   796 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:42.512928   796 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:42.514269   796 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.515731   788 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:42.445200   793 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:42.516631   788 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.517889   793 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:42.470705   706 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.803s	user 0.008s	sys 0.001s
I20260709 12:26:42.519203   793 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.523165   706 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.524065   788 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:42.478271   719 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.814s	user 0.010s	sys 0.000s
W20260709 12:26:42.525894 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.525657   719 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.477989   548 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.481841   516 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:42.486462   776 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:42.556501   776 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:42.488008   779 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.005s	user 0.002s	sys 0.000s
W20260709 12:26:42.489921   610 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.689s	user 0.011s	sys 0.000s
I20260709 12:26:42.494840   517 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:42.513396 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1628) took 1051 ms. Trace:
I20260709 12:26:42.448278   715 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:42.468709   803 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.516837   411 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:42.450441 32638 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.472321   577 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:42.528434   719 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:42.548808   757 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:42.433097   688 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:42.558853 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.569804 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:42.572662 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:42.566411 26786 rpcz_store.cc:276] 0709 12:26:41.461932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.462160 (+   228us) service_pool.cc:224] Handling call
0709 12:26:41.462529 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.462943 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.462986 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.463587 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.598743 (+135156us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:41.663373 (+ 64630us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.663731 (+   358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.663808 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.510984 (+847176us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18001a3b60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:42.513111 (+  2127us) raft_consensus.cc:1685] finished
0709 12:26:42.513197 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.513334 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2657,"prepare.queue_time_us":96647,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":1208,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":26940,"spinlock_wait_cycles":46491008,"thread_start_us":129047,"threads_started":1,"wal-append.queue_time_us":233309}
W20260709 12:26:42.578277 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.559257   548 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:42.628085 26200 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.632191 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.633106 26051 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:42.628603 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:42.635391 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.574002   610 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:42.645236 26123 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:42.573057   411 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:42.709266 26049 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:42.528925   706 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:42.713691 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 85151.6us
I20260709 12:26:42.596643   794 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:42.714493 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:42.715828 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.716794 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.721172 26052 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:42.632771   803 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:42.724399 26049 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:42.726583 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.573380   646 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
W20260709 12:26:42.747097 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.750352 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.018s	sys 0.026s
W20260709 12:26:42.750751 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.018s	sys 0.026s
W20260709 12:26:42.751314 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 542470us
I20260709 12:26:42.706046   694 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.763168 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.774365 26049 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:42.774509 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.632339 32583 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.123s
I20260709 12:26:42.745927   680 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:42.750155   809 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:42.797993 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.024s	sys 0.028s
W20260709 12:26:42.817832 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.024s	sys 0.028s
I20260709 12:26:42.819959 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:42.821063 26124 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:42.822783 26122 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:42.823877 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:42.828279 26200 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:42.830583 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:42.596791   777 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:42.842787 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 90729.3us
I20260709 12:26:42.808779   680 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:42.848790 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:42.850800 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:42.852432 26197 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:42.586292   757 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:42.860452 26129 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:42.829171   809 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.862640 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:42.866613 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:42.908087 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.948491 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.798040   694 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:42.980136 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:42.852334   552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.403s	user 0.010s	sys 0.014s
I20260709 12:26:42.992372 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:42.995636 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:42.995862 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 133373us
I20260709 12:26:43.001327 26050 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:43.009799 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.863426   680 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:43.013520 26198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.064s	sys 0.102s
W20260709 12:26:43.017408 26198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.064s	sys 0.102s
I20260709 12:26:42.872308   719 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:26:42.999171   552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.550s	user 0.010s	sys 0.014s
I20260709 12:26:43.004153   305 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.752475   776 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:43.006887   761 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.002s
I20260709 12:26:42.974220   809 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.854239   788 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.162s
I20260709 12:26:43.008682   714 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.018579   610 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:43.034237   822 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:26:43.042158   776 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.030598   305 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:42.818400   411 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.050714 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:43.058073 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:43.059208 26045 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:43.059265 26053 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:26:43.069145 26047 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:43.042614   761 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:43.112048 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.078823   305 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.133457 26128 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:26:43.112093   813 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
I20260709 12:26:43.049952   719 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 12:26:43.165232 26049 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.173776 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:43.157554   820 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:26:43.156940   827 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:26:43.152925   819 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:26:43.164592   748 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:43.143774   558 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.059574   714 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.110605   816 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 12:26:43.171701   688 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:43.209254 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3692) took 1589 ms. Trace:
I20260709 12:26:43.166878   706 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:43.202836   818 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:43.214314 26789 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:43.080078   757 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:43.206102   826 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:43.136689   821 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:26:43.180322   761 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:26:43.210465 26126 rpcz_store.cc:276] 0709 12:26:41.619350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.619447 (+    97us) service_pool.cc:224] Handling call
0709 12:26:41.619774 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.440614 (+820840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.208886 (+768272us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54002ca1d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:43.209180 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873922432}
I20260709 12:26:43.234475   634 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:43.182050   657 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:26:43.234615   707 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:43.236085 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 1.38174e+06us
W20260709 12:26:43.313767 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.233420 26789 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:43.235339   711 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.203482   748 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:43.311211 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:43.216975   641 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:43.338017 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:43.300168   623 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:43.318882 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:43.333453 26784 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:43.204391   558 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.330626 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:43.306829   577 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:43.365046 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:43.377233 26791 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:43.363929 26782 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:43.360402 26121 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:43.362352 26789 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
W20260709 12:26:43.386709 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 319829us
I20260709 12:26:43.390818 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.391793 26195 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:43.395018 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:43.398504 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.402733 26194 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:43.404212 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:43.405030 26121 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:43.406203 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:43.406823 26124 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:43.317080   757 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:43.337407   796 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.821s	user 0.007s	sys 0.006s
I20260709 12:26:43.410841 26194 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:43.373904   516 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.341846   694 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.380023   753 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.400477   788 consensus_queue.cc:1243] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:43.400534   623 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:43.418565 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:43.415618 26200 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:43.387614   537 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:43.361634   641 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.423517 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:43.379784   538 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:43.409641   577 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.475276 26791 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:43.307147   634 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.342273   707 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.386391   688 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.458400 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.493285 26195 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:43.335812   818 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 12:26:43.459966 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.486934 26126 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:43.496930 26200 raft_consensus.cc:2368] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:43.343189   711 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.498283 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.502071 26200 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:43.414896   748 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.510398 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.442677   516 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.465700   537 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:43.513288   829 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:26:43.513959 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.515545 26195 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:43.521911   641 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:43.522495 26195 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:43.523811 26195 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:43.518226   623 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.507668   644 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:43.487597   538 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.520037 26782 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:43.520408   830 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:43.520989   516 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.455591   753 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.491820   577 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.530611 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:43.516245   825 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.349s	user 0.001s	sys 0.000s
I20260709 12:26:43.535132 26195 raft_consensus.cc:2368] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:43.499454   688 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.516393   714 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:43.527678   748 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.546440 26782 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:43.549077 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:43.551340 26757 leader_election.cc:336] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.755s (SENT)
I20260709 12:26:43.552572 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:43.559446   558 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:43.532243   644 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.552896   809 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.605942   711 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:43.607378   803 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.003s
I20260709 12:26:43.598544   846 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:43.620528   848 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.621344   848 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.618460   846 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.631099   846 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.616223   845 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.635360   845 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.642944   847 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:43.646790   848 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.631969   846 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.016s
W20260709 12:26:43.682268 26053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.026s
W20260709 12:26:43.683223 26053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.007s	sys 0.026s
W20260709 12:26:43.686396 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.687655 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.682268 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 496885us
I20260709 12:26:43.722474 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:43.418563 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:43.725927 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:43.733927 26166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.013s	sys 0.082s
W20260709 12:26:43.734892 26166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.013s	sys 0.082s
I20260709 12:26:43.735571 26128 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:43.733543 26127 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:43.750069 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.751679 26195 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:43.752327 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.754951 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.750237 26199 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:26:43.756465 26199 pending_rounds.cc:85] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 14
I20260709 12:26:43.756932 26199 pending_rounds.cc:107] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:26:43.757527 26199 raft_consensus.cc:2889] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:43.759419 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.760785 26195 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:43.766358 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:43.774891 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:43.785019 32546 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.785815   519 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.787240   519 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.792901   519 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:43.793659   305 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.011s
I20260709 12:26:43.794356   305 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.768967 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:43.812338 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.004s	sys 0.009s
W20260709 12:26:43.814467 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.004s	sys 0.010s
W20260709 12:26:43.820092 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 122024us
I20260709 12:26:43.786854 32546 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.848349   809 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:43.849031   634 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:43.850601 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53503us
I20260709 12:26:43.852227 32741 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:43.853034 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:43.836782 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:43.615854   776 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.539s	user 0.003s	sys 0.011s
I20260709 12:26:43.861577   719 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.862363   537 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:43.862676   719 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:43.863405 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 313208us
I20260709 12:26:43.864213   577 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.864977 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:43.866976   681 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.870627 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2276) took 1030 ms. Trace:
I20260709 12:26:43.872398   714 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:43.873392   714 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.874228 32546 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.874495   644 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.876494   538 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:43.880542 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.854091 32741 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:43.881354   498 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:43.881910 32741 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:43.871410 26051 rpcz_store.cc:276] 0709 12:26:42.843714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.843910 (+   196us) service_pool.cc:224] Handling call
0709 12:26:42.844379 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.844686 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:43.240893 (+396207us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.869671 (+628778us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.869811 (+   140us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:43.869965 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.870114 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.870458 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":536142,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":20162,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":341,"thread_start_us":950975,"threads_started":2,"wal-append.queue_time_us":571409}
I20260709 12:26:43.882061   498 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:43.886222   498 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:43.854116   305 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:43.905977   672 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:43.909802 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:43.911123 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:43.911914 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:43.912923 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:43.914134 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:43.913869   672 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:43.916479 26126 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:43.919576 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:43.898183 32741 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:43.925720 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:43.926836 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:43.927014 32741 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:43.928496 26121 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.522s	user 0.019s	sys 0.001s
I20260709 12:26:43.880589 26791 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:43.929055 26121 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:43.928134 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:43.939000   688 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:43.897550   714 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:43.943827   793 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.423s	user 0.008s	sys 0.008s
W20260709 12:26:43.945887 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81330.6us
I20260709 12:26:43.947243   644 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:43.948652 26196 raft_consensus.cc:2368] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
W20260709 12:26:43.949370 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 397931us
I20260709 12:26:43.949898   641 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 12:26:43.640062   845 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.966315   714 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.967569 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:43.973659 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:43.974524   701 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.975072   701 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.981889   707 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:43.984632   803 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:26:43.985720   803 consensus_queue.cc:1243] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:43.986308   803 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.986984   803 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:43.970125   644 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:43.988770   644 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.999411 32546 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:44.007347   714 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:44.032514 26049 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.822s	user 0.016s	sys 0.013s
I20260709 12:26:44.033156 26049 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:44.006320 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:44.037413 26196 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:44.039038 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.039810 26196 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:44.017531 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:44.081383 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64097.3us
I20260709 12:26:44.083190 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:44.020543 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.995963   305 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:44.085690   305 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:44.087442   757 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.665s	user 0.014s	sys 0.003s
I20260709 12:26:44.087144   305 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.089109   305 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.092432   846 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:44.093019   846 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:44.093725   846 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:44.095482   516 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:44.040904 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73657.4us
I20260709 12:26:44.041658   793 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:44.100978   793 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.101560   793 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:44.042580   794 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:44.103188   794 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:44.103237 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.104740   794 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.107069   794 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.043179   719 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.043597 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2336) took 1472 ms. Trace:
W20260709 12:26:44.047017   548 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.410s	user 0.015s	sys 0.006s
I20260709 12:26:44.109086 32638 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.058495   701 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:44.111173 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.060320 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2760) took 1698 ms. Trace:
I20260709 12:26:44.111904 26127 rpcz_store.cc:276] 0709 12:26:42.361892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.362585 (+   693us) service_pool.cc:224] Handling call
0709 12:26:42.362889 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.745383 (+1382494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.059975 (+314592us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.060198 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3149516160}
I20260709 12:26:44.114154 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:44.115516 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:44.116110   787 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 2.094s	user 0.011s	sys 0.008s
I20260709 12:26:44.116407 26201 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:44.117453 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:44.118026   548 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.118799   558 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.118868   548 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:44.015271 26120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1236) took 2680 ms (client timeout 1755 ms). Trace:
W20260709 12:26:44.120445 26120 rpcz_store.cc:269] 0709 12:26:41.334302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.336394 (+  2092us) service_pool.cc:224] Handling call
0709 12:26:44.014442 (+2678048us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.015020 (+   578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693215104}
I20260709 12:26:44.109572   519 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:44.065158   832 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.565s	user 0.000s	sys 0.003s
I20260709 12:26:44.108690 26203 rpcz_store.cc:276] 0709 12:26:42.571358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.572305 (+   947us) service_pool.cc:224] Handling call
0709 12:26:44.043298 (+1470993us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002b4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.043501 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087307136}
I20260709 12:26:44.075048 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2317) took 2371 ms. Trace:
I20260709 12:26:44.123742 26197 rpcz_store.cc:276] 0709 12:26:41.703032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.703171 (+   139us) service_pool.cc:224] Handling call
0709 12:26:41.703430 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.857819 (+1154389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.074438 (+1216619us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400398354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.074817 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2641634688}
I20260709 12:26:44.124645 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.125677 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.126686 26197 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:44.127261 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.129011 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2272) took 1340 ms. Trace:
I20260709 12:26:44.077239 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2332) took 1658 ms. Trace:
I20260709 12:26:44.130524 26199 rpcz_store.cc:276] 0709 12:26:42.419184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.419606 (+   422us) service_pool.cc:224] Handling call
0709 12:26:42.419849 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.758303 (+1338454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.076795 (+318492us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.077154 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3056686976}
I20260709 12:26:44.132475 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:44.077330 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 82952.3us
I20260709 12:26:44.081225 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.134742 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:44.135344 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:44.025311 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2321) took 2304 ms. Trace:
I20260709 12:26:44.048532 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2277) took 1198 ms. Trace:
I20260709 12:26:44.136864 26196 raft_consensus.cc:2368] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:44.137061 26049 rpcz_store.cc:276] 0709 12:26:42.850177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.850382 (+   205us) service_pool.cc:224] Handling call
0709 12:26:44.048248 (+1197866us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54000ac3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.048434 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706948480}
I20260709 12:26:44.057255   498 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.138235 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:44.139815   824 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.923s	user 0.003s	sys 0.000s
I20260709 12:26:44.060304   803 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:44.140697   623 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:44.119639   558 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.136456 26202 rpcz_store.cc:276] 0709 12:26:41.720893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.724734 (+  3841us) service_pool.cc:224] Handling call
0709 12:26:41.724998 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.844858 (+119860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.844890 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.844929 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:41.844983 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.024626 (+2179643us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400399c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.024725 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.025009 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275044608}
I20260709 12:26:44.143080 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.143548 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.145452 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.146908 26202 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:44.148248 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2330) took 1731 ms. Trace:
I20260709 12:26:44.149925 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2767) took 1019 ms. Trace:
I20260709 12:26:44.148773 26198 rpcz_store.cc:276] 0709 12:26:42.416880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.417007 (+   127us) service_pool.cc:224] Handling call
0709 12:26:42.417358 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.417728 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.417757 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.418257 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.017934 (+599677us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:43.017973 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.018126 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.138699 (+1120573us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:44.147669 (+  8970us) raft_consensus.cc:1685] finished
0709 12:26:44.147693 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.147996 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22426,"raft.queue_time_us":4452,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6282,"spinlock_wait_cycles":30587264,"thread_start_us":574126,"threads_started":1,"wal-append.queue_time_us":744596}
I20260709 12:26:44.150532 26128 rpcz_store.cc:276] 0709 12:26:43.130317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.134520 (+  4203us) service_pool.cc:224] Handling call
0709 12:26:43.134899 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.736835 (+601936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.149479 (+412644us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400186ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.149841 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378604928}
I20260709 12:26:44.151541 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:44.152067   822 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:44.152541   822 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:44.117583 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.153858   822 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:44.154294 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2759) took 1792 ms (client timeout 1507 ms). Trace:
I20260709 12:26:44.129627 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.155525 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.156427 26049 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:44.156579 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.156867 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2757) took 1796 ms. Trace:
I20260709 12:26:44.157459 26129 rpcz_store.cc:276] 0709 12:26:42.360337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.361154 (+   817us) service_pool.cc:224] Handling call
0709 12:26:42.361436 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.871255 (+509819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.156271 (+1285016us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540037c154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.156652 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146636160}
W20260709 12:26:44.155195 26123 rpcz_store.cc:269] 0709 12:26:42.361534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.361674 (+   140us) service_pool.cc:224] Handling call
0709 12:26:44.153744 (+1792070us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.154021 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583999744}
I20260709 12:26:44.155704   818 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.159687   818 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.160768   558 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.139989 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.162297   818 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.162974 26196 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:44.163240   498 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.164368 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.165380   711 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:44.121833   519 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.168705   715 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.129671 26052 rpcz_store.cc:276] 0709 12:26:42.788838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.789540 (+   702us) service_pool.cc:224] Handling call
0709 12:26:42.812097 (+ 22557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.812556 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.812604 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.813167 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.238377 (+425210us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:43.238420 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.238492 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.064038 (+825546us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:44.128358 (+ 64320us) raft_consensus.cc:1685] finished
0709 12:26:44.128389 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.128768 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1184,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":14064,"spinlock_wait_cycles":6485376,"thread_start_us":233441,"threads_started":1,"wal-append.queue_time_us":593830}
W20260709 12:26:44.170077 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87102.1us
I20260709 12:26:44.172719   753 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.187717 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.188406   577 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.158283   748 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.189139   577 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.189932 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:44.061707   714 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:44.173251 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.176105 26203 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:44.148087 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.127851 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.189741   748 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.087942 32546 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:44.191766 26199 raft_consensus.cc:2368] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:44.193121 26195 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:44.211118 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.191051 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.212095 26196 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:44.216318 26045 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:44.226280 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.210632 32546 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.225922   707 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:44.233031 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.237269 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.233989 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.238524 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.182392   623 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:26:44.235623 26202 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:44.244453 26051 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:26:44.298106 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108275us
W20260709 12:26:44.257288 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.296929 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.253849 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.301000 26199 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:44.289831 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.247972 26050 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:44.230367   818 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.304575 26203 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
W20260709 12:26:44.305963 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.306977 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.253432 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:44.316053 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.311410 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.252938 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.248710 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:44.307585   623 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:44.253566   748 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.319387 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:44.326140 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.340576 26197 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:44.333513 26054 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:44.345767 26052 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:44.341816 26124 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:44.257023   548 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.333184 26782 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:44.352077 26128 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:44.330884 26048 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:44.308442   822 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:44.357453 26129 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:44.357321 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.355542 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.288506   644 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.360268 26054 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:44.356861 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:44.329720 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.365119 26791 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:44.363484 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:44.372494   680 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.339s	user 0.007s	sys 0.022s
I20260709 12:26:44.373126   680 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:44.375442   861 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:26:44.384402 26046 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:44.328405   678 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.385485 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.385646   678 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.386181 26046 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.388697 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.389576 26052 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:44.362211 26019 connection.cc:256] RPC call timeout handler was delayed by -1.2552100009997957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 12:26:44.391360 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.391875 26019 connection.cc:256] RPC call timeout handler was delayed by -1.1472100010000759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260709 12:26:44.392289 26019 connection.cc:256] RPC call timeout handler was delayed by -1.1237550349997036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3126s for a response
W20260709 12:26:44.392892 26019 leader_election.cc:336] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.620s (SENT)
I20260709 12:26:44.393318 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:44.394474   840 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.395104   840 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.402024   411 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:44.406551   681 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.008s
I20260709 12:26:44.407505 26122 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:44.408308   645 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.409055   753 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:44.408972   645 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.409584   846 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.388134   678 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.411896   846 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.409600   753 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:44.412916 26045 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.413509   776 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:44.413529   743 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.409807   711 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.317627   577 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.417743 26201 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:26:44.408430 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89080.3us
I20260709 12:26:44.432708 26784 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.432157 26198 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:26:44.449887 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 141975us
I20260709 12:26:44.449301 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3706) took 1391 ms. Trace:
I20260709 12:26:44.459436 26199 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:44.458689 26122 rpcz_store.cc:276] 0709 12:26:43.057662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.057805 (+   143us) service_pool.cc:224] Handling call
0709 12:26:44.414157 (+1356352us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001f77d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.449215 (+ 35058us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039385984}
I20260709 12:26:44.462265   646 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:44.462989   646 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:44.463575   646 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.464686   707 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:44.465245   707 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:44.469770   793 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.472455   305 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.472671   743 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.473241   305 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.475246   743 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.476020   610 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.476750   610 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.479255   610 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.479856   610 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.481180   644 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.480952   753 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.482205   753 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.449303   818 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.484808   818 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.489673   793 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.492398   761 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.493049   761 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.493448   707 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.494910   707 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.495499   761 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.498219   709 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:44.500339   548 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.501003   548 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.502297 26053 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:26:44.503473   548 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.457481   678 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.505064   678 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.510190   678 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.510968 26051 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:44.504002 26756 leader_election.cc:400] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19.
I20260709 12:26:44.512787 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:44.513635 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.514618 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:44.512832 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.516085 26053 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:26:44.516302 26051 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.516549 26053 pending_rounds.cc:85] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 11
I20260709 12:26:44.516961 26053 pending_rounds.cc:107] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:26:44.517203 26053 raft_consensus.cc:2889] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.520946 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.475328   305 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:44.522459   748 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:44.515039   644 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:44.526495 26019 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.538s (SENT)
I20260709 12:26:44.528883 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.530999 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.531628 32546 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.532840   610 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.533721   715 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.563141 26784 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:44.536638 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:44.558931 26195 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:44.375478   680 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.561612 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.565367 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:44.568349   748 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:44.606498 26046 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:26:44.571267 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 193) took 26368 ms (client timeout 15000 ms). Trace:
I20260709 12:26:44.571704 26194 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:44.577586   846 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:44.580641   610 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.582073 26782 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:44.592187 32546 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.593133   715 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.597414 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.599866   818 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:44.600495 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:44.601629   645 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.451134   558 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.570262 26203 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:44.612269 26045 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:44.613456   305 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.620126   796 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.606870   748 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:44.632462 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118019us
I20260709 12:26:44.504130   548 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 12:26:44.566557   678 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.614794   846 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:44.625324   794 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.634682 26053 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:44.639101 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.645309   644 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:44.645650   874 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:44.665508   874 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:44.666962   874 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.625672   743 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:44.669234 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.670008 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.654615 26049 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:44.657613   846 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.671955 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:44.671756 26052 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:44.611013   761 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:26:44.675767 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.674216 26199 raft_consensus.cc:2376] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21. Re-sending same reply.
I20260709 12:26:44.677548 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:44.679555 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.681766   840 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.684659 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:44.687371 26054 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:44.688380 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.689496 26047 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:44.694083   793 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:44.694569   793 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.695001   793 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:44.700002   516 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.009s
I20260709 12:26:44.700619   516 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:44.711807 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:44.712659 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 186117us
I20260709 12:26:44.714069 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:44.714653 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.715507 26019 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.868s (SENT)
I20260709 12:26:44.716104 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:44.687441   711 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.720755 26046 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:44.726781   715 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.712030   846 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.731563 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.732366 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:44.733708   846 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:44.733912   519 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.650503   818 consensus_queue.cc:1243] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 19.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:44.649817   796 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.716886 26053 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:44.738026 26791 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:44.739184 26202 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:26:44.736006 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2669) took 1320 ms (client timeout 1573 ms). Trace:
W20260709 12:26:44.740288 26194 rpcz_store.cc:269] 0709 12:26:43.415466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.418224 (+  2758us) service_pool.cc:224] Handling call
0709 12:26:44.735729 (+1317505us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b5400301954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.735909 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164025728}
I20260709 12:26:44.741372   796 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:44.686733   558 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.716419 26784 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:44.736440   793 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.753130 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
W20260709 12:26:44.754750 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 282628us
I20260709 12:26:44.632843   305 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.762223 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:44.766165 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72278.7us
I20260709 12:26:44.766884 26123 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:44.669806   794 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.773784 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.777133 26122 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:44.777040 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:44.769045 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.722s (SENT) [suppressed 40 similar messages]
I20260709 12:26:44.776391 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.788625 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.744825   644 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.760922   678 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.788393 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.765554   840 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.782158 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:44.607389 25946 rpcz_store.cc:269] 0709 12:26:18.202900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.204770 (+  1870us) service_pool.cc:224] Handling call
0709 12:26:19.043459 (+838689us) spinlock_profiling.cc:243] Waited 982 us on lock 0x7b5400240590. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71281640 00007f5b7a158846 00007f5b7a26290e 00007f5b787232a4 00007f5b78723232 00007f5b78723161 00007f5b787230dc 00007f5b78722392 00007f5b726e048c 00007f5b726df8c6
0709 12:26:44.570979 (+25527520us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:37.112928 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:37.149576 (+ 36648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:37.149657 (+    81us) write_op.cc:435] Acquired schema lock
0709 12:26:37.149687 (+    30us) tablet.cc:662] Decoding operations
0709 12:26:37.228810 (+ 79123us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:37.228905 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:37.228932 (+    27us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:26:37.230608 (+  1676us) tablet.cc:701] Row locks acquired
0709 12:26:37.230670 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:26:37.230889 (+   219us) write_op.cc:270] Start()
0709 12:26:37.231038 (+   149us) write_op.cc:276] Timestamp: P: 1783599997230822 usec, L: 0
0709 12:26:37.231071 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:26:37.231827 (+   756us) log.cc:844] Serialized 8260 byte log entry
0709 12:26:38.388499 (+1156672us) op_driver.cc:464] REPLICATION: finished
0709 12:26:39.480593 (+1092094us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b77ddbef0
0709 12:26:39.940351 (+459758us) write_op.cc:301] APPLY: starting
0709 12:26:39.940458 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:40.325017 (+384559us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:40.325058 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:40.482483 (+157425us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:40.483638 (+  1155us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 12:26:40.483723 (+    85us) write_op.cc:312] APPLY: finished
0709 12:26:40.484418 (+   695us) log.cc:844] Serialized 188 byte log entry
0709 12:26:40.744532 (+260114us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.744739 (+   207us) write_op.cc:454] Released schema lock
0709 12:26:40.744828 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:40.745315 (+   487us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.745342 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3267533,"reported_tablets":36,"spinlock_wait_cycles":2258176,"child_traces":[["op",{"apply.queue_time_us":1551434,"apply.run_cpu_time_us":46299,"apply.run_wall_time_us":804985,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92033,"num_ops":21,"prepare.queue_time_us":247235,"prepare.run_cpu_time_us":31166,"prepare.run_wall_time_us":448906,"raft.queue_time_us":637051,"raft.run_cpu_time_us":63202,"raft.run_wall_time_us":1524495,"replication_time_us":1157251,"spinlock_wait_cycles":16547072,"thread_start_us":2352216,"threads_started":5,"wal-append.queue_time_us":701206,"wal-append.run_cpu_time_us":53399,"wal-append.run_wall_time_us":2420759}]]}
I20260709 12:26:44.803051 26121 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:26:44.797674 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.796778 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.803768 26786 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:44.801944 26122 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:44.806080 26785 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:44.806154 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:44.807382 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 134153us
I20260709 12:26:44.808525 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.808525 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.810422 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:44.811962 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.812160 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.812552 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.812724 26790 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:44.812932 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.814373 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.815234 26198 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:44.817806   678 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:44.818642 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:44.808897 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:44.821600 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.812795 26203 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:44.808705   645 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:44.804453 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 267966us
I20260709 12:26:44.824028 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.812446 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.822525 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:44.829310 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.826187 26199 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:44.834322 26126 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:44.837114 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:44.838850 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 123379us
I20260709 12:26:44.838933 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.841655 26201 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:26:44.842594 26784 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:44.847373 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.854665 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:44.858804   646 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.861425   645 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.833935   840 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:44.869372 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.845979 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:44.878898 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.902348 26198 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
W20260709 12:26:44.883963 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.891976 26202 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:44.913401 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:44.914463 26019 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.895s (SENT)
I20260709 12:26:44.914605 26196 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:26:44.920637 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48454.1us
I20260709 12:26:44.913357 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.868921 32638 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:44.868513 32546 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:44.928630 26129 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:44.927407 26050 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.001s
I20260709 12:26:44.929606 26050 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:44.880307   793 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.932333 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.933832 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.900574   846 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:44.926878 26200 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 14.
I20260709 12:26:44.935092 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:44.942111 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 96198.5us
I20260709 12:26:44.944298 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.945103 26126 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:44.929426   516 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.927613   796 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:44.950739   869 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:44.950809   516 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.953719   516 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:44.954311   822 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.003s
I20260709 12:26:44.955236 26199 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:26:44.956744   871 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:26:44.940968   681 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:26:44.959669   794 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:44.960358 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 89670.3us
I20260709 12:26:44.930477   709 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 12:26:44.961242 26019 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.924s (SENT)
I20260709 12:26:44.961966   709 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:26:44.962337 26019 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.991s (SENT)
I20260709 12:26:44.946044 32638 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:44.963901 26019 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.005s (SENT)
I20260709 12:26:44.941828   714 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:44.975018 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.980731 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.980502   714 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:44.981323 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.981457   646 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.964159 32638 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.965579   840 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.966641   517 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:44.984591 26784 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:44.984715   517 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:44.946305 32546 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:44.968641 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:44.985846 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 131433us
I20260709 12:26:44.986266 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.986004 32546 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.986764 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:44.968722 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:44.987458 26123 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:44.969538 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.988775 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2673) took 1534 ms. Trace:
I20260709 12:26:44.989152 26200 rpcz_store.cc:276] 0709 12:26:43.454412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.502933 (+ 48521us) service_pool.cc:224] Handling call
0709 12:26:44.988563 (+1485630us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400302854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:44.988698 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3159190272}
I20260709 12:26:44.989549 26789 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:44.990231 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.990798 26788 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:44.991490 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:44.991922 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:44.992398 26787 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:26:44.991039 26200 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:44.993708 26790 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:44.994084 26200 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:26:44.994727 26128 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:44.994817 26202 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:44.967306 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.977051   846 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:44.976320 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:44.997706 26201 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 14.
I20260709 12:26:44.971661   806 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:44.970321 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:44.998657   806 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:44.999104   846 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.000494 26123 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:44.972856   794 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:44.968758   645 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.000874 26785 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:45.001402 26782 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:44.975093 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:44.981735 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.971832 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.009734 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.011466 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:45.004066 26202 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:45.003464 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.013587 26196 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:45.015105 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.018026 26790 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 12:26:45.014761 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 221881us
I20260709 12:26:45.001718   822 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:45.027984 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.018052 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.026904 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.024047   796 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:45.001675   794 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.030645   793 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:45.032472   794 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:44.985854   517 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:44.977811   558 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.035529 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.035837 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:45.021749 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:44.977860   748 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.029139 26787 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:45.036988 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:45.042482 26128 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.979782 25944 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 10 to 16, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 16 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:45.040428 26048 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:44.999675   806 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.049995 26784 raft_consensus.cc:2368] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17.
I20260709 12:26:45.064412 26785 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.027832   776 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:26:45.074129 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.075039 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.063704   577 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:45.087873   881 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.088639   881 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.092620 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.093380 26787 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:45.086624   517 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.096135   840 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:45.099521   748 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.070066   761 consensus_queue.cc:1243] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:45.100440   776 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:26:45.087899 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59871.3us
I20260709 12:26:45.104147 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.073279   558 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.105397   558 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:45.105824   558 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.106238   558 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:45.107159 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.108594   822 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:45.109213   822 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:45.110219   822 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.117437 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.117977 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:45.118419   793 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:45.118578 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.118917   793 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.119362 26789 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:45.119441 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.121152 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.120945   793 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.122308 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.124861 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.125548 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.127727 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.126324 26787 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:45.134181 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.061915   548 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:26:45.116010 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:45.133972   796 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:45.145187 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.123668   305 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.148669 26123 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:45.147859   794 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.151098   794 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.145677   796 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.156745   796 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:45.161427   411 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.161937   411 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.162339   411 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.170547 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.175532 26128 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:45.182080 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.170099 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:45.185233 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 236574us
I20260709 12:26:45.194299 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:45.201153 26785 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.221226 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:45.236222 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.243531 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.173938   881 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:45.194442   305 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:45.244421 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.233779 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.245262 26784 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:26:45.245637 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 270732us
I20260709 12:26:45.244295 26126 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:45.255019 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.257012 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:45.251201   610 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:45.258056   610 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.258457   610 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.261022   794 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.262605 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.263245 26050 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:26:45.263863 26785 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:26:45.269490 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.271541 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:45.249544   305 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.273924 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.311820 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.312691 26049 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:45.314846 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.324604 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.325098 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.325680 26784 raft_consensus.cc:2368] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17.
I20260709 12:26:45.327921 26053 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:45.311793   891 raft_consensus.cc:2764] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:45.306799   305 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.335006 26045 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:45.336681 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.339524 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.340904 26124 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:45.316071   818 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:45.350034 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.353994 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.355064 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.355325 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.357239 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:45.351872 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.361670 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.362509 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:45.327108   794 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:45.366470 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 113737us
I20260709 12:26:45.386327 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.388281 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.389447 26128 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:45.390461 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.395560 26123 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:26:45.396180 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.351809   715 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.406940 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.251456 25944 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:45.364487   743 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:45.412952 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:45.417233 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.417610 26787 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:45.425458   840 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:45.425985   840 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:45.426970   840 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.381769   548 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.429953 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.381403   883 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:45.392493   874 raft_consensus.cc:2764] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 12:26:45.441015 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47166.6us
I20260709 12:26:45.397899   678 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:45.442482 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 175470us
I20260709 12:26:45.428225   840 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:45.454191 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.397614   794 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.416792   715 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.468699   903 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.468518   516 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.478189   903 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.478706   516 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.479598   806 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.456287   711 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:26:45.464426   761 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:26:45.481518   761 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:45.443179   883 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.481966   761 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:45.482183   883 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:45.483093   761 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.483755 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:45.476622   548 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.485035 26784 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:45.485440 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:45.464339 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 500426us
W20260709 12:26:45.461063 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.436972   822 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:45.487797 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.417313   743 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:45.495627 26120 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.044s	sys 0.019s
W20260709 12:26:45.496028 26120 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.044s	sys 0.019s
I20260709 12:26:45.496989 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2774) took 1202 ms. Trace:
W20260709 12:26:45.463618   880 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:26:45.468765   888 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
W20260709 12:26:45.466768   882 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:26:45.406905   644 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.080s
W20260709 12:26:45.469621   892 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:26:45.505908 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.428828   796 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.513175 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.489715   874 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:45.466893   714 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.513983 26128 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:45.519133 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:45.504024   680 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:45.493100   846 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:26:45.507925 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 152650us
I20260709 12:26:45.508940   519 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:45.499492 26120 rpcz_store.cc:276] 0709 12:26:44.294522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.302411 (+  7889us) service_pool.cc:224] Handling call
0709 12:26:44.326526 (+ 24115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.326947 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:44.640178 (+313231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:44.972975 (+332797us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:26:45.496501 (+523526us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:45.496588 (+    87us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:45.496659 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.496769 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.496929 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72521,"prepare.queue_time_us":69961,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":81028,"raft.run_cpu_time_us":983,"raft.run_wall_time_us":14696,"spinlock_wait_cycles":31874304,"thread_start_us":1098489,"threads_started":3,"wal-append.queue_time_us":324485}
I20260709 12:26:45.506505   761 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.504522   577 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:45.534739 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.542752 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.494138   678 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.548415 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:45.532334   776 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:45.551608 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51981.6us
I20260709 12:26:45.559046 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.537277   519 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:45.549229   822 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:45.545817   577 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:45.563876 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.583971 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.584960 26120 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:45.586050   548 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.590432   519 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.555205   903 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.563417   776 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:45.597154 26019 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.051s (SENT)
I20260709 12:26:45.602015   906 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:45.596671   905 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:45.609364   558 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 12:26:45.662025 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.576556   822 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.611384   793 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.576617   714 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.614058 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2777) took 1259 ms. Trace:
I20260709 12:26:45.616830 26195 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:45.618276   411 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:45.620394   748 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.000s
I20260709 12:26:45.621510 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.623906 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.624387 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.625196 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.625958 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:45.705199 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 117333us
I20260709 12:26:45.626909 26194 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:45.658234   516 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:45.628325 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.629084 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.473127 25944 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:45.692792   822 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:45.635447 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98587.8us
I20260709 12:26:45.636067   715 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:45.642194   898 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:45.649592   794 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.650977 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.697280 26127 rpcz_store.cc:276] 0709 12:26:44.356936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.363122 (+  6186us) service_pool.cc:224] Handling call
0709 12:26:44.540554 (+177432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.606448 (+ 65894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.606484 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:44.656115 (+ 49631us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.045650 (+389535us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:45.045714 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.060679 (+ 14965us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:45.613833 (+553154us) raft_consensus.cc:1685] finished
0709 12:26:45.613855 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.613995 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2209,"raft.queue_time_us":13154,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":374157,"threads_started":1,"wal-append.queue_time_us":573583}
I20260709 12:26:45.716554 26049 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:45.652019 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.652462 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.652613 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.653365 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.654040 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.657761   796 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.659268   906 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.661736   903 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:45.661218   776 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.701220   714 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.627540 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.662619   558 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.707315 26051 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 20 because replica has last-logged OpId of term: 18 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 7.
I20260709 12:26:45.703701 26200 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:45.709378 26197 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:45.704826 26199 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:45.714206   715 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:45.632798   743 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:45.710594 26203 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 16: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:45.712810 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.736276 26198 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:45.736063 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 154271us
I20260709 12:26:45.711843 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:45.696048   711 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:26:45.697096   680 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:45.640045   610 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.719635 26052 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:45.720146 26019 leader_election.cc:336] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.081s (SENT)
I20260709 12:26:45.748427 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:45.716122 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:45.717566 26047 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:45.608044   305 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.725349 26048 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:45.692669   874 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:45.726339 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.658191   883 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.710713   644 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.732671 26046 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: [18.14-20.15]
W20260709 12:26:45.727674 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 130201us
I20260709 12:26:45.734261 26201 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:45.755797 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:45.735594   714 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:45.759675 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.761502 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.729609   906 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.749081   846 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:45.765568 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.762104 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.775143 26049 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:45.772298 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.773450 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:45.774577 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.772183 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.778469 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.779002 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:45.779173 26200 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:45.736297   903 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:45.768985 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:45.760315   680 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:45.788828 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:45.789924 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:45.792701 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:45.759447   818 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:45.785672 26018 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.803856 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:45.778709   885 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:45.780473   548 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:45.744318   748 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:45.807874 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.823750 26194 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:45.808305   678 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.810330 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:45.810822   911 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:45.811025   516 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.811380   610 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.856777   903 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:45.859313 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51283.7us
I20260709 12:26:45.812520   910 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:45.813397 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:45.817157   793 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:45.820387   900 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
W20260709 12:26:45.822388 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.825819 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:45.826896 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 112433us
I20260709 12:26:45.795878   881 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:45.834270 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48712.1us
I20260709 12:26:45.829655   846 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.843537   874 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.845486   883 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:45.900866 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75407.3us
I20260709 12:26:45.846349   519 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:45.902297 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.847998   794 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:45.848459   796 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:45.848800   905 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:45.849011   877 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.849104   411 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:45.861276 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94897.7us
I20260709 12:26:45.790665   913 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 12:26:45.885169 26124 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:26:45.888692 26785 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:45.895751 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:45.822306   885 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:45.899886 26019 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.107s (SENT)
I20260709 12:26:45.894419 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:45.910980 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.772027   644 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:45.911144 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50644.8us
I20260709 12:26:45.939728 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:45.940560 26051 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:45.871348   610 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:26:45.893735   881 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.935643 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:45.937908 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43576.9us
I20260709 12:26:45.944656 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:45.951011 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.911711   794 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:45.912099   516 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:45.909907   796 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.884966   919 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
W20260709 12:26:45.960636 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 198587us
W20260709 12:26:45.954182 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 152213us
I20260709 12:26:45.967402 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:45.951694   881 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:45.969739 26052 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:45.943039   885 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:45.963337   411 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.981384 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:45.965716   796 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.993455 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:45.994668 26127 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:45.995193 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:45.986791 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50968.5us
I20260709 12:26:45.964293   516 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:45.988180 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:46.001744 26784 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:46.001977 26785 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:45.972298   793 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.004024 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.004650 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.004750 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.005681   711 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:46.019093   711 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:46.006100   886 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:46.006968 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.008575 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.008898   846 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:46.012655   305 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:45.971513   577 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.014458 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.014878   743 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:45.981524   548 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.040849   548 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.043265   548 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:45.983824   877 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:46.045519   906 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.028478 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.046806 26120 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 14.
I20260709 12:26:46.045532   920 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:46.048779   920 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.049567   920 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:45.999379   516 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.027019 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.013375   794 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:46.034061 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 134237us
I20260709 12:26:46.034170 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.036072 26124 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.963984 25944 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 5 to 14, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 14 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.033207   926 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.028716   905 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:46.065620   912 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:26:46.067464   761 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.004s
I20260709 12:26:46.016884   678 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:46.068876 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88135.5us
I20260709 12:26:46.069484   806 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.029914   886 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:46.029626   919 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.073033   577 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.074054   558 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.076737   644 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.032655 26127 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:26:46.085810 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58859.2us
W20260709 12:26:46.068121 26019 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.640s (SENT)
I20260709 12:26:46.104117 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:46.106137 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 112750us
I20260709 12:26:46.048003   906 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.033505   610 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.059775   776 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:46.122797 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 187310us
I20260709 12:26:46.218931 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.124173 26126 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:46.237093 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.176548 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:46.238353 26785 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:46.158492 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.248175 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:46.249253 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.051088   920 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.098888   912 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.250671 26048 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:46.257697 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.250527 26121 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:46.252225 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.077706   921 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:46.077275   905 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.306985 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.308028 26049 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:46.308224 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56119.2us
I20260709 12:26:46.123781   886 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:46.076823   926 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:46.264933 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.123135 26124 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:46.265021 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:46.103750   913 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:46.211933   840 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.733s	user 0.005s	sys 0.009s
I20260709 12:26:46.202011   794 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:46.326411 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.337355 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.343652 26045 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.339548 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.345219 26785 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:46.130254   678 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.259701 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.338681 26047 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.339169 26046 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:46.339383 26051 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:46.347216 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.355046 26050 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:46.356798 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.357435 26046 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:46.347853 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.306912   644 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.099386 25944 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:46.360935 26054 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:46.232040   891 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.110s
I20260709 12:26:46.212379 32546 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.129s
I20260709 12:26:46.125085   680 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.434594   680 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:46.169950   714 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.337455   929 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:46.196556   822 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:46.369131 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150318us
I20260709 12:26:46.437412 26129 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:46.340396   902 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:46.394040 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.398855 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 330737us
I20260709 12:26:46.338989   919 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.341925   926 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.344745   761 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.443296 26200 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:46.428452 26196 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:46.344777   921 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:26:46.428849 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 89389.5us
I20260709 12:26:46.338449   793 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:46.428815 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:46.343129   913 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.200604   711 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.451833 26045 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:46.366937   411 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:46.453823 26019 leader_election.cc:336] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.180s (SENT)
I20260709 12:26:46.454404 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.428081   305 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.467904 26046 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:46.472167 26050 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:46.442075   678 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.477977 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:26:46.474161   927 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.437s	user 0.003s	sys 0.005s
W20260709 12:26:46.473155 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 213572us
I20260709 12:26:46.437059   714 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.472729 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.483495 26052 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.472630 26049 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:46.438439   610 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:46.485445 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.442780   903 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:46.455281   921 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:46.464818   913 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.491564 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.468585   846 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:46.494139 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.495400 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:46.497356 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.498307 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.498582 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:46.499868 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46102.2us
W20260709 12:26:46.500401 26019 connection.cc:256] RPC call timeout handler was delayed by -1.2592215679997025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2917s for a response
W20260709 12:26:46.500952 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.502844   743 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:46.502995 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:46.504431   902 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.504951   924 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:46.505981 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:46.506633 26053 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.504014 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:46.512691 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.512876 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.455189   919 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.493636   793 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.491935   761 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:46.477357   305 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.478971   411 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.495270   610 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.531167   933 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:46.535547   881 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.003s
I20260709 12:26:46.540287 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.539764 26053 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:46.517925   743 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.491593   903 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.542001   938 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.522079   822 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.527854   748 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:46.559305 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.545301   846 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:46.520973   709 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:26:46.523046   902 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:26:46.555199   840 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:46.569977 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 57174.9us
I20260709 12:26:46.559757   776 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.551460   881 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.566906   748 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:46.558838   519 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:26:46.508342 25944 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
W20260709 12:26:46.579428   885 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.597s	user 0.006s	sys 0.005s
W20260709 12:26:46.576620 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 98697.1us
I20260709 12:26:46.560228   818 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:26:46.592991   305 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.593755   919 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.588732 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.589761   748 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.594956 26167 leader_election.cc:400] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 16 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 16. Current term is 17.
I20260709 12:26:46.585819   610 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.581048   904 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:46.582960   923 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:46.605996 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.604911 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:46.607992 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:46.610776 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.611461 26128 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:46.598133 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:46.611557 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:46.614260 26127 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:26:46.615156 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.615764 26124 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.601622 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.617152 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.618235 32638 raft_consensus.cc:2764] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:46.620623 26120 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:46.621474 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:46.625922 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:46.626930 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47071.3us
I20260709 12:26:46.605230   846 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:46.627663 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.628942 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.618815 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:46.637874 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.635167 26124 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:46.600589   885 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.616418   940 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:46.621107   942 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.629925   644 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:46.659263 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71194.3us
I20260709 12:26:46.611109   919 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:26:46.684163 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.647624   903 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.660658   411 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.661044   806 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:46.664778   922 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:46.669313 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:46.672392   910 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.776s	user 0.008s	sys 0.004s
I20260709 12:26:46.616698   305 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:46.678717 26047 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:46.630649   936 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.619274   923 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:46.652814   519 raft_consensus.cc:2764] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:46.632331   846 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:46.641846   938 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:46.626053   904 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:46.679870   516 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.560s	user 0.005s	sys 0.007s
I20260709 12:26:46.642279   943 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:46.691403   822 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.692906   931 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:46.694273 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:46.654783   940 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:46.700330 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.645483   941 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:46.652632   902 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:26:46.709517 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:46.720285 26202 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:46.778808 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.781312 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.781385   904 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.782068   925 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:46.719715 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:46.721128 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:46.784277 26195 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:46.719519 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1147) took 1063 ms. Trace:
I20260709 12:26:46.720679 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:46.785583 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.786299 26199 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:26:46.703869   935 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 12:26:46.785015 26196 rpcz_store.cc:276] 0709 12:26:45.656084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.818351 (+162267us) service_pool.cc:224] Handling call
0709 12:26:45.818633 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.443863 (+625230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.719277 (+275414us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:46.719462 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1319582976}
W20260709 12:26:46.702849   937 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.229s	user 0.000s	sys 0.002s
I20260709 12:26:46.787441   925 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:46.686503   776 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:26:46.707412   938 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.812178 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:46.812609 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103160us
I20260709 12:26:46.797659 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.801954 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:46.807482 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 123362us
I20260709 12:26:46.814215 26194 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:46.699213   517 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:46.843025   517 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.843444   517 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.848395   558 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:46.849695   558 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.850868   558 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.713301   910 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:46.815702 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:46.853572   919 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:46.854529   919 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.855682   939 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.267s
W20260709 12:26:46.857048 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 162302us
I20260709 12:26:46.856146   919 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.718081   903 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.782704   519 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.824635 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:46.858841 26200 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:46.707721   516 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.861032   516 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:46.861711 26787 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:46.861510   926 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:46.862000 26094 connection.cc:256] RPC call timeout handler was delayed by -2.3774735890001466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:46.862893 26786 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:46.705113   923 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.698925   840 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.113s
I20260709 12:26:46.863732   926 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:46.714517   881 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.725284   900 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:46.787693   411 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:26:46.781971 25944 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 3 to 15, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 15 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:46.802194   906 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:46.777256   305 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.836460   680 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:46.841423   886 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.842120 26052 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:46.842483 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.853703 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:46.707370   846 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:46.857816 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.858930   903 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.719265   822 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.726598   940 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:46.787916   874 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:46.869580 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 240665us
I20260709 12:26:46.889899 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.873164   900 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:46.718901   912 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.707916   943 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.886901   929 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.776403   941 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:46.866640   938 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:46.881992 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1992105449999144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 12:26:46.871248   906 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:46.888204 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.889061   822 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.686570   942 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.892058 26045 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:46.893497 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.904537 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.895188   943 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:46.897765   929 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.898346   941 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.909312 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.895354   881 raft_consensus.cc:2764] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:46.895548   840 raft_consensus.cc:2764] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:46.885262   714 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.915266 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:46.886096   715 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.885311   680 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:46.906128   796 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.890s	user 0.010s	sys 0.002s
I20260709 12:26:46.912391 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:46.890851   925 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.913121 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:46.929973   925 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:46.930330 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:46.930883 26782 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.915671 26785 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: [20.15-22.16]
I20260709 12:26:46.890705   793 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.932164 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:46.932134   793 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.907685 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:46.935150   793 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.918110 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:46.936203 26788 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:46.914892   776 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:46.936825   949 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:46.937083   776 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:46.919795 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:46.937505   776 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.937780 26196 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.938690   942 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.939831   885 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.940488   885 raft_consensus.cc:2764] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:46.914525   519 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:46.925522 25944 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:46.942584 25944 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:46.927552   714 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.895452   874 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.945806   714 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.907205   906 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:46.958076   900 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.929059   796 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.960709   411 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:46.961308   411 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:46.962321   796 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.962188   411 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.927767   680 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.946159   874 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:46.944173 25944 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:46.965368   913 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.965695 25944 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 12 to 13, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 13 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.918952   912 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:46.968936 25944 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:46.969323   921 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.949421   903 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.946316   941 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:46.949980 32546 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:46.951306   886 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:46.950680   776 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:46.949501 26196 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:46.912739   922 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:46.923622   822 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:46.998483 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81000.9us
W20260709 12:26:46.915937   794 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 12:26:46.972419 25944 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:47.046844 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:46.986822   941 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:46.987291 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:46.990545 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:46.982262 26784 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:46.976014   610 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:47.053871   610 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:47.047972 26194 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:47.055176 26196 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:47.049397 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:47.052943 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62519.5us
I20260709 12:26:47.055855   954 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:47.052862 26201 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:47.053917   874 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:47.057415   874 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:47.058444   874 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.058560 25944 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:26:47.060786 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:47.061573 26202 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:47.062403 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:47.062572 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:47.062973 26202 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:47.063225 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:47.054988   610 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.056731   941 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:47.053239   776 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.066300 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:47.080511 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:47.076735 26782 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:26:47.001665   920 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.743s	user 0.005s	sys 0.006s
I20260709 12:26:47.082007 26198 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:47.085618 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 197529us
I20260709 12:26:46.916353   948 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.168s
I20260709 12:26:47.090793 26199 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:26:47.093374 26199 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 12
I20260709 12:26:47.093686 26199 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:26:47.093948 26199 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.048337   822 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:47.099931 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:47.085299   794 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:47.084571   926 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:47.105058 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 936) took 1132 ms. Trace:
I20260709 12:26:47.106727 26051 rpcz_store.cc:276] 0709 12:26:45.972623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.982276 (+  9653us) service_pool.cc:224] Handling call
0709 12:26:46.015881 (+ 33605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.354881 (+339000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.104023 (+749142us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002d0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:47.104997 (+   974us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678894976}
I20260709 12:26:47.085364   929 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:47.107806 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:47.085978   913 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:47.111756 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:46.996179   305 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.115944 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1677) took 1147 ms. Trace:
I20260709 12:26:47.116456 26787 rpcz_store.cc:276] 0709 12:26:45.965170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.965771 (+   601us) service_pool.cc:224] Handling call
0709 12:26:45.980023 (+ 14252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.862111 (+882088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.112595 (+250484us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54000e2c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:47.112768 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027197184}
I20260709 12:26:47.083707   941 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.118427 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:47.105268   714 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:26:47.121384 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 927) took 1469 ms (client timeout 1669 ms). Trace:
W20260709 12:26:47.121971 26054 rpcz_store.cc:269] 0709 12:26:45.659136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.661733 (+  2597us) service_pool.cc:224] Handling call
0709 12:26:47.121162 (+1459429us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54001b49d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:47.121305 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605215104}
I20260709 12:26:47.118588   411 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:47.109658   920 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:26:47.147511 26094 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:47.161437 26094 leader_election.cc:403] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:47.155485 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:47.159063 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71589.5us
I20260709 12:26:47.159153 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:47.160063 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.114527   923 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:47.113930   902 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:26:47.167555 26052 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:47.171257 26047 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:26:47.180881 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.187664 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:47.186760 26053 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:47.190001 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.190269 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.105957   822 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.187057   516 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:47.194191 26051 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:47.173801 26052 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:26:47.077698 25944 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 14 to 15, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 15 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:47.162204   709 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:26:47.248761   711 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:47.169086   776 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:47.194326   516 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.252803   953 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:26:47.146505   904 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:26:47.257315   943 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.258581   709 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:47.263170 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.194465   939 raft_consensus.cc:2764] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:47.265367 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.318794 26788 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 12:26:47.263952 26202 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:47.273275 26053 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:47.169401   743 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.293431 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:47.320362 26196 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:47.295794 26052 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:26:47.162968   411 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:47.299680   906 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.192677   910 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.122s
I20260709 12:26:47.331349 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:47.331961 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:47.319222 26788 pending_rounds.cc:85] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 17
I20260709 12:26:47.344677 26788 pending_rounds.cc:107] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 12:26:47.294631   891 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:47.345131 26788 raft_consensus.cc:2889] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.341766 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:47.307574 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.331557 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:47.329005 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 157054us
I20260709 12:26:47.322516 26783 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:47.322491 26046 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:47.308884 26047 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:47.348817 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:47.320914   709 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:47.348094 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:47.352561 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.334825 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:47.354182 26202 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:47.355129 26202 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:47.355623 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:47.361914 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.297946   516 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:47.368037 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:47.371724 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.374399 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:47.374568 26050 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:47.372902 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.377660 26052 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
W20260709 12:26:47.380429   966 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:26:47.380975 26194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:47.381788 26194 raft_consensus.cc:606] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:47.382349 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.382987 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
W20260709 12:26:47.383450   933 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.836s	user 0.010s	sys 0.000s
I20260709 12:26:47.383947   933 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:47.384380   952 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:26:47.386040 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:47.386741 26196 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:47.387037   411 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.388764   709 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.383448 26194 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:26:47.391662 26789 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:47.367201 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:47.392362 26197 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:47.403364 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.341336   886 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.406002 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.296458   776 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.343261   925 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:47.406462 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 259132us
I20260709 12:26:47.413045 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:47.414472 26050 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:47.367831   948 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:47.376773   840 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:47.385970   906 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:47.390190   709 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:47.316555   940 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.404886   881 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:47.424994 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.504439 26046 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
W20260709 12:26:47.425840   846 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.000s
W20260709 12:26:47.426712 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.391402   913 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:47.391147   714 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:47.435534   517 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.003s
W20260709 12:26:47.436784   938 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.001s
I20260709 12:26:47.348696   743 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.424852 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3659) took 1066 ms. Trace:
I20260709 12:26:47.518831 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.440521 26047 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:47.449178 26196 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:47.435001 26049 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:47.438216 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.509107 26782 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:47.377130   519 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 12:26:47.523368 26048 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:47.437616   948 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:47.518693 26045 rpcz_store.cc:276] 0709 12:26:46.358654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.472683 (+114029us) service_pool.cc:224] Handling call
0709 12:26:46.486233 (+ 13550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.904499 (+418266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.424580 (+520081us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54002d05d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:47.424775 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932617984}
I20260709 12:26:47.541774 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.542690 26048 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:47.438308 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.530457 26790 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:47.585181 26050 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:47.413509   886 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.612708 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.512959   904 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.505297   881 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:47.614197 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.437812   925 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.635068 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.497916   891 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:26:47.532346   714 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:47.407250   818 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:47.461326   558 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.071s
W20260709 12:26:47.665115 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 274208us
I20260709 12:26:47.666046 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:47.638773 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.585181   913 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:47.667371 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:47.664422 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:47.525707   517 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.647516   881 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:47.677723 26046 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:47.525090   938 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.647142   519 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:47.661599   818 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:47.648159   904 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.587123   885 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:26:47.518622   846 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.658569   910 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:47.732560 26195 raft_consensus.cc:2368] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23.
I20260709 12:26:47.735875 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:47.739665 26126 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:26:47.439083   905 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.740007 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:47.740726 26124 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:47.743873   910 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.744302 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:47.749123   965 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:26:47.750233   846 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.753294 26124 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:26:47.756309 26129 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:47.758774 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:47.761278 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:47.765213 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:47.773051 26122 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:47.773443 26122 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:47.741425   930 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.282s	user 0.003s	sys 0.007s
W20260709 12:26:47.777590   610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.002s	sys 0.003s
I20260709 12:26:47.782420   891 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:47.787999   902 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:47.788803   976 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:47.731220   822 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:47.801493 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:47.737138   971 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.801332 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:47.737677   806 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.842159   519 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.804978 26122 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.772161   305 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:47.806572   610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.002s	sys 0.003s
I20260709 12:26:47.859211 26122 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:47.853437 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66125.3us
I20260709 12:26:47.851235   905 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:47.682184   948 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.871222 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:47.860867   776 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:47.884465 26785 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:47.884888 26788 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:26:47.888397 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 221085us
I20260709 12:26:47.849727   818 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.890154   818 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.893229 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:47.883373   912 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.916s	user 0.012s	sys 0.001s
I20260709 12:26:47.864099   986 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:47.857868   516 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.903366 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:47.856905   970 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:47.902299 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:47.858037   981 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:26:47.857191   806 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.863132   305 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.886880   948 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:47.849550   714 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.882370   743 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.885780   940 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.904395   986 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.859866   519 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.908658 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:47.868598   913 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:47.938068 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:47.944077 26784 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:47.905642   886 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.914063   976 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:26:47.954617   974 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:26:47.914937   969 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:26:47.914115   922 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:26:47.957414   922 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:47.958033   922 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:47.959824   806 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:47.916903   977 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:26:47.916738   970 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:47.920220   904 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:47.921553   967 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:26:47.921926   973 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:47.856679   971 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.987046   971 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:47.934619   840 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:48.001009   840 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:48.002099   840 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.003553   840 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:47.935575   881 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.006049   881 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.972574   776 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:47.946045 26045 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:47.943336   986 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:47.910437 26123 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:26:47.950569 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47302.5us
I20260709 12:26:48.010761 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:47.960717 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.019508   305 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:47.961030   902 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:47.971455 26049 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:47.976743 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.980335 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:47.942323   743 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:47.940587   517 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:47.970611   970 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:47.959384   922 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:47.987843   963 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.989027   680 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:47.991711   955 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.744s	user 0.000s	sys 0.011s
I20260709 12:26:47.931459   921 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:47.912078   925 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:47.973425   938 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:48.069264 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 58579us
I20260709 12:26:48.070322 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:48.070051   680 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.072428   776 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:48.072774   680 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:48.073316   776 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:48.073729   776 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:48.074589   776 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.025722   881 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:47.969400 26048 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 15.
I20260709 12:26:48.077381 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:48.077896 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.079435 26199 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:48.071842   743 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:48.095332 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:48.056093   940 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.098845 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:48.102411 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:48.056475   902 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:48.099696 26788 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:48.099615 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.076014   776 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:48.103592 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.087812   971 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.103439 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.121295 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:48.121829 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.011s	sys 0.022s
W20260709 12:26:48.122202 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.011s	sys 0.022s
I20260709 12:26:48.122303 26122 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 15: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:48.078481   963 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.127614 26194 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:48.131012 26048 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:26:48.133056 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2367) took 1073 ms. Trace:
I20260709 12:26:48.133970 26050 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:26:48.134949 26200 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:48.121523   911 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:48.133486 26197 rpcz_store.cc:276] 0709 12:26:47.060112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.349838 (+289726us) service_pool.cc:224] Handling call
0709 12:26:47.380724 (+ 30886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.400679 (+ 19955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.132772 (+732093us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5400399c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:48.132985 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26300288}
I20260709 12:26:48.137782 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:48.126775   678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.035s	user 0.004s	sys 0.019s
I20260709 12:26:48.141731 26046 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:48.106123   940 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.144781 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:48.145577 26046 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:48.147490 26048 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:48.127404   516 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.177299   516 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.177857   516 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.178334 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1155) took 1023 ms. Trace:
I20260709 12:26:48.178719 26194 rpcz_store.cc:276] 0709 12:26:47.154640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.516420 (+361780us) service_pool.cc:224] Handling call
0709 12:26:47.584529 (+ 68109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.133112 (+548583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.178081 (+ 44969us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400335254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:48.178278 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245754240}
W20260709 12:26:48.182443 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.151624 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:48.185539 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2742) took 1092 ms. Trace:
W20260709 12:26:48.186656 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.185890 26202 rpcz_store.cc:276] 0709 12:26:47.092691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.356458 (+263767us) service_pool.cc:224] Handling call
0709 12:26:47.393578 (+ 37120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.414029 (+ 20451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.414061 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.424620 (+ 10559us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.891657 (+467037us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:48.093916 (+202259us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.094189 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.094270 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.185335 (+ 91065us) raft_consensus.cc:1685] finished
0709 12:26:48.185354 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.185491 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2863,"prepare.queue_time_us":200907,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2003,"raft.queue_time_us":2489,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":15898,"thread_start_us":304095,"threads_started":1,"wal-append.queue_time_us":575293}
I20260709 12:26:48.152807 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:48.159066 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:48.161911   991 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 12:26:48.168305   947 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.067s	user 0.008s	sys 0.007s
W20260709 12:26:48.169638   983 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:26:48.171089   941 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:48.180033   516 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.184509 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:48.227689 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:48.200160   925 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:48.200755 26126 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:48.234541 26126 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:48.201409 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2358) took 1207 ms. Trace:
I20260709 12:26:48.202016   939 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:48.202230 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1208) took 1297 ms. Trace:
W20260709 12:26:48.202488   711 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.908s	user 0.007s	sys 0.005s
I20260709 12:26:48.203910 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1207) took 1299 ms. Trace:
I20260709 12:26:48.204919 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1151) took 1298 ms. Trace:
I20260709 12:26:48.205055   940 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.140743   911 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:48.125607   971 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:48.206321   988 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:26:48.207445 26045 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:48.208199 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:48.259312 26120 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.208963 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.223069   846 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.210527 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.210726 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:48.264513 26046 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:26:48.181710   709 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.005s
I20260709 12:26:48.174811 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:48.239642   929 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:48.226891 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.232909 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.106598   970 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.236934   519 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.207926   954 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:26:48.238401 26200 rpcz_store.cc:276] 0709 12:26:46.906163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.906860 (+   697us) service_pool.cc:224] Handling call
0709 12:26:46.922409 (+ 15549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.144178 (+1221769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.204669 (+ 60491us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54002b4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:48.204851 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788005760}
I20260709 12:26:48.228595 32546 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:26:48.235293 26195 rpcz_store.cc:276] 0709 12:26:46.993964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.995240 (+  1276us) service_pool.cc:224] Handling call
0709 12:26:48.201180 (+1205940us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54002189d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:48.201342 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574678400}
I20260709 12:26:48.237578 26789 rpcz_store.cc:276] 0709 12:26:46.904354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.904465 (+   111us) service_pool.cc:224] Handling call
0709 12:26:46.929090 (+ 24625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.402007 (+472917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.203690 (+801683us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:48.203853 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063341440}
I20260709 12:26:48.248281 26784 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:48.249231   911 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.252012   971 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.209836 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1690) took 1692 ms. Trace:
I20260709 12:26:48.224463   933 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.262452 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.291460 26790 rpcz_store.cc:276] 0709 12:26:46.517514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.517658 (+   144us) service_pool.cc:224] Handling call
0709 12:26:46.528306 (+ 10648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.585742 (+1057436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.209614 (+623872us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54005ce254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:48.209781 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304216832}
W20260709 12:26:48.224352   900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.019s	sys 0.006s
I20260709 12:26:48.146178   517 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:48.278003 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51174.3us
W20260709 12:26:48.143419   678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.005s	sys 0.019s
W20260709 12:26:48.280455 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47615.7us
I20260709 12:26:48.235541 26783 rpcz_store.cc:276] 0709 12:26:46.904648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.906948 (+  2300us) service_pool.cc:224] Handling call
0709 12:26:47.318332 (+411384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.346966 (+ 28634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.202035 (+855069us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54004501d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:48.202167 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919814528}
I20260709 12:26:48.231331   680 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:26:48.235273   923 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:26:48.272794   711 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.279501   929 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.302616 26121 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:48.282013   939 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:48.314306 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.315233 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:48.263304   926 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:26:48.281570   881 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:48.317512 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:48.312466 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.266850   846 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.197400   902 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.302044   970 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:48.308848 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.365157 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.372189 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2316) took 1030 ms. Trace:
I20260709 12:26:48.373570 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.375264 26047 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:48.318105 26128 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:48.392431 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.392840 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2366) took 1333 ms. Trace:
I20260709 12:26:48.393204 26199 rpcz_store.cc:276] 0709 12:26:47.059827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.135175 (+ 75348us) service_pool.cc:224] Handling call
0709 12:26:48.392685 (+1257510us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b5400398d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:48.392786 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1709425920}
I20260709 12:26:48.394579 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.300921   919 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:48.300766   900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.019s	sys 0.007s
I20260709 12:26:48.418917 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.314807   929 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:48.419533 26194 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:48.420583 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.372795 26054 rpcz_store.cc:276] 0709 12:26:47.341782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.343949 (+  2167us) service_pool.cc:224] Handling call
0709 12:26:47.521195 (+177246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.521453 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:48.094667 (+573214us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.229795 (+135128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:48.229870 (+    75us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:48.229916 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.229984 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.372021 (+142037us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54000f8f50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:48.372132 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":691514,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":1788,"raft.queue_time_us":296,"raft.run_wall_time_us":9220,"spinlock_wait_cycles":336269312,"thread_start_us":569056,"threads_started":2,"wal-append.queue_time_us":163136}
I20260709 12:26:48.398885   992 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:26:48.424616 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:48.399725  1006 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:26:48.400593 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2314) took 1102 ms. Trace:
I20260709 12:26:48.426901 26051 rpcz_store.cc:276] 0709 12:26:47.297913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.298042 (+   129us) service_pool.cc:224] Handling call
0709 12:26:47.329231 (+ 31189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.642803 (+313572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.642845 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.651038 (+  8193us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.782792 (+131754us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:47.782839 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.376883 (+594044us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400173c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:48.376928 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.376974 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:48.376987 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.400505 (+ 23518us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8346,"raft.queue_time_us":133720,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":27208,"spinlock_wait_cycles":700870912,"thread_start_us":80807,"threads_started":1,"wal-append.queue_time_us":258426}
I20260709 12:26:48.400756   709 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:26:48.402420 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84978.3us
I20260709 12:26:48.428604   709 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:48.429504   709 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.430521   709 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:48.404465   924 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:48.432399   924 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:48.433317   924 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.434739   924 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:48.437307 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:48.438210 26154 heartbeater.cc:646] Failed to heartbeat to 127.24.71.190:42395 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.24.71.190:42395 timed out after 15.000s (SENT)
I20260709 12:26:48.409636   973 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:48.414177   943 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:48.415055   905 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.440523   905 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.443024   905 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:48.443753 26203 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:48.417116   922 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:48.417567   913 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.447319   913 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.449925   913 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:48.417819 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.450695 26200 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:48.403283 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:48.453249 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50071.8us
W20260709 12:26:48.453451  1004 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:26:48.454061   971 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:48.454553   971 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:48.455454   971 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.403191   904 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:48.476643 26198 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:48.418231 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:48.478175 26202 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:48.479256 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.463196 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:48.395946 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:48.467052 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.485592 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.301653   911 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.306573   979 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.472250 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.489975 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.494345 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:48.301525   933 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.498364 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.533394 26201 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:48.498963 26122 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:48.499790 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:48.536320 26194 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:48.505889 26200 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:48.299469   925 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.188s
I20260709 12:26:48.506819 26127 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:48.504700 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:48.532044 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.543931 26197 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:48.316838   711 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:48.544626  1001 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:26:48.545601 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:48.543135 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57668.3us
I20260709 12:26:48.474131   904 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:48.546804 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.555513   711 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.555735   933 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.492805   919 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:26:48.538255   818 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.006s
I20260709 12:26:48.558476 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:48.567866 26201 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:48.497223   979 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.533408   996 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.573551 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:48.574856 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:48.575419 26051 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:48.557716   938 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:26:48.558095   904 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:48.556854   743 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.556501   984 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:48.504140 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 108281us
I20260709 12:26:48.531124 26199 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:26:48.550237 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.474115   943 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:48.315395   517 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.543215 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:48.612706 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.551329 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:48.473454   926 consensus_queue.cc:1243] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:26:48.538458   948 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.003s
I20260709 12:26:48.563644   919 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.567430   910 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.566836   963 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:48.581079 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108925us
I20260709 12:26:48.623190 26791 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
W20260709 12:26:48.623916 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.557363   305 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:48.581152   905 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:48.532830   920 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.567589   939 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:26:48.566615   519 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:26:48.548120 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.611342 26196 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:48.613765 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.627832   776 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.609359   933 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.609457   900 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.617827 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:48.609284   941 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:48.631676 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 272551us
I20260709 12:26:48.614192 26053 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.620518   919 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.557880   881 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:26:48.620031 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:48.635908 26194 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:48.618348   984 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:48.667924 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:48.668850 26789 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
W20260709 12:26:48.672586  1009 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:48.674708 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:48.676136 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:48.677107 26783 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:26:48.643021 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 95000.1us
I20260709 12:26:48.678704 26052 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:48.679638 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.680742 26052 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:48.627406   305 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.682663   305 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:48.682886 26052 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:48.614259   743 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.690651   743 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.610517   996 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.693557   996 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:48.653702 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:48.693977 26784 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:48.668008   984 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.695674 26054 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:48.696472   846 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:48.698771 26054 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:48.623587   943 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.661473 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:48.666504 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:48.621822 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.702996 26120 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:48.714557 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.714915 26051 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:48.715324 26194 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
W20260709 12:26:48.715827 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54495.2us
W20260709 12:26:48.714437 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:48.721408 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173539us
I20260709 12:26:48.722844 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:48.723181 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.723551 26782 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:48.724390 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:48.747352 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.757915 26051 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:26:48.774581 26051 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:48.776736 26051 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 12:26:48.778515 26051 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:26:48.781203 26051 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:48.783437 26051 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:48.639773   920 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.789645   925 consensus_queue.cc:1243] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:48.722489   923 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:48.791484 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:48.791319 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65047.3us
W20260709 12:26:48.801631 26045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.007s	sys 0.023s
W20260709 12:26:48.802210 26045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.007s	sys 0.023s
I20260709 12:26:48.655052   776 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.805819 26094 leader_election.cc:400] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23.
I20260709 12:26:48.666424   933 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.781211   939 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.809888 26094 leader_election.cc:403] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:26:48.814448 26197 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:48.665971   900 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.822113 26052 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:48.814677   921 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:48.802649   680 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:26:48.836524 26129 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:26:48.809337   987 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:26:48.845604 26129 pending_rounds.cc:85] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 11
I20260709 12:26:48.850484 26129 pending_rounds.cc:107] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:26:48.850935 26129 raft_consensus.cc:2889] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:48.844897 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.796669   904 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.848721  1018 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:48.855846 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 176305us
I20260709 12:26:48.812281   902 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:26:48.820526   939 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.933810 26202 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:48.871511 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:48.868503 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:48.856528 26053 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:48.857049 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.877849 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.831653   925 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.912143  1018 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:48.873939   990 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.100s
I20260709 12:26:48.973951  1018 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.879218 26128 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:48.975842   925 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:48.930052   981 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:26:48.869344   517 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:48.872563 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 66944.5us
I20260709 12:26:48.913699   680 consensus_queue.cc:1243] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:48.881031 26198 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:48.981194  1018 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:48.982029 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:48.873651   920 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:48.984512 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 116077us
W20260709 12:26:48.985999 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 129032us
I20260709 12:26:48.987764 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:48.980118 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:48.978803 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:48.983726 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:48.986557 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:48.986716 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 115306us
I20260709 12:26:48.855196   970 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23.
W20260709 12:26:48.852752   986 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.800s	user 0.012s	sys 0.000s
I20260709 12:26:48.855638   913 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:48.849431  1010 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:48.997535 26198 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 15: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:48.997546 26199 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:26:49.002362 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.004447   981 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:48.838918   929 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:48.855065   984 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.017212 26203 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:26:49.012307  1018 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:48.980669   925 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:49.032986 26199 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.027248   881 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.034178   943 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.034499   996 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.034631   881 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.035336   996 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.042958 26198 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:48.996605   910 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:26:49.051388 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.872977   924 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.169s
I20260709 12:26:49.018647  1019 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:49.054561 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65444.7us
I20260709 12:26:49.017136   923 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.017524   939 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.057978 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:49.070468 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 82829.1us
I20260709 12:26:49.033138   986 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.056418  1019 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.037891   913 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.041775   984 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.098882 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.106567 26053 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:49.064795  1020 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.068815   881 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:49.096761  1014 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:26:49.123361 26048 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:26:49.132503 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.106861   776 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.098448  1019 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.120113   938 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:49.117965   818 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:49.186687 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 115396us
I20260709 12:26:49.138037   970 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:49.187584 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.187651 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:49.154852 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.190310 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2322) took 1058 ms. Trace:
I20260709 12:26:49.233052 26050 rpcz_store.cc:276] 0709 12:26:48.135056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.135189 (+   133us) service_pool.cc:224] Handling call
0709 12:26:48.136260 (+  1071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.136597 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.136626 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.226990 (+ 90364us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.494923 (+267933us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:48.540189 (+ 45266us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.540506 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.540599 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.189981 (+649382us) raft_consensus.cc:1685] finished
0709 12:26:49.190013 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.190204 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45668,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":9497,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":37764,"thread_start_us":222242,"threads_started":1,"wal-append.queue_time_us":386269}
W20260709 12:26:49.186219   922 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.000s
W20260709 12:26:49.242852 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55291.8us
I20260709 12:26:49.185869   938 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.249255 26123 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:49.248809 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.185186   900 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.135264  1020 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.260085 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:49.192832   970 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.236315   776 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.225595   971 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.146018  1021 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 12:26:49.273408 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:49.268729 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 211756us
I20260709 12:26:49.250541   996 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.251786   921 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:49.264094   938 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.283386 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.237795   818 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:49.276217  1015 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:26:49.265407  1029 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:49.255838  1018 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:49.277218   678 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:49.288656  1016 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:49.290904   929 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:49.290748   743 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:49.236707   923 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.296463  1028 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:49.310560  1024 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 12:26:49.279628   776 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.322710  1040 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:49.323390  1035 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:26:49.323604  1038 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:49.324450  1032 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
W20260709 12:26:49.324965  1037 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 12:26:49.326340  1034 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:26:49.329602  1020 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.330333   913 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.330464 26126 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:49.333532  1036 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:26:49.333813   919 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
W20260709 12:26:49.334843 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.278856  1026 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.176s
I20260709 12:26:49.279842   971 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.315986  1047 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:49.347630   986 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.370975 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2335) took 1213 ms. Trace:
I20260709 12:26:49.497208 26048 rpcz_store.cc:276] 0709 12:26:48.157058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.681419 (+524361us) service_pool.cc:224] Handling call
0709 12:26:48.681846 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.123935 (+442089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.370544 (+246609us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54000f8a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:49.370859 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014465408}
I20260709 12:26:49.378168   806 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:49.498415   806 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.498750   971 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.499344   806 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:49.385615  1011 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.658s	user 0.002s	sys 0.002s
I20260709 12:26:49.387506   517 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:49.502801   517 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.503942 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2331) took 1355 ms. Trace:
I20260709 12:26:49.504376 26046 rpcz_store.cc:276] 0709 12:26:48.148559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.273339 (+124780us) service_pool.cc:224] Handling call
0709 12:26:48.279342 (+  6003us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.279729 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.279758 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.286655 (+  6897us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.474788 (+188133us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:48.474828 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.474895 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.385163 (+910268us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b2400137078. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:26:49.503617 (+118454us) raft_consensus.cc:1685] finished
0709 12:26:49.503651 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.503879 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7185,"spinlock_wait_cycles":5763840,"thread_start_us":115795,"threads_started":1,"wal-append.queue_time_us":334802}
I20260709 12:26:49.393496 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2808) took 1083 ms. Trace:
I20260709 12:26:49.506354 26124 rpcz_store.cc:276] 0709 12:26:48.309910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.421562 (+111652us) service_pool.cc:224] Handling call
0709 12:26:48.421844 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.627597 (+205753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.627626 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:48.627666 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:48.627698 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.393131 (+765433us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:49.393202 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.393413 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472720256}
I20260709 12:26:49.507915 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.509130 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.510272 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.394487   940 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:49.396407   516 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.512396   516 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:49.513499   516 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.514882   516 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:49.398128  1042 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:26:49.405260   305 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.722s	user 0.013s	sys 0.000s
I20260709 12:26:49.511974   940 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:49.516817   305 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.518149   940 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.519446   940 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:49.409662   904 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.511423 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.521879   305 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.522769 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:49.415129 26053 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:49.417368 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:49.524722 26049 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.362376   984 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.418782   709 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:49.529527 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.460345   922 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:26:49.422454 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:49.533311 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 110950us
W20260709 12:26:49.445698 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 162430us
W20260709 12:26:49.429639 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.018s	sys 0.005s
W20260709 12:26:49.425668   954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.015s	sys 0.015s
I20260709 12:26:49.426632   921 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.452960   905 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:49.461252  1043 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.202s	user 0.002s	sys 0.000s
I20260709 12:26:49.489265 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2329) took 1344 ms. Trace:
I20260709 12:26:49.459488  1016 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:49.494871   976 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:49.563853 26122 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:49.524024 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.474584   743 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.455863   900 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:26:49.565130 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.473150   881 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.457806   987 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:26:49.565518  1019 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.566107   881 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.566951 26198 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:49.360608   941 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.663s	user 0.004s	sys 0.008s
I20260709 12:26:49.525439   919 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.568456   941 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:49.534538 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.018s	sys 0.007s
W20260709 12:26:49.569512 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 720211us
I20260709 12:26:49.531481 26120 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:26:49.570097   919 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:49.536612 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:49.570854 26199 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:49.571396 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.572072  1016 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:49.531601   992 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.032s	sys 0.060s
I20260709 12:26:49.545641  1033 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 12:26:49.573700   992 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.032s	sys 0.061s
I20260709 12:26:49.574162   971 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.574713   971 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.575071 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:49.576409   979 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.576841 26783 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:49.577030   971 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.577869 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:49.538339   954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.015s	sys 0.016s
I20260709 12:26:49.538195   305 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:49.552685  1056 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:49.554682   776 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.541527  1047 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:26:49.530395   709 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.541164   921 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:49.537667   984 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.533155  1038 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.492127   902 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:49.559062 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.473232  1028 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.648548  1028 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:49.561146   905 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:49.651542   905 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.645452   714 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:49.520459 26200 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:49.529027  1040 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.566450   743 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.572614   881 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.580143 26122 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:26:49.590652   678 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:49.535590 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:49.528442   963 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.033s	sys 0.070s
I20260709 12:26:49.545022 26045 rpcz_store.cc:276] 0709 12:26:48.144606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.257138 (+112532us) service_pool.cc:224] Handling call
0709 12:26:48.257548 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.257987 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.258025 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.258581 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.802646 (+544065us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:48.802698 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.802769 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.489015 (+686246us) raft_consensus.cc:1685] finished
0709 12:26:49.489038 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.489201 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":732,"raft.queue_time_us":2595,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15272,"thread_start_us":543569,"threads_started":1,"wal-append.queue_time_us":568223}
I20260709 12:26:49.584715 26784 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.545025  1018 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:49.587896 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.592868 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.567776  1019 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:49.594738 26049 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:49.603096 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.605260 26202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:49.690785 26202 raft_consensus.cc:606] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:49.691254 26202 raft_consensus.cc:614] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:49.607818 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:49.618129 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59157.1us
I20260709 12:26:49.645357 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.603144   986 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.655224  1038 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.578963 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:49.590914   305 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.601640   919 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:49.704568 26785 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:49.706835 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.693553 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.657037   776 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.658555   891 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.708788   305 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.709136   891 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.710042   891 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:49.668838  1051 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
W20260709 12:26:49.700925 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 165379us
I20260709 12:26:49.656991   987 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:49.692018 26201 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
W20260709 12:26:49.663571   963 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.033s	sys 0.070s
I20260709 12:26:49.661563   678 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.712654   919 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:49.728609 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.724821 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.714366  1054 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:49.671270 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56083.6us
I20260709 12:26:49.733747 26045 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:49.719045  1026 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:26:49.772926 26046 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:49.734519 26049 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:49.719866 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.774431 26120 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:49.664600   922 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.665174   743 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.745522 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.657797   984 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.664968   881 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.789269 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:49.789963 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.708863   776 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.755694 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.747184  1028 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.769062 26045 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:49.777055 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:49.760620 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:49.739076   933 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.737972   678 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:49.786866 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:49.774972   919 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.704952   902 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:26:49.808615 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.762655 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.752017  1054 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.812386 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.791846 26053 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:49.748178   971 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:49.719126   938 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.798110 26049 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.732403   987 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.798871 26048 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.714035   986 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.799336 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3676) took 1429 ms. Trace:
I20260709 12:26:49.799479 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.802062 26047 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.832609 26052 rpcz_store.cc:276] 0709 12:26:48.370391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.684045 (+313654us) service_pool.cc:224] Handling call
0709 12:26:48.684408 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.822726 (+138318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.799090 (+976364us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54000f9954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:49.799280 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315929472}
I20260709 12:26:49.783363 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3766) took 1286 ms. Trace:
I20260709 12:26:49.737537  1019 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:49.807394 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1565) took 1146 ms. Trace:
I20260709 12:26:49.742060   680 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:26:49.741277   924 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:26:49.815836 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3759) took 1664 ms. Trace:
I20260709 12:26:49.818212 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:49.832906 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.834225 26129 rpcz_store.cc:276] 0709 12:26:48.496555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.496724 (+   169us) service_pool.cc:224] Handling call
0709 12:26:48.497005 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.855029 (+358024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.778348 (+923319us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54001c4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:49.783250 (+  4902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770956032}
I20260709 12:26:49.840608 26123 rpcz_store.cc:276] 0709 12:26:48.151576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.151807 (+   231us) service_pool.cc:224] Handling call
0709 12:26:48.152077 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.253336 (+1101259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.815566 (+562230us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:49.815765 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2522379648}
I20260709 12:26:49.789912   743 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.708837  1040 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.786232   922 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.773589   709 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.797572   881 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.816620  1054 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.844718 26049 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
W20260709 12:26:49.845203 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 227175us
I20260709 12:26:49.846400 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.807991  1028 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.819211   933 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:49.847795 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.849033 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.843236 26787 rpcz_store.cc:276] 0709 12:26:48.661474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.662218 (+   744us) service_pool.cc:224] Handling call
0709 12:26:48.669958 (+  7740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.828444 (+158486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.828494 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:48.828556 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:48.840554 (+ 11998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.807140 (+966586us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:49.807187 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.807335 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_wall_time_us":16611,"spinlock_wait_cycles":374946944}
I20260709 12:26:49.822218   940 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:49.861303 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.528s (SENT) [suppressed 8 similar messages]
W20260709 12:26:49.862329  1059 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:49.863010   806 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:49.865499  1058 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:49.865345   517 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.868000 26047 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:49.870311   986 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.872910   987 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.873685   678 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:49.876915  1061 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:49.878312   984 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.824129   818 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.878132   970 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.880052   743 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.880900 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.882529 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.886958  1031 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.883813   840 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.884589   922 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.885825   776 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:49.811306   919 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.890909 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.891466 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:49.936055 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44707.4us
I20260709 12:26:49.936879 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.893514 26202 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:49.894150 26048 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:49.894167 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:49.898094 26046 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:49.893567 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 164609us
W20260709 12:26:49.940405 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46301.6us
I20260709 12:26:49.940657 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:49.941608 26194 raft_consensus.cc:2368] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:49.822857   971 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:49.900731 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:49.930323 26195 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:49.927888 26200 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:49.945042 26200 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:26:49.946619 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.947381 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.947795 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.948791 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.948975 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:49.929911  1021 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.900338   973 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:26:49.957409 26051 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:26:49.933350   818 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.930061   840 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:49.930909  1031 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:49.931448 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:49.895572   933 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.962483 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.950213   979 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:49.977511 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:49.921731  1019 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:49.954717   743 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:49.964174 26200 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.937827   970 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.934909   905 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:26:49.962736   996 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:49.964900   517 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:49.986130 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2389) took 1009 ms. Trace:
I20260709 12:26:49.957883 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:49.991940 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:49.992661  1028 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:49.992928 26791 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1209) took 3088 ms (client timeout 2042 ms). Trace:
I20260709 12:26:49.993407 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:49.993340 26791 rpcz_store.cc:269] 0709 12:26:46.904929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.905044 (+   115us) service_pool.cc:224] Handling call
0709 12:26:48.174648 (+1269604us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:26:49.992677 (+1818029us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:49.992825 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819946496}
W20260709 12:26:49.968796 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.995476  1064 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.109s	user 0.000s	sys 0.002s
I20260709 12:26:49.969143 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.897977   940 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.944085   776 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:49.927706   305 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:49.975657 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:49.960016   902 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:49.999550 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.007647 26200 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:50.008684 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.009277 26200 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:49.974828  1026 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.010162 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.014134 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:50.023747 26200 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:50.025228 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:50.032750 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.043969 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.993198   881 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:49.980472   979 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:50.033491   941 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.981841   743 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:49.964421  1021 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.032768 26046 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:50.054981 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:49.973937   818 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.057608 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.016917   984 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.022751  1026 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.032881 26200 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:26:49.981171   714 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:26:50.048672  1056 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:50.050778  1068 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.064960 26202 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:50.051632   938 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:50.056128   979 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.057827   881 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:50.067013 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:50.056762 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63418.4us
I20260709 12:26:50.056612  1028 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:50.069087 26051 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:50.053422   743 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.081928 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.063380  1026 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.060997   984 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:50.061367   902 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:50.090457 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1165) took 1526 ms. Trace:
I20260709 12:26:50.087289 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.090556 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.090939 26197 rpcz_store.cc:276] 0709 12:26:48.564197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.564767 (+   570us) service_pool.cc:224] Handling call
0709 12:26:50.090301 (+1525534us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:50.090404 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462331136}
I20260709 12:26:50.064960 32546 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:50.086961  1056 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:50.100275   938 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:50.100984  1056 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.102259 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.103024 26048 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:50.079723  1068 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.099470 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.105777 26052 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 12:26:50.119621 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.119976 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.120611 26790 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:50.120942 26791 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:50.066752  1013 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:26:50.125242 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:50.127075 26194 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:50.058305   941 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.089402  1033 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.058255   840 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.097285   984 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.145509 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.147737 26791 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:50.145474 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.148483 26787 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:50.095546  1028 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:50.107169  1060 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:26:50.150848 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:50.144738 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 160366us
I20260709 12:26:50.151479 26784 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.108976  1040 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.154179 26790 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:26:50.097985  1082 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:50.154847 26790 pending_rounds.cc:85] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 12
I20260709 12:26:50.157394 26790 pending_rounds.cc:107] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:26:50.157701 26790 raft_consensus.cc:2889] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.098829   990 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 12:26:50.155347 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55962.5us
I20260709 12:26:50.164611 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:50.130725   678 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.171480 26122 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.168905 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:50.174598 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:50.132928   881 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:50.129607   924 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:26:50.150102   919 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.121987   922 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
W20260709 12:26:50.178072 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.173731 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:50.163605  1073 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:26:50.146405   973 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:50.150627   987 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:26:50.189370  1071 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:26:50.159402   971 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.164963   941 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.158596  1086 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.205873 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:50.209307 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:49.986747 26203 rpcz_store.cc:276] 0709 12:26:48.976415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.040698 (+ 64283us) service_pool.cc:224] Handling call
0709 12:26:49.051371 (+ 10673us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:49.445543 (+394172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.445597 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:49.454650 (+  9053us) log.cc:844] Serialized 52 byte log entry
0709 12:26:49.821205 (+366555us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:49.980964 (+159759us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.984440 (+  3476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.984554 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.985837 (+  1283us) raft_consensus.cc:1685] finished
0709 12:26:49.985861 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.986053 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91403,"prepare.queue_time_us":128547,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":83094,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":16372,"thread_start_us":699257,"threads_started":3,"wal-append.queue_time_us":326130}
I20260709 12:26:50.219022   709 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:50.231177  1076 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:26:50.236755 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2390) took 1248 ms. Trace:
I20260709 12:26:50.237313 26198 rpcz_store.cc:276] 0709 12:26:48.988742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.043914 (+ 55172us) service_pool.cc:224] Handling call
0709 12:26:49.044220 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.567353 (+523133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.236468 (+669115us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5400302354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:50.236690 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201647488}
I20260709 12:26:50.255892   891 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.260993 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2385) took 1285 ms. Trace:
I20260709 12:26:50.261530 26199 rpcz_store.cc:276] 0709 12:26:48.975265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.033894 (+ 58629us) service_pool.cc:224] Handling call
0709 12:26:49.034220 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.571260 (+537040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.260650 (+689390us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:50.260911 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233636096}
I20260709 12:26:50.263144 26126 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.264129   840 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:50.271998  1070 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:26:50.276908 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:50.281479 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.283213 26785 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:50.283834 26784 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:50.285310 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:50.287451 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.288165 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:50.292379  1031 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.293028 26790 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:26:50.293586  1031 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.300983 26790 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 12:26:50.303861 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:50.304589 26784 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.317999 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.318885 26791 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:50.324328 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.325201 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:50.338424 26126 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:26:50.338933 26126 pending_rounds.cc:85] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 11
I20260709 12:26:50.339186 26126 pending_rounds.cc:107] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:26:50.339434 26126 raft_consensus.cc:2889] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.349105 26124 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:50.352325 26120 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:50.185139  1052 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:50.359270 26049 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:26:50.364403 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1287) took 1369 ms. Trace:
W20260709 12:26:50.366849 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.169000   938 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 12:26:50.372663 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74654.2us
I20260709 12:26:50.373703 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:50.368966 26784 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:50.364782 26121 rpcz_store.cc:276] 0709 12:26:48.995508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.142168 (+146660us) service_pool.cc:224] Handling call
0709 12:26:50.364159 (+1221991us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54003d1654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:50.364332 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950978432}
W20260709 12:26:50.372073 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 87025.9us
W20260709 12:26:50.378696 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.394889 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:50.399655 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 385580us
I20260709 12:26:50.361418  1085 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.404624 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.405544 26122 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:50.360888  1052 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:50.191685   678 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.192s
I20260709 12:26:50.373368   941 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.458355 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3761) took 2184 ms. Trace:
I20260709 12:26:50.371682   973 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.372303   305 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.416486  1093 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.501658 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:50.388630   519 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:50.181789   902 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.378132   933 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:50.392755   981 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.510936 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.513252   900 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:50.369045  1086 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:50.514086  1090 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.304s	user 0.000s	sys 0.000s
W20260709 12:26:50.514585 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.514791   963 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:50.514623 32638 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:50.586910 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62774.7us
I20260709 12:26:50.514837  1013 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:26:50.517416 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:50.574317  1075 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.416s	user 0.000s	sys 0.000s
I20260709 12:26:50.371349   971 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:50.522655 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.386260  1028 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.614430 26201 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:50.617066 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:50.520923 26126 rpcz_store.cc:276] 0709 12:26:48.196710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.196845 (+   135us) service_pool.cc:224] Handling call
0709 12:26:48.200609 (+  3764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.818355 (+617746us) spinlock_profiling.cc:243] Waited 81 ms on lock 0x7b54000e8920. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7432306f 00007f5b7439bf60 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62
0709 12:26:49.330090 (+511735us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54000e8920. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b74322520 00007f5b7439bf70 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62
0709 12:26:50.233120 (+903030us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b58000749d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:26:50.339906 (+106786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.381262 (+ 41356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441320832}
I20260709 12:26:50.524467 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:50.625089   921 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.967s	user 0.011s	sys 0.001s
I20260709 12:26:50.625294 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.625567   921 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.511710 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:50.629733   970 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.646s	user 0.003s	sys 0.007s
W20260709 12:26:50.502092 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128484us
I20260709 12:26:50.407346  1021 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.413786  1068 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:50.550457 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 145921us
I20260709 12:26:50.552361 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:50.644783 26754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.018s	sys 0.014s
I20260709 12:26:50.553707 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.645815 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:50.646396 26754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.018s	sys 0.014s
I20260709 12:26:50.646777 26203 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:50.513717   905 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.557343 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:50.523721   714 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.663431 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.556717  1093 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:50.623272   979 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.002s
I20260709 12:26:50.524919 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.545593   818 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.518432 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:50.626080 26045 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:50.633379 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:50.600782   940 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:50.641764 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.014s	sys 0.017s
W20260709 12:26:50.686390 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.014s	sys 0.018s
W20260709 12:26:50.676888 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43620.1us
W20260709 12:26:50.687009 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 561843us
I20260709 12:26:50.676263 26202 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:50.645339 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:50.582752   900 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:50.572450  1094 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.573617   305 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:50.662897 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.538576   973 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.664067 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:50.686609   517 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.698s	user 0.003s	sys 0.007s
I20260709 12:26:50.577759   981 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:50.500775  1085 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.537659   982 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:26:50.687785 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:50.566324  1095 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.700778   940 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.688812 26126 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:50.684901 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.698966 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:50.746678 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47835us
W20260709 12:26:50.700529 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.644882   516 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.538707   941 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.659461   954 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:50.662652   963 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.660125   933 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.677232   743 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.581s	user 0.008s	sys 0.001s
I20260709 12:26:50.726891   305 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.674190 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:50.631292  1054 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.000s
I20260709 12:26:50.683444  1087 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.691329  1068 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:50.690867   971 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:50.613575 32638 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.698969   900 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.687446 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:50.739858 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52169.2us
I20260709 12:26:50.816570 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:50.817479  1026 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.000s
I20260709 12:26:50.817582   305 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:50.808223   970 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.818550   970 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.820683   970 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.821244   970 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:50.821722   970 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.818104  1026 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.823825   970 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.824625 26197 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:26:50.815439  1093 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:50.814844  1086 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.814365   941 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:50.828670  1087 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.807015   516 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.831743   963 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.807850  1039 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.843195 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:50.834560  1039 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.827730  1026 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.798818  1033 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.843016 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.823514   806 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.922s	user 0.009s	sys 0.000s
I20260709 12:26:50.782776 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:50.733942   981 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.675288   905 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.756923   940 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.860381 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:26:50.754161  1096 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:50.773267  1088 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:26:50.690912   979 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.800889 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.801270 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:50.742425   921 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:50.719219 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55235us
W20260709 12:26:50.797633  1103 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.142s	user 0.000s	sys 0.002s
I20260709 12:26:50.773864   680 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:26:50.797499   818 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:26:50.682682   714 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.838763   922 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:50.844194 26049 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:26:50.854436 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:50.859740 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:50.859854 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:50.802748  1094 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:50.797816   840 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:50.731626   992 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:50.860841  1115 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:50.864924 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.804131   954 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:50.895287  1098 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.912672   954 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:50.912909  1098 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.913408  1098 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.914335 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.915266 26050 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:50.876698  1096 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.916419   818 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.916602 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3691) took 1130 ms. Trace:
W20260709 12:26:50.917459   984 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.000s
I20260709 12:26:50.917073   818 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.918054 26048 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:50.917966   984 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.877763 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.920490 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:50.921499 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.881247 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80456.4us
I20260709 12:26:50.921715   714 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.881673 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:50.922477 26019 leader_election.cc:400] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:50.922807 26045 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.923804 26053 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.924744   905 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.925376   905 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:50.925913   905 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.922839 26019 leader_election.cc:403] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.926626 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.928298 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.814922   933 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:50.929800   996 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:50.815362   743 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.931318   996 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:50.837733   973 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.932132 26052 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:50.861857  1068 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.932984  1068 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.933561  1016 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.869654 26123 raft_consensus.cc:606] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:50.935189 26123 raft_consensus.cc:614] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:50.845925  1033 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.936448   984 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:50.866881  1095 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.902171  1040 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:50.902797   979 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.947973 26125 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:50.904428  1053 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:50.862013  1054 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:50.906704   970 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:50.889679   922 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:50.889240   971 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.909273   519 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.880127  1026 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.911180   940 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.917006 26046 rpcz_store.cc:276] 0709 12:26:49.786759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.788991 (+  2232us) service_pool.cc:224] Handling call
0709 12:26:50.916416 (+1127425us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:50.916540 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248614784}
I20260709 12:26:50.892654   840 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.883955 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:50.971410 26050 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:50.888314 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:26:50.972438 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84273.9us
I20260709 12:26:50.974929 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:50.943343 26053 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:50.921780 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:50.939378 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:50.984386 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2365) took 1017 ms. Trace:
W20260709 12:26:50.984498 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45198.4us
I20260709 12:26:50.984987 26050 rpcz_store.cc:276] 0709 12:26:49.967008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.969246 (+1002238us) service_pool.cc:224] Handling call
0709 12:26:50.971124 (+  1878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.984114 (+ 12990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.984327 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:50.985545 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.986828 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.899494   938 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:26:50.961258   519 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:50.889454   678 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.947103  1115 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:50.957993 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:50.958184  1100 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.955648 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.938371  1085 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.948123  1039 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.948571  1040 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:50.988260   919 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:50.977239 26126 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 12:26:50.980314 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 96431us
I20260709 12:26:50.958658   963 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.876164   806 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:50.947134 26049 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:51.012053 26054 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:51.012115 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:50.942502 26045 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:50.968391   920 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.961071  1082 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.976176   840 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:50.984061   940 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:50.976677 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:50.976115   984 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:50.994351 26050 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:26:51.027559 26054 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:50.989230 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:51.001039 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45466.7us
I20260709 12:26:50.992404 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:50.999890 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:50.987772   979 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:50.982523  1026 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.033043 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.990902   973 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:51.053927 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.997135  1094 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.997442   933 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:50.994185   996 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:51.055961 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:51.056667 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:51.022588  1029 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:51.018245   881 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:50.947839  1095 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.068331 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:51.069630 26200 raft_consensus.cc:2368] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:26:51.070819 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:51.081919 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:51.027252   904 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.024047   920 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.056483   984 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.077026 26791 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:51.027503   940 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.022117   806 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:51.078092 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.078881 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49706.9us
I20260709 12:26:51.105032 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:50.994299  1056 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:26:51.058394   973 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.052402   986 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.110128  1110 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:51.110495  1112 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:26:51.052747   963 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.053988   919 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:51.003417   678 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.115787   919 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.116210  1091 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:51.117751  1091 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:51.118197  1091 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.118417   919 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.118662  1091 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.120404   920 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:51.095312   979 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:51.121126   979 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:51.121510   979 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.103791   940 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.124429   709 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:51.009161   922 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.095726  1115 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:51.128881  1106 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:26:51.027050  1082 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.053143 32546 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:51.063194   941 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.101969  1086 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.105033 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:51.004269  1100 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.118295   900 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.055351  1085 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:51.076880 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:51.098215  1029 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:51.110172   973 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:51.116294  1124 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:26:51.149891   516 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.113942  1094 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:51.182790 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:51.162339 26167 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.576s (SENT)
I20260709 12:26:51.185992 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:51.164606 26785 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:51.165690  1026 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:51.166090   806 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.192708   806 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:51.167269 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.195050 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.196141 26788 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:26:51.168839 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:51.197352 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.198004 26049 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:51.198181 26783 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:51.130295   971 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.099730  1040 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.187849  1068 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.100694   904 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.138752  1112 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:51.204931 26790 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:51.206404 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.202160 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.207736 26787 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:51.209944 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.211175 26121 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:51.207067 26783 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:51.213223 26019 leader_election.cc:400] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:26:51.213610 26019 leader_election.cc:403] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.229936 26785 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:51.231124 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.210875 26791 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:26:51.273537 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64149.1us
I20260709 12:26:51.169971  1110 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:51.179288  1085 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:51.178151  1019 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:51.279893 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.185606  1094 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.284181 26122 raft_consensus.cc:2368] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
I20260709 12:26:51.207279  1026 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:51.136528   963 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.210044  1018 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.303920 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:51.170885  1086 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.324735 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.316532 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.141252 32546 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.317915 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:51.208215  1112 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:51.281859  1110 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:51.213281   971 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.307550 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.273314  1054 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.287957  1085 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:51.382469 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:51.382957 26783 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:51.385412 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:51.347450 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.278910  1131 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:26:51.392959 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:51.460602 26202 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:51.465051 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:51.466070 26196 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:51.396221 26791 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:51.429662 26123 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:51.459040 26199 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:51.459502 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.459051 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:51.471922 26045 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:51.474337  1129 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 12:26:51.442076  1029 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.306510  1094 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:51.322688  1040 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.321265  1095 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.308370  1019 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:51.485867  1098 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.572s	user 0.005s	sys 0.006s
W20260709 12:26:51.495659   981 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 12:26:51.396613  1110 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:51.525362 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.525170 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:51.527979 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:51.516541   954 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.003s
I20260709 12:26:51.531548 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.534873 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:51.532366 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:51.533537 26783 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:51.535487 26790 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:51.535877 26202 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:51.489158   979 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.537417 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.391770  1018 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.541626 26203 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:51.539633 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.544729 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:51.545748 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:51.548420 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:51.552618 26202 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:51.552888 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:51.557323 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:51.559760 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:51.560716 26782 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:51.561885 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2359) took 1606 ms. Trace:
I20260709 12:26:51.562404 26052 rpcz_store.cc:276] 0709 12:26:49.955482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.107574 (+152092us) service_pool.cc:224] Handling call
0709 12:26:50.107845 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.932679 (+824834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.561537 (+628858us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:51.561806 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895246720}
I20260709 12:26:51.563966 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.574990 26786 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:26:51.575459 26786 pending_rounds.cc:85] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 12
I20260709 12:26:51.575791 26786 pending_rounds.cc:107] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:26:51.576107 26786 raft_consensus.cc:2889] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.513175  1098 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.572348 26052 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:51.583719 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.586973   979 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.587772   979 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:51.590093  1016 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.001s
I20260709 12:26:51.570304   516 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:51.591078   516 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.591468   516 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:51.588176   979 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.593730   979 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.591552   940 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.596465   940 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.594444 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2356) took 1639 ms. Trace:
I20260709 12:26:51.596581 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:51.597461  1019 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:51.537004  1082 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.599817 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:51.597000 26048 rpcz_store.cc:276] 0709 12:26:49.954813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.103536 (+148723us) service_pool.cc:224] Handling call
0709 12:26:50.917861 (+814325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.918551 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.594225 (+675674us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b540008e8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:51.594383 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872029056}
I20260709 12:26:51.604915 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.606498 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.608283 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:51.610059 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:51.520925   981 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:51.537458   900 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:51.573948  1125 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
I20260709 12:26:51.611331 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.613454 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.584590 26052 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:51.624055 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.624720 26783 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:51.909010   981 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.631264 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.634608 26782 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:51.604486  1068 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.548102   954 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:51.649746  1111 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.850s	user 0.004s	sys 0.000s
I20260709 12:26:51.652432 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:51.655134 32638 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.655297  1086 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.658756   963 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.660392 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3777) took 1498 ms. Trace:
I20260709 12:26:52.028795 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2827) took 1875 ms. Trace:
I20260709 12:26:52.029217 26129 rpcz_store.cc:276] 0709 12:26:50.152899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.153039 (+   140us) service_pool.cc:224] Handling call
0709 12:26:50.153350 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.153687 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.153719 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.159157 (+  5438us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.512656 (+353499us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:50.512705 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.517339 (+  4634us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.648973 (+1131634us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:52.028396 (+379423us) raft_consensus.cc:1685] finished
0709 12:26:52.028418 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.028729 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2884,"raft.queue_time_us":2228,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":10120,"spinlock_wait_cycles":3913472,"thread_start_us":305437,"threads_started":1,"wal-append.queue_time_us":513018}
I20260709 12:26:51.661162 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3778) took 1499 ms. Trace:
I20260709 12:26:52.031278 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.032181 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3797) took 1043 ms. Trace:
I20260709 12:26:52.031466 26128 rpcz_store.cc:276] 0709 12:26:50.161752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.851901 (+690149us) service_pool.cc:224] Handling call
0709 12:26:50.852449 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.853009 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.853056 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.853781 (+   725us) log.cc:844] Serialized 26 byte log entry
0709 12:26:51.291904 (+438123us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:51.291955 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.292018 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.660953 (+368935us) raft_consensus.cc:1685] finished
0709 12:26:51.660977 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.661097 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3503,"raft.queue_time_us":4456,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6282,"thread_start_us":437605,"threads_started":1,"wal-append.queue_time_us":704025}
I20260709 12:26:52.033244 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:52.032531 26129 rpcz_store.cc:276] 0709 12:26:50.989131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.030988 (+1041857us) service_pool.cc:224] Handling call
0709 12:26:52.032120 (+  1132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.034437 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:51.662501 32546 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.662352 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3703) took 1134 ms. Trace:
I20260709 12:26:51.566753  1112 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:52.036370 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1169) took 2039 ms (client timeout 1832 ms). Trace:
I20260709 12:26:52.036885  1112 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:52.036815 26195 rpcz_store.cc:269] 0709 12:26:49.997317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.023153 (+ 25836us) service_pool.cc:224] Handling call
0709 12:26:52.036165 (+2013012us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54002175d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:52.036298 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3535956736}
I20260709 12:26:51.622413  1100 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:51.628612   904 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:51.752115 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.700911  1033 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.764701   924 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.624858 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:51.850822   905 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.038929   924 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:51.863782 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:52.027871 26125 rpcz_store.cc:276] 0709 12:26:50.161497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.839893 (+678396us) service_pool.cc:224] Handling call
0709 12:26:50.840360 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.948367 (+108007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.660074 (+711707us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b54001c4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:51.660333 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224089984}
I20260709 12:26:52.041226 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3793) took 1069 ms. Trace:
I20260709 12:26:52.041707 26124 rpcz_store.cc:276] 0709 12:26:50.971856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.863438 (+891582us) service_pool.cc:224] Handling call
0709 12:26:52.041163 (+177725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.042596   924 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.035233 26054 rpcz_store.cc:276] 0709 12:26:50.527592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.053799 (+526207us) service_pool.cc:224] Handling call
0709 12:26:51.068158 (+ 14359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:51.068518 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:51.068536 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:51.068861 (+   325us) log.cc:844] Serialized 26 byte log entry
0709 12:26:51.334002 (+265141us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:51.334054 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.334155 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.662153 (+327998us) raft_consensus.cc:1685] finished
0709 12:26:51.662179 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.662297 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49057,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":55,"thread_start_us":225626,"threads_started":1,"wal-append.queue_time_us":441185}
I20260709 12:26:52.042898 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.044354 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:52.044775 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.044960 26054 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 16.
I20260709 12:26:51.864284  1052 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.045971 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.046396  1052 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:52.046710 26124 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 15.
I20260709 12:26:52.047185   954 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:51.917685 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:51.900046 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2366) took 1714 ms. Trace:
I20260709 12:26:52.064796 26046 rpcz_store.cc:276] 0709 12:26:50.185992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.981831 (+795839us) service_pool.cc:224] Handling call
0709 12:26:50.982144 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.104668 (+122524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:51.104697 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:51.104739 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:51.104769 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.899694 (+794925us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:51.899775 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.899963 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281362304}
I20260709 12:26:51.899499  1098 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:51.899751  1149 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.630808 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.066599 26123 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:52.066489  1149 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.067178 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3798) took 1070 ms. Trace:
I20260709 12:26:52.067526 26123 rpcz_store.cc:276] 0709 12:26:50.996680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.629982 (+633302us) service_pool.cc:224] Handling call
0709 12:26:52.067119 (+437137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.068768 26123 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:52.069479 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2830) took 1116 ms. Trace:
I20260709 12:26:52.069849 26123 rpcz_store.cc:276] 0709 12:26:50.953507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.068259 (+1114752us) service_pool.cc:224] Handling call
0709 12:26:52.068591 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.069225 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.069425 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:51.913939   992 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:52.091032 26123 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:52.036486   941 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.890s	user 0.007s	sys 0.013s
W20260709 12:26:51.991937   519 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.989s	user 0.019s	sys 0.015s
I20260709 12:26:52.092624 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2831) took 1138 ms. Trace:
I20260709 12:26:52.093750 26123 rpcz_store.cc:276] 0709 12:26:50.954040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.070582 (+1116542us) service_pool.cc:224] Handling call
0709 12:26:52.090789 (+ 20207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.091942 (+  1153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.092486 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:51.967362 26052 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:51.923521  1019 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.975580 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1291) took 1589 ms. Trace:
I20260709 12:26:51.980077 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:51.996371 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3702) took 1487 ms (client timeout 1716 ms). Trace:
I20260709 12:26:51.905524  1087 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:51.991760 26121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3782) took 1829 ms (client timeout 2179 ms). Trace:
I20260709 12:26:52.034504 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.047582   678 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.047747 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.051118  1096 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:52.185056 26789 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.019s	sys 0.026s
I20260709 12:26:52.052779   517 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:52.054836 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:51.981240   921 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.042960 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:52.058080 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 458298us
I20260709 12:26:52.058602   881 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.058892   743 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.042002 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.060220  1095 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:52.061945  1133 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.727007  1018 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:52.071650 26049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.000s	sys 0.012s
W20260709 12:26:52.259907 26049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.000s	sys 0.012s
W20260709 12:26:52.103569 26755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.012s	sys 0.029s
W20260709 12:26:52.260517 26755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.014s	sys 0.035s
W20260709 12:26:52.261332 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.151767 26167 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.018s	sys 0.029s
W20260709 12:26:52.263610 26167 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.020s	sys 0.029s
W20260709 12:26:52.172806 26121 rpcz_store.cc:269] 0709 12:26:50.162747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.652428 (+489681us) service_pool.cc:224] Handling call
0709 12:26:51.991536 (+1339108us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:51.991693 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851616000}
I20260709 12:26:52.174252 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2834) took 1043 ms. Trace:
I20260709 12:26:52.176008 26122 rpcz_store.cc:276] 0709 12:26:50.386085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.673831 (+287746us) service_pool.cc:224] Handling call
0709 12:26:51.975353 (+1301522us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400369cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:51.975508 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972519168}
I20260709 12:26:51.990723   922 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:52.182441 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.264772 26129 rpcz_store.cc:276] 0709 12:26:51.131218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.034295 (+903077us) service_pool.cc:224] Handling call
0709 12:26:52.174161 (+139866us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408}
I20260709 12:26:52.266156 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.266656 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:52.266970 26129 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:52.129442 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:52.175035 26053 rpcz_store.cc:269] 0709 12:26:50.508845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.854166 (+345321us) service_pool.cc:224] Handling call
0709 12:26:51.996112 (+1141946us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b540035fcd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:51.996290 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87909760}
I20260709 12:26:52.268453 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:51.898824 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.269613 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.185668 26789 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.019s	sys 0.026s
W20260709 12:26:52.189419 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:52.270610 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.200290 26127 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
I20260709 12:26:52.203651   905 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.210225   678 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:52.210629 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.185344  1098 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.211930 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3701) took 1710 ms. Trace:
W20260709 12:26:52.136823  1146 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:26:52.135195  1091 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.016s	user 0.019s	sys 0.001s
I20260709 12:26:52.296029  1018 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:52.298192 26050 rpcz_store.cc:276] 0709 12:26:50.501632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.028413 (+526781us) service_pool.cc:224] Handling call
0709 12:26:51.045194 (+ 16781us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:51.100058 (+ 54864us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:51.137702 (+ 37644us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.528368 (+390666us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.900746 (+372378us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18003a86a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:26:52.041766 (+141020us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18003a86a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219
0709 12:26:52.043266 (+  1500us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18003a86a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:26:52.043444 (+   178us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.043909 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:26:52.044078 (+   169us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:52.044109 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.044176 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:52.211712 (+167536us) raft_consensus.cc:1685] finished
0709 12:26:52.211733 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.211868 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516143,"prepare.queue_time_us":44965,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":99,"raft.queue_time_us":609900,"raft.run_cpu_time_us":573,"raft.run_wall_time_us":13595,"spinlock_wait_cycles":18734080,"thread_start_us":379872,"threads_started":2,"wal-append.queue_time_us":1036284}
I20260709 12:26:52.302156  1091 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.302762 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.222410 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3786) took 2046 ms. Trace:
I20260709 12:26:52.245328 26128 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:52.304396 26126 rpcz_store.cc:276] 0709 12:26:50.176177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.874037 (+697860us) service_pool.cc:224] Handling call
0709 12:26:50.957537 (+ 83500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:51.010366 (+ 52829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.222084 (+1211718us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b54001f86d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:52.222330 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44505344}
I20260709 12:26:52.305229 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.305581 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.306345 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.306985 26126 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:52.307562 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3788) took 1338 ms. Trace:
W20260709 12:26:52.211517 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 231577us
I20260709 12:26:52.288920  1133 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.308733 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:52.244953 26791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.015s	sys 0.034s
W20260709 12:26:52.309391 26791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.016s	sys 0.034s
W20260709 12:26:52.203030 26199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.008s	sys 0.043s
W20260709 12:26:52.310056 26199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.008s	sys 0.043s
I20260709 12:26:52.068802  1149 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.311434  1049 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:52.298950   905 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.312009  1049 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:52.186065  1019 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.312996  1049 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.313743   905 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.314459  1019 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:52.137249   979 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.543s	user 0.017s	sys 0.018s
I20260709 12:26:52.308403  1091 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.316493   979 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.307857 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3789) took 1338 ms. Trace:
I20260709 12:26:52.317800 26128 rpcz_store.cc:276] 0709 12:26:50.970657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.307152 (+1336495us) service_pool.cc:224] Handling call
0709 12:26:52.307409 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.307579 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.307610 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:52.307651 (+    41us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:52.307680 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.307729 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.307812 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.318634   979 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:52.186282   973 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.986s	user 0.015s	sys 0.001s
I20260709 12:26:52.319380   973 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.177704   971 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.089663   992 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:52.322525   971 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.323038   992 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.324488   992 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:52.031203  1040 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.006s
I20260709 12:26:52.326498 26047 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:52.263974 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 956565us
I20260709 12:26:51.982717   984 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.268961 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3708) took 1446 ms. Trace:
I20260709 12:26:52.344651 26049 rpcz_store.cc:276] 0709 12:26:50.822908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.198798 (+375890us) service_pool.cc:224] Handling call
0709 12:26:51.199073 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.199262 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:51.489419 (+290157us) log.cc:844] Serialized 14 byte log entry
0709 12:26:52.268617 (+779198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:52.268679 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:52.268726 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.268793 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.268903 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8180,"prepare.queue_time_us":127548,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":11557,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14979,"thread_start_us":1060305,"threads_started":2}
I20260709 12:26:52.272732 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:52.346678 26782 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:52.277766 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:52.278138 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:52.278505 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:52.348855 26045 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:52.278697 26052 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:52.279589 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:52.215510   996 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.149s	user 0.011s	sys 0.008s
I20260709 12:26:52.271525  1112 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:52.357857 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.224011   818 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.281663 26784 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:52.358569 26129 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:52.285959 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:52.288652 26790 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.751s	user 0.009s	sys 0.001s
I20260709 12:26:52.359490 26790 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:52.290195 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.360276 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:51.995294  1094 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.137809   941 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:52.284584 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:52.381686 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97448.8us
I20260709 12:26:52.382225 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.178403   519 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.301463 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.309612 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:52.310750  1133 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.254328  1095 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:52.394551  1095 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.396766 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.396963  1094 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.284399  1029 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 12:26:52.398095 26786 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:26:52.398699 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.399312 26129 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
W20260709 12:26:52.399664   516 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.808s	user 0.021s	sys 0.019s
I20260709 12:26:52.399812 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:52.262687   921 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.321942   973 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.402092 26123 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:26:52.322010 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.199867   517 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:52.307848 26126 rpcz_store.cc:276] 0709 12:26:50.969235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.306449 (+1337214us) service_pool.cc:224] Handling call
0709 12:26:52.306712 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.307359 (+   647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.307516 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.338948 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:52.368669 26199 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:52.369576 26198 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:26:52.264276   881 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.374255 26196 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:52.428856 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3791) took 1457 ms. Trace:
I20260709 12:26:52.375309 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.435899 26123 rpcz_store.cc:276] 0709 12:26:50.971307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.401504 (+1430197us) service_pool.cc:224] Handling call
0709 12:26:52.401798 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.428587 (+ 26789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.428801 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.376576 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:52.437902 26045 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:52.377583 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.440534 26054 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:52.378007   996 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:52.345690   979 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:52.445232 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:52.445896 26046 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
W20260709 12:26:52.444495 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.040s	sys 0.042s
I20260709 12:26:52.297036 26757 leader_election.cc:400] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
W20260709 12:26:52.446780 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.040s	sys 0.042s
I20260709 12:26:52.447000 26757 leader_election.cc:403] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:52.447265 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 894335us
W20260709 12:26:52.447892 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 153679us
W20260709 12:26:52.448891 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.449419 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:52.376049 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:52.388093 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.285185   981 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:52.390720 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81175.4us
W20260709 12:26:52.450278 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 74332.9us
I20260709 12:26:52.336951 26120 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:26:52.426081 26786 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:52.400506 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:52.427490 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.400177 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:52.327841  1098 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.378432  1112 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.453965 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:52.435137  1110 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.899s	user 0.009s	sys 0.005s
I20260709 12:26:52.403960   519 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.448913 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:52.477833 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.388832  1018 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:52.481263 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:52.481882 26199 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.434625 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:52.482633 26782 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:52.485716 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:52.452076 26049 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:52.402935  1085 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.917s	user 0.005s	sys 0.012s
W20260709 12:26:52.421540   840 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.368s	user 0.037s	sys 0.021s
I20260709 12:26:52.421810   971 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.439339 26045 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:52.516968 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:52.473529 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.513234 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.400103   516 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.517406 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:52.441125   905 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:52.526386 26124 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:52.475082 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:52.386233  1040 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:52.476809  1094 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:52.456432 26052 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:52.465030 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.400499   984 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:52.502875 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68358.2us
I20260709 12:26:52.508042 26054 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:52.509145 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.514714 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.549238 26201 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:52.515812 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.519096 26202 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:52.520697 26784 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:52.456687 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3796) took 1471 ms. Trace:
I20260709 12:26:52.529781 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:52.550210 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:52.376205   970 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:52.538815 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 118691us
I20260709 12:26:52.540290 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:52.554287 26045 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.412217  1095 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.556999 26048 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:52.558174  1095 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.543409 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.553167 26120 rpcz_store.cc:276] 0709 12:26:50.985252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.917298 (+932046us) service_pool.cc:224] Handling call
0709 12:26:52.456489 (+539191us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54001860d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:52.456634 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614853376}
I20260709 12:26:52.561441 26120 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:26:52.537184 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:52.577561 26195 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.550765 26050 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.480604  1086 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:52.444607  1100 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:26:52.499019  1087 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:52.602389  1087 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:52.602896  1087 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:52.570535 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70046.3us
I20260709 12:26:52.582706 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:52.611043 26201 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.572611 26046 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:52.455945  1133 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:52.547432 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:52.574470 26200 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:52.498668   941 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
I20260709 12:26:52.617156 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:52.617854 26202 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:52.480931  1018 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:52.595500  1098 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.595572  1152 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.596194   987 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:52.479737   678 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:26:52.455665   982 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:26:52.480120   996 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 12:26:52.501334   973 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 12:26:52.596910 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:52.598760   516 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.581851  1086 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.622208 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:52.457712   981 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.581755   984 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:52.770386 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.818034  1098 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:52.571909  1040 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.570979  1094 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.545481   924 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.984609   924 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.620985  1095 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:52.986002  1040 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.985916  1095 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.525478   840 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.989964   840 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.990592   840 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:52.991139   840 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:52.481594  1110 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.992872 26783 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:52.994029  1110 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.994741  1110 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.995285  1110 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.996558 26051 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:52.457368   979 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.997979   981 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.998435   979 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:52.618961  1133 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.999084  1133 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.999943   984 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:52.743891 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.000619 26203 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.518527  1085 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.819321 26196 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:26:52.616511   519 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:52.818724 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.823665  1041 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.529800 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:52.924187 26202 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
W20260709 12:26:52.926218  1151 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:52.928745 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1578) took 1401 ms. Trace:
W20260709 12:26:52.929279  1159 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:26:52.930199   881 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:52.610299   970 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:52.866832 26199 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:53.039906 26198 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:53.035424 26791 rpcz_store.cc:276] 0709 12:26:51.527811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.528372 (+   561us) service_pool.cc:224] Handling call
0709 12:26:51.528763 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:51.529171 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:51.529200 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:51.544320 (+ 15120us) log.cc:844] Serialized 26 byte log entry
0709 12:26:52.309773 (+765453us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:52.309810 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.309873 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:52.928395 (+618522us) raft_consensus.cc:1685] finished
0709 12:26:52.928424 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.928662 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":5085,"thread_start_us":716957,"threads_started":1,"wal-append.queue_time_us":1038319}
I20260709 12:26:53.044415 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.001325  1094 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:53.046461 26788 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:53.046906 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:53.048004 26791 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:52.858453  1018 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:52.942606 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.938565 26195 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:52.960184 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:52.923024   971 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:52.962850 26018 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.009s	sys 0.026s
W20260709 12:26:53.059633 26789 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.010s	sys 0.006s
I20260709 12:26:52.968791 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
W20260709 12:26:53.060165 26789 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.011s	sys 0.006s
W20260709 12:26:52.974411 26053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.008s	sys 0.008s
I20260709 12:26:52.975435 26786 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
W20260709 12:26:52.964545   938 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.010s	sys 0.025s
I20260709 12:26:52.978750 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1297) took 1014 ms. Trace:
I20260709 12:26:52.980109 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:52.979471 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.970373  1019 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:52.972687  1049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.030s	sys 0.116s
I20260709 12:26:52.979538  1086 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:52.983913 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:52.860550 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 306619us
W20260709 12:26:52.970005  1131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.004s	sys 0.011s
I20260709 12:26:53.028735 26201 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:52.907961 26045 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:53.029599 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3794) took 2057 ms. Trace:
I20260709 12:26:53.082474 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.082911 26121 rpcz_store.cc:276] 0709 12:26:50.972120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.410195 (+1438075us) service_pool.cc:224] Handling call
0709 12:26:52.410658 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.996936 (+586278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.996984 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:52.997043 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:52.997089 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.029313 (+ 32224us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54001f68d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:53.029374 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.029529 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347654528}
I20260709 12:26:52.975585  1112 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:53.035202 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3696) took 2626 ms (client timeout 2834 ms). Trace:
I20260709 12:26:52.981639  1098 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:53.076932  1049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.031s	sys 0.117s
I20260709 12:26:52.937892 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.049687 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:53.092564 26783 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:53.092609 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 154781us
W20260709 12:26:53.059719 26018 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.010s	sys 0.026s
I20260709 12:26:53.054049   921 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:53.093621 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 1.03538e+06us
I20260709 12:26:53.060864   933 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:53.073334  1019 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:53.070248 26124 rpcz_store.cc:276] 0709 12:26:51.964499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.047465 (+ 82966us) service_pool.cc:224] Handling call
0709 12:26:52.977687 (+930222us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:52.978686 (+   999us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792652032}
W20260709 12:26:53.069800 26053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.009s	sys 0.009s
I20260709 12:26:53.071839 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:53.056535 26125 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:53.102913 26786 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:53.108747 26786 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:53.085925 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.095441  1098 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:53.064057  1096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.004s	sys 0.038s
W20260709 12:26:53.088303 26051 rpcz_store.cc:269] 0709 12:26:50.402968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.926353 (+523385us) service_pool.cc:224] Handling call
0709 12:26:53.028681 (+2102328us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400228654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:53.028825 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4758404992}
I20260709 12:26:53.089323  1052 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:52.982160   516 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:53.090126 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.035413   519 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:53.094728 26018 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.042s (SENT)
I20260709 12:26:53.151417 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:53.052219 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:53.064318 26128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.010s	sys 0.034s
I20260709 12:26:53.073192   987 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:53.106949  1019 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:53.115545  1096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.004s	sys 0.039s
I20260709 12:26:53.115651  1049 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:53.041214  1041 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.070585 26784 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:52.879442 26049 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:53.102727 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.108672 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:53.169790 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61228.4us
I20260709 12:26:53.170284 26782 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
W20260709 12:26:53.069762   938 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.010s	sys 0.030s
I20260709 12:26:53.110924 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.170589 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.112404 26790 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:53.115945 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:53.121218 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1589) took 1571 ms. Trace:
W20260709 12:26:53.124728 26197 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.015s	sys 0.037s
I20260709 12:26:53.124215  1150 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:53.061120 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1577) took 2936 ms. Trace:
I20260709 12:26:53.075533   971 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:53.148214 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.067s	sys 0.084s
I20260709 12:26:53.099679   963 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:26:53.152860 26128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.011s	sys 0.034s
I20260709 12:26:53.125746  1052 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:53.182996 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 67175.4us
W20260709 12:26:53.155464 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 60791.9us
I20260709 12:26:53.156683 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1579) took 1627 ms. Trace:
I20260709 12:26:53.061640   924 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:53.162401  1095 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:53.081584  1131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.005s	sys 0.012s
I20260709 12:26:53.185443  1038 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:53.204864  1038 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.205371  1038 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.208524   924 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:53.199360   982 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.210695   982 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:53.211036   982 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:53.154441   954 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.015s	sys 0.040s
W20260709 12:26:53.212466   954 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.016s	sys 0.041s
W20260709 12:26:53.177598 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2837) took 1660 ms (client timeout 1868 ms). Trace:
W20260709 12:26:53.178027 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.067s	sys 0.084s
W20260709 12:26:53.213544 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 754605us
W20260709 12:26:53.213383 26125 rpcz_store.cc:269] 0709 12:26:51.516902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.041728 (+524826us) service_pool.cc:224] Handling call
0709 12:26:53.172048 (+1130320us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400287654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:53.177542 (+  5494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866569600}
I20260709 12:26:53.214241   517 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.214881 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.215310   517 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:53.216045 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:53.155018 26047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.025s	sys 0.030s
W20260709 12:26:53.217540 26047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.025s	sys 0.030s
I20260709 12:26:53.218672 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.219228 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.177299 26788 rpcz_store.cc:276] 0709 12:26:51.540912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.652103 (+111191us) service_pool.cc:224] Handling call
0709 12:26:53.112062 (+1459959us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54004f79d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:53.112223 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354713856}
I20260709 12:26:53.211854   982 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.115054   709 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:26:53.220911   709 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:53.221247   709 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:53.222101   709 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.219890 26199 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
W20260709 12:26:53.162442   990 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.026s	sys 0.040s
W20260709 12:26:53.223317   990 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.026s	sys 0.040s
I20260709 12:26:53.223628 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.186084 26785 rpcz_store.cc:276] 0709 12:26:51.529723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.530416 (+   693us) service_pool.cc:224] Handling call
0709 12:26:51.532412 (+  1996us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:51.537412 (+  5000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:51.537445 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:51.538000 (+   555us) log.cc:844] Serialized 52 byte log entry
0709 12:26:51.902831 (+364831us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b48000b0ad8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:26:52.093476 (+190645us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:52.093550 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.093658 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.146798 (+1053140us) raft_consensus.cc:1685] finished
0709 12:26:53.146830 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.156594 (+  9764us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":8765,"raft.run_wall_time_us":369171,"spinlock_wait_cycles":18116992,"thread_start_us":464185,"threads_started":1,"wal-append.queue_time_us":949452}
I20260709 12:26:53.225431   709 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.226488   970 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:53.181916  1040 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:53.201987 26051 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:53.177871 26789 rpcz_store.cc:276] 0709 12:26:50.124353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.124485 (+   132us) service_pool.cc:224] Handling call
0709 12:26:50.150521 (+ 26036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.191742 (+ 41221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.191793 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.205228 (+ 13435us) log.cc:844] Serialized 26 byte log entry
0709 12:26:51.665676 (+1460448us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800309280. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1
0709 12:26:51.665897 (+   221us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:52.277338 (+611441us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.060650 (+783312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.060757 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.060800 (+    43us) raft_consensus.cc:1685] finished
0709 12:26:53.060893 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.061017 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4560,"prepare.queue_time_us":365275,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":385484,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":16706,"spinlock_wait_cycles":3491584,"thread_start_us":1810734,"threads_started":4,"wal-append.queue_time_us":531292,"wal-append.run_cpu_time_us":3064,"wal-append.run_wall_time_us":1096160}
I20260709 12:26:53.203308 26050 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:53.176815  1049 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.184221   963 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.202759 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.219314 26049 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:53.219866   922 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:53.227829 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:53.177485 26197 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.016s	sys 0.038s
I20260709 12:26:53.192600 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.157198   516 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:53.162184 26126 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.000s	sys 0.017s
W20260709 12:26:53.249080 26126 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.000s	sys 0.018s
I20260709 12:26:53.231576 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.249933 26046 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 15.
I20260709 12:26:53.231068 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:53.191227  1087 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.003s
I20260709 12:26:53.232657 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.255204 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:53.182641   678 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:26:53.262265 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:53.262322 26122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.007s	sys 0.015s
W20260709 12:26:53.263469 26122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.007s	sys 0.016s
I20260709 12:26:53.190789  1068 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:26:53.242622  1131 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.230685 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:53.268381 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.014s	sys 0.036s
I20260709 12:26:53.248615   963 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:53.268994 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.014s	sys 0.036s
I20260709 12:26:53.269235 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:53.270007 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 719872us
I20260709 12:26:53.247035 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.239702  1040 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.244680 26124 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:53.251722 26047 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:53.192986   840 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.164819   987 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.187548  1052 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:53.258013 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.299361 26199 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.260296 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.300997 26121 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:53.226521   971 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.265097 26200 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:53.244345  1049 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:53.311517 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:53.164690  1013 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.013s	sys 0.046s
I20260709 12:26:53.244343   922 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:53.270991 26054 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:53.202838  1149 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
W20260709 12:26:53.281679 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.007s	sys 0.015s
I20260709 12:26:53.281728 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.271428 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:53.263652  1085 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.260291  1087 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.265069   970 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:26:53.321357 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:53.331187 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3790) took 2360 ms. Trace:
W20260709 12:26:53.339088 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.007s	sys 0.018s
I20260709 12:26:53.338726 26128 rpcz_store.cc:276] 0709 12:26:50.971018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.318701 (+1347683us) service_pool.cc:224] Handling call
0709 12:26:52.319037 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.326351 (+  7314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.326381 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.326839 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.178294 (+851455us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:53.178333 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.178402 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.330897 (+152495us) raft_consensus.cc:1685] finished
0709 12:26:53.330928 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.331102 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7690,"raft.queue_time_us":6834,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":851079,"threads_started":1,"wal-append.queue_time_us":981497}
W20260709 12:26:53.339970 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 864904us
I20260709 12:26:53.342832 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.345026 26203 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
W20260709 12:26:53.346213 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64581.1us
I20260709 12:26:53.273649  1095 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:53.280117  1040 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:53.326125   933 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.325364  1029 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:26:53.349980   922 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.410068   922 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:53.401520  1123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.008s	sys 0.020s
W20260709 12:26:53.322046  1162 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:26:53.355940 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84752.4us
I20260709 12:26:53.413223 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:53.413371   933 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.414557 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:53.415560   933 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.358035  1085 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.319240  1110 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.326918   519 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:53.332213  1041 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:53.328810   987 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.325726   516 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:53.318468  1013 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.013s	sys 0.047s
I20260709 12:26:53.435415  1133 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.355690  1096 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:53.355443   941 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:53.314917  1033 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:26:53.363327   970 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:53.379827 26199 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:53.388258 26127 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:53.344164 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:53.473325 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 129221us
I20260709 12:26:53.399747 26195 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:53.352833   981 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:26:53.437290  1123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.008s	sys 0.020s
W20260709 12:26:53.373126  1100 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.004s	sys 0.008s
W20260709 12:26:53.370584 26787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.023s	sys 0.035s
W20260709 12:26:53.504135 26787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.023s	sys 0.035s
I20260709 12:26:53.420147   933 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.421175  1112 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:53.440044 26166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.008s	sys 0.019s
W20260709 12:26:53.516031 26166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.008s	sys 0.019s
W20260709 12:26:53.468066 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.438925   992 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.114s	user 0.004s	sys 0.008s
I20260709 12:26:53.417740 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:53.523747 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 106075us
I20260709 12:26:53.403921  1095 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.484540 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3799) took 1218 ms. Trace:
I20260709 12:26:53.436204  1098 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.476159   516 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.442072  1096 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.445616   941 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.476015   981 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:53.558387 26786 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:53.524992  1033 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:53.526129 26127 rpcz_store.cc:276] 0709 12:26:52.266157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.454399 (+188242us) service_pool.cc:224] Handling call
0709 12:26:52.454700 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.472314 (+1017614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.479621 (+  7307us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54001c5554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:53.484451 (+  4830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145552768}
I20260709 12:26:53.543651 26128 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:53.546034  1110 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:53.471201   970 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:53.551368 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.002s	sys 0.033s
I20260709 12:26:53.548194 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:53.553710   941 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:53.570128 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.578490 26127 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
W20260709 12:26:53.505095  1100 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.004s	sys 0.009s
I20260709 12:26:53.550493  1096 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:53.572526 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.006s	sys 0.015s
W20260709 12:26:53.552955   996 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.009s	sys 0.017s
W20260709 12:26:53.574855 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.002s	sys 0.033s
W20260709 12:26:53.588917 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.04145e+06us
W20260709 12:26:53.541991 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.010s	sys 0.010s
I20260709 12:26:53.574769 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:53.556514  1087 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:53.592741 26202 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.014s	sys 0.011s
W20260709 12:26:53.593149 26202 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.014s	sys 0.011s
W20260709 12:26:53.584589 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.007s	sys 0.015s
W20260709 12:26:53.586678 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.542647  1038 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:53.590068 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.011s	sys 0.010s
I20260709 12:26:53.573972  1110 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.600268 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.601044 26054 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:53.603835 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:53.604784 26054 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:53.556695   933 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:26:53.582080   516 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:53.608907 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.612540 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.619197 26047 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:53.615891 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.620604 26203 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:53.617024 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:53.592314   996 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.009s	sys 0.017s
I20260709 12:26:53.622323 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:53.595827 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 731440us
I20260709 12:26:53.634261 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:53.624832  1038 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.614271 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:53.638235  1094 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.005s
I20260709 12:26:53.638779   516 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:53.622483 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:53.623553  1177 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:26:53.646555 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.678210 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.657076 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:53.652318 26123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.008s	sys 0.016s
I20260709 12:26:53.577214  1018 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.574714  1033 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.607743  1110 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:53.670370 26783 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.014s
I20260709 12:26:53.613682  1087 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:26:53.692456 26783 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.006s	sys 0.014s
W20260709 12:26:53.658895   905 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.008s	sys 0.029s
I20260709 12:26:53.655459 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.695937 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.696897 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.697410 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.698489 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.699424 26203 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:53.701150 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.689922 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:53.673188 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.702589 26199 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:53.666456   996 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:53.682121 26123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.009s	sys 0.016s
I20260709 12:26:53.640511  1100 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:53.619799  1052 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.710968 26201 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
W20260709 12:26:53.692972 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 211386us
W20260709 12:26:53.713505 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.012s	sys 0.016s
I20260709 12:26:53.705142   519 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.716346   519 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.711633 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:53.719002  1153 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:53.719657  1153 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:53.705037 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:53.707029 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:53.699226   905 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.008s	sys 0.029s
W20260709 12:26:53.714956 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.028s
I20260709 12:26:53.705252 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:53.719799   519 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.749079 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3792) took 2777 ms. Trace:
I20260709 12:26:53.732211 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:53.714118 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 448231us
I20260709 12:26:53.749506 26122 rpcz_store.cc:276] 0709 12:26:50.971586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.402840 (+1431254us) service_pool.cc:224] Handling call
0709 12:26:52.403151 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.442263 (+ 39112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.442290 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.442723 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.264029 (+821306us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:53.264071 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.264137 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.748866 (+484729us) raft_consensus.cc:1685] finished
0709 12:26:53.748887 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.749015 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38749,"raft.queue_time_us":48177,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":17870,"thread_start_us":749679,"threads_started":1,"wal-append.queue_time_us":1077090}
I20260709 12:26:53.751564 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.751884 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.699153  1150 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:53.754415 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.000s	sys 0.030s
W20260709 12:26:53.738119 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.018s
I20260709 12:26:53.755595 26125 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:53.606920   981 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.665141   970 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:53.760890 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53929.7us
I20260709 12:26:53.706977  1087 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:53.747392   938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.015s	sys 0.003s
I20260709 12:26:53.764791 26199 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:26:53.765587 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
W20260709 12:26:53.765780 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 581717us
I20260709 12:26:53.766398 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:53.733872  1029 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:26:53.728863   996 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.730751  1133 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:53.686781  1094 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.763540 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.738082   905 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:53.785858  1112 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:53.761716   922 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:53.758603 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:53.872146 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113616us
I20260709 12:26:53.873612 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:53.803243  1189 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:26:53.812497 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.779642 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:53.785275  1180 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:53.842836 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76527.2us
I20260709 12:26:53.815755  1098 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:53.739399  1182 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:26:53.859514 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.800477 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:53.884683 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.885469 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 85115.3us
I20260709 12:26:53.862600 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3815) took 1427 ms. Trace:
I20260709 12:26:53.887099 26128 rpcz_store.cc:276] 0709 12:26:52.435214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.340639 (+905425us) service_pool.cc:224] Handling call
0709 12:26:53.340953 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.568988 (+228035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.862181 (+293193us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001f54d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:53.862505 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465096064}
I20260709 12:26:53.888725 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.806538   981 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:53.806228   938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.015s	sys 0.003s
I20260709 12:26:53.869383 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:53.847218   933 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:53.789492   921 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:53.809532   519 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:53.878465 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:53.813822   840 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:53.814891  1190 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:53.897338 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.899207 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.901752 26050 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:53.924754 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55494.7us
I20260709 12:26:53.876479 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:53.903317 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:53.871860   922 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.913671 26053 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:53.863157  1184 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:53.875537   954 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:53.927073 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:53.881733  1085 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:53.907593  1133 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.795398   516 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.919322  1098 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:53.944736 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:53.805896  1153 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.924350 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:53.986409 26051 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:53.981642 26126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.012s	sys 0.008s
I20260709 12:26:53.859200   517 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 12:26:53.918951   921 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:53.908650  1180 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:53.875599  1112 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.960373 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:54.010737 26122 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
W20260709 12:26:53.977600 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.012300 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.985320 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1749) took 1570 ms. Trace:
I20260709 12:26:53.922755  1189 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.012864 26786 rpcz_store.cc:276] 0709 12:26:52.415218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.114088 (+698870us) service_pool.cc:224] Handling call
0709 12:26:53.114335 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.558905 (+444570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.984980 (+426075us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:53.985235 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020668288}
I20260709 12:26:53.929157  1131 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:53.974222 26791 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:53.881825   990 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:26:53.948758 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:53.945775  1085 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:53.988092 26126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.012s	sys 0.009s
W20260709 12:26:53.974330 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 312548us
I20260709 12:26:53.940384   922 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:53.994453 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:53.998158 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 363974us
I20260709 12:26:53.989578 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3811) took 1606 ms. Trace:
W20260709 12:26:54.002420 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116155us
W20260709 12:26:54.005656 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 225915us
I20260709 12:26:54.008894 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:54.036439 26788 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:54.019100 26790 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:26:54.032938 26124 rpcz_store.cc:276] 0709 12:26:52.383549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.273129 (+889580us) service_pool.cc:224] Handling call
0709 12:26:53.273509 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.273747 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:53.748319 (+474572us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.989101 (+240782us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.989178 (+    77us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:53.989252 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.989342 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.989504 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134734,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"thread_start_us":659694,"threads_started":2}
I20260709 12:26:53.946875   954 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:53.987802 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:54.020851 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:53.981269   973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.012s	sys 0.012s
I20260709 12:26:53.960359   992 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:53.900331  1150 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.965337  1013 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
W20260709 12:26:54.028831  1188 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 12:26:53.959280  1184 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:54.042593 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:53.923141   933 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.045578 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:54.046574 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 159890us
W20260709 12:26:54.107105 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64657.5us
I20260709 12:26:54.107741 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:54.039988 26128 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 18.
I20260709 12:26:54.055855 26051 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:53.922478   519 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:53.930821   840 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:54.055194 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181665us
I20260709 12:26:54.063601  1098 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:54.074805 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:54.086927 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.122330 26202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.113s	sys 0.106s
W20260709 12:26:54.115753 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70262us
W20260709 12:26:54.122931 26202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.113s	sys 0.106s
I20260709 12:26:54.123404 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:54.123942 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2423) took 1304 ms. Trace:
I20260709 12:26:54.108325 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:54.124440 26202 rpcz_store.cc:276] 0709 12:26:52.819202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.037817 (+218615us) service_pool.cc:224] Handling call
0709 12:26:53.053877 (+ 16060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.054109 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:53.593604 (+539495us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.123513 (+529909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.123593 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:54.123657 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.123720 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.123830 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":214532,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":13815,"thread_start_us":1042529,"threads_started":2}
W20260709 12:26:54.075517   973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.012s	sys 0.012s
I20260709 12:26:54.034055  1085 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:53.936371  1087 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.937948  1190 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.078565  1033 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.058784   954 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:53.981434  1193 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.153s
I20260709 12:26:54.033453  1018 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:54.139454 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.141117 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:53.965193  1133 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:53.964426   905 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.077768  1186 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 12:26:54.121637  1183 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:26:54.218266 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:54.212580   517 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:54.385875   517 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:54.196184  1182 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:54.387532  1096 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.797s	user 0.006s	sys 0.002s
W20260709 12:26:54.303203   963 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.995s	user 0.004s	sys 0.008s
I20260709 12:26:54.388330   963 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.389609 26120 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:54.144927   992 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:54.263084  1206 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:26:54.215059 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:54.220870  1094 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:54.394313 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.109382   990 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:54.220445 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3801) took 1953 ms. Trace:
I20260709 12:26:54.420670 26123 rpcz_store.cc:276] 0709 12:26:52.266794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.534944 (+268150us) service_pool.cc:224] Handling call
0709 12:26:52.579428 (+ 44484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.776083 (+196655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.776118 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.776800 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.708749 (+931949us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:53.708810 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.708904 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.220105 (+511201us) raft_consensus.cc:1685] finished
0709 12:26:54.220136 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.220371 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227746,"raft.queue_time_us":227684,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15843,"thread_start_us":823640,"threads_started":1,"wal-append.queue_time_us":1048674}
I20260709 12:26:54.127857 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2405) took 1746 ms. Trace:
I20260709 12:26:54.176795 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:54.317906 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3814) took 1883 ms. Trace:
I20260709 12:26:54.254102 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:54.395220 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1292) took 2631 ms. Trace:
I20260709 12:26:54.405821 26128 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:54.080705 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:54.428783 26124 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:54.174198 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:54.429354 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 175360us
I20260709 12:26:54.429833 26201 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:26:54.419637 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 278580us
I20260709 12:26:54.430990 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:54.431825 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:54.634230 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.010s	sys 0.010s
W20260709 12:26:54.635064 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.010s	sys 0.010s
W20260709 12:26:54.433080 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.636085 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:54.637434 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.638763 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.638949 26123 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:54.428166 26126 rpcz_store.cc:276] 0709 12:26:51.763883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.433204 (+669321us) service_pool.cc:224] Handling call
0709 12:26:52.441688 (+  8484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.441877 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:53.249446 (+807569us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.394644 (+1145198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.394778 (+   134us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:54.394890 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.394981 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.395158 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19877,"prepare.queue_time_us":966426,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":26,"raft.queue_time_us":180607,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4432,"thread_start_us":1893640,"threads_started":3,"wal-append.queue_time_us":677779}
W20260709 12:26:54.636108 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.423197 26197 rpcz_store.cc:276] 0709 12:26:52.381366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.381540 (+   174us) service_pool.cc:224] Handling call
0709 12:26:52.381830 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.382040 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:53.245242 (+863202us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.127522 (+882280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.127574 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:54.127621 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.127685 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.127804 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":856911,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":4526,"raft.queue_time_us":152921,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":11233,"thread_start_us":1694279,"threads_started":3}
W20260709 12:26:54.642886 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.016s
W20260709 12:26:54.643424 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.005s	sys 0.016s
W20260709 12:26:54.645887 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 522491us
W20260709 12:26:54.648617 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.022s
W20260709 12:26:54.649250 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.000s	sys 0.022s
W20260709 12:26:54.649703 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 525224us
W20260709 12:26:54.650511   941 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.062s	user 0.010s	sys 0.006s
I20260709 12:26:54.651273   941 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.425791 26121 rpcz_store.cc:276] 0709 12:26:52.434851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.315696 (+880845us) service_pool.cc:224] Handling call
0709 12:26:53.316151 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.316582 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.316632 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.317205 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.754648 (+437443us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:53.793526 (+ 38878us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.883972 (+ 90446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.884148 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.317634 (+433486us) raft_consensus.cc:1685] finished
0709 12:26:54.317662 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.317835 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140955,"prepare.queue_time_us":49531,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":61,"raft.queue_time_us":151893,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":27169,"thread_start_us":437104,"threads_started":1,"wal-append.queue_time_us":937321}
W20260709 12:26:54.652571 26783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.022s
W20260709 12:26:54.653121 26783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.006s	sys 0.022s
I20260709 12:26:54.653631   519 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.654736   519 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.655400   941 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.657383 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:54.237496   938 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:54.660831   938 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:54.661300 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:54.661681   938 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:54.114611   933 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.665722   933 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:26:54.256973   709 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.031s	user 0.013s	sys 0.000s
I20260709 12:26:54.666466   516 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:54.667045   516 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.667479   516 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:54.206466   973 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:54.640739 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:54.669763   973 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:54.666503   933 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.671821   973 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.672629 26122 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:54.674229   933 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.437605   963 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.669775 26123 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
W20260709 12:26:54.683975   981 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.781s	user 0.016s	sys 0.001s
I20260709 12:26:54.684654 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:54.241385   996 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:54.256760  1049 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.912s	user 0.013s	sys 0.000s
W20260709 12:26:54.306370  1019 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.130s	user 0.009s	sys 0.004s
I20260709 12:26:54.686375  1100 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:54.686744  1049 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.687266  1019 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.688807  1100 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:54.689869  1100 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.690315  1049 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.690502  1019 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:54.522154  1040 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.101s	user 0.009s	sys 0.005s
I20260709 12:26:54.482338  1094 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
W20260709 12:26:54.304658  1086 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.214s	user 0.012s	sys 0.002s
W20260709 12:26:54.692638 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.693325  1040 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.693529  1086 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.694823 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:54.557732   992 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.393632  1096 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.696619 26196 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:54.696790 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:54.699560  1096 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.179718   921 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.700507   941 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:54.700914 26126 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:54.507243 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 432555us
I20260709 12:26:54.251185  1098 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.701715  1086 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.702041  1040 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:54.703217 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 271335us
I20260709 12:26:54.703687  1100 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.434724  1131 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:54.705168  1133 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:54.261268  1112 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.545540  1153 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.530855  1149 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:54.079532  1175 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.357s
I20260709 12:26:54.805213  1153 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.807253  1175 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:54.807766  1175 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:54.473021   990 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.215371  1180 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.261457  1184 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.162211  1189 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:54.812997  1189 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.813571  1189 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:54.164065  1194 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:54.816956  1194 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:54.818444  1194 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.818928 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:54.819870  1194 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.434078  1197 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.260370  1199 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:54.831851  1199 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:54.833480  1199 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.834422  1068 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.835304 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2424) took 1972 ms (client timeout 2160 ms). Trace:
I20260709 12:26:54.835682  1199 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.835269  1068 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:54.260524  1201 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
W20260709 12:26:54.295357  1208 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
W20260709 12:26:54.835947 26196 rpcz_store.cc:269] 0709 12:26:52.863201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.023087 (+159886us) service_pool.cc:224] Handling call
0709 12:26:54.834889 (+1811802us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400378554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:54.835118 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3797486336}
W20260709 12:26:54.452003  1211 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
W20260709 12:26:54.325028  1218 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:26:54.838546   881 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:54.228953  1222 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.288s
I20260709 12:26:54.839206   881 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:54.127646  1223 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:54.491344  1230 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:54.841185 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1744) took 2427 ms. Trace:
W20260709 12:26:54.642367 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.841728 26787 rpcz_store.cc:276] 0709 12:26:52.413653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.524472 (+110819us) service_pool.cc:224] Handling call
0709 12:26:52.524852 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.541480 (+ 16628us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:52.541896 (+   416us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.881309 (+1339413us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.186904 (+305595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.186960 (+    56us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:54.187006 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.840671 (+653665us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b54005c2954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:54.840733 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.840958 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12080,"prepare.queue_time_us":415212,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":91,"raft.queue_time_us":335853,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":30842,"spinlock_wait_cycles":51942528,"thread_start_us":1638997,"threads_started":3,"wal-append.queue_time_us":1149278}
I20260709 12:26:54.843729  1095 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:54.590943  1212 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:54.439231  1018 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.845546  1018 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.694096  1094 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:54.691457   984 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:54.848839   984 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:26:54.705196 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2843) took 1665 ms (client timeout 1980 ms). Trace:
I20260709 12:26:54.717422 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:54.708313   941 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:54.850407   984 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:54.850968 26125 rpcz_store.cc:269] 0709 12:26:53.039943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.214588 (+174645us) service_pool.cc:224] Handling call
0709 12:26:54.704702 (+1490114us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54001da954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:54.704960 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224137472}
I20260709 12:26:54.716971   933 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:54.852103   933 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:54.750527 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 319410us
I20260709 12:26:54.758786 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:54.713037   938 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:54.776618 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 138493us
W20260709 12:26:54.790796  1110 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.084s	user 0.015s	sys 0.000s
I20260709 12:26:54.411777   517 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.792346  1228 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:54.793164 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1745) took 2379 ms. Trace:
I20260709 12:26:54.793977   954 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.797115  1190 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.801640  1236 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.802337 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:54.295521  1195 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 12:26:54.805125  1149 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:54.827936 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:54.828058  1131 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:54.864114  1068 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:54.845491   881 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.691526  1019 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:54.893127  1019 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:54.893615 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65914.4us
I20260709 12:26:54.893630   881 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.894666  1019 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:54.706372 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.003s	sys 0.018s
W20260709 12:26:54.896641 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.004s	sys 0.019s
I20260709 12:26:54.854223   933 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:54.897471 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 180323us
I20260709 12:26:54.863580 26125 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:54.899091 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:54.899636  1179 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.155s	user 0.002s	sys 0.002s
I20260709 12:26:54.901940 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:54.902988  1185 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:54.903215  1197 raft_consensus.cc:2764] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:54.904628  1235 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:54.906016  1235 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:54.906601  1235 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:54.867535 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.908398   984 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:54.909641   982 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:54.896533  1095 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:54.914472  1095 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:54.914568  1052 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:54.917342 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:54.918131 26045 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.891741  1149 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.920789 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:54.921308  1149 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:54.921554 26047 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:54.925557 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:54.925661 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 123561us
I20260709 12:26:54.927068 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:54.916088  1095 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.928978   709 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.929778   709 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:54.931223 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2392) took 1726 ms. Trace:
I20260709 12:26:54.932684  1228 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:54.933653  1228 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:26:54.935063 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 222037us
W20260709 12:26:54.938870 26756 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.402s (SENT) [suppressed 26 similar messages]
I20260709 12:26:54.829811  1153 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.903429 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2853) took 1116 ms. Trace:
I20260709 12:26:54.940670 26122 rpcz_store.cc:276] 0709 12:26:53.787220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.011532 (+224312us) service_pool.cc:224] Handling call
0709 12:26:54.903170 (+891638us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:54.903346 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450075008}
W20260709 12:26:54.941751  1187 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.186s	user 0.001s	sys 0.001s
I20260709 12:26:54.942462 26122 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:54.945159 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1304) took 1231 ms. Trace:
I20260709 12:26:54.945542 26126 rpcz_store.cc:276] 0709 12:26:53.713388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.641510 (+928122us) service_pool.cc:224] Handling call
0709 12:26:54.642157 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.710642 (+ 68485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.944881 (+234239us) spinlock_profiling.cc:243] Waited 58.4 ms on lock 0x7b54001f5ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:54.945082 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134255488}
I20260709 12:26:54.948078 26787 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
W20260709 12:26:54.949493 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1187) took 2433 ms (client timeout 2402 ms). Trace:
W20260709 12:26:54.950121 26194 rpcz_store.cc:269] 0709 12:26:52.516474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.516703 (+   229us) service_pool.cc:224] Handling call
0709 12:26:54.948965 (+2432262us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:54.949201 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5206214016}
I20260709 12:26:54.953158 26125 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:26:54.953112 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2840) took 1913 ms. Trace:
I20260709 12:26:54.954361 26120 rpcz_store.cc:276] 0709 12:26:53.039103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.270960 (+231857us) service_pool.cc:224] Handling call
0709 12:26:54.952585 (+1681625us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540006fed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:54.952911 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2476143488}
I20260709 12:26:54.861814  1040 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.955761 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:54.955616  1040 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:54.956144  1215 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.375s	user 0.001s	sys 0.001s
I20260709 12:26:54.957026 26093 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:54.958241  1040 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:54.959540   938 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:54.896381  1242 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.890467 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:54.812055   990 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:54.969815  1040 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:54.970361 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:54.970391  1040 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.971047  1040 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:54.971249   963 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:54.971786   963 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.972136   963 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.972489  1193 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:26:54.975863 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.979784   519 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.870473  1110 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.876566  1018 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:54.878046  1094 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:54.852355   941 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.982303  1040 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:54.983103   922 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.904s	user 0.021s	sys 0.001s
I20260709 12:26:54.984803 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:54.985126 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3800) took 2718 ms. Trace:
I20260709 12:26:54.986626 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:54.970124  1236 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:54.966135   516 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:54.990658 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91793.7us
I20260709 12:26:54.993271 26045 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:54.865427 26783 rpcz_store.cc:276] 0709 12:26:52.413951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.099007 (+685056us) service_pool.cc:224] Handling call
0709 12:26:53.114098 (+ 15091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.114450 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:53.692877 (+578427us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.759588 (+1066711us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.792539 (+ 32951us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.792620 (+    81us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:54.792686 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.792805 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.793008 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14358,"prepare.queue_time_us":939280,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":111,"raft.queue_time_us":325913,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":96016,"spinlock_wait_cycles":2555648,"thread_start_us":1556210,"threads_started":4,"wal-append.queue_time_us":877956}
I20260709 12:26:54.996516 26047 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:54.931618 26052 rpcz_store.cc:276] 0709 12:26:53.204509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.230035 (+ 25526us) service_pool.cc:224] Handling call
0709 12:26:53.230303 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.230489 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:53.689205 (+458716us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.930228 (+1241023us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b180040fd20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:26:54.930531 (+   303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.930631 (+   100us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:54.930726 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.930853 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.931045 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":648071,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":15751,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":71036544,"thread_start_us":648768,"threads_started":2,"wal-append.queue_time_us":757623}
I20260709 12:26:54.999533 26129 rpcz_store.cc:276] 0709 12:26:52.266509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.499110 (+232601us) service_pool.cc:224] Handling call
0709 12:26:52.544610 (+ 45500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.768974 (+224364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.769008 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.807655 (+ 38647us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.376595 (+568940us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:53.376665 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.376759 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.940837 (+1564078us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:26:54.960645 (+ 19808us) raft_consensus.cc:1685] finished
0709 12:26:54.960671 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.984914 (+ 24243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223925,"raft.queue_time_us":224061,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":16125,"spinlock_wait_cycles":50213632,"thread_start_us":486755,"threads_started":1,"wal-append.queue_time_us":787334}
I20260709 12:26:55.012944 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.013381  1087 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:55.013893 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:55.013724   922 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.014111  1087 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.016052   954 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.019411 26789 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:55.016407 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.026726 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 99764.7us
W20260709 12:26:55.027329 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 123501us
W20260709 12:26:55.030933 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 140847us
I20260709 12:26:55.032146 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:55.034299   954 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:55.034906   954 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.031147 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2399) took 1139 ms. Trace:
I20260709 12:26:55.022588 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:55.036587 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.037093 26054 rpcz_store.cc:276] 0709 12:26:53.892259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.892427 (+   168us) service_pool.cc:224] Handling call
0709 12:26:53.892783 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.162107 (+269324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.162154 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.162213 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:54.162258 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.028642 (+866384us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400172d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:55.028714 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.030968 (+  2254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618740480}
I20260709 12:26:55.040972 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:55.041345 26124 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.612s	user 0.015s	sys 0.005s
I20260709 12:26:55.041944 26125 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:55.041975 26124 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
I20260709 12:26:55.042903 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2851) took 1256 ms. Trace:
I20260709 12:26:55.043277 26124 rpcz_store.cc:276] 0709 12:26:53.786597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.065147 (+278550us) service_pool.cc:224] Handling call
0709 12:26:55.042823 (+977676us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.020983   709 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.041023 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:55.046665   516 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.036224   954 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.051896  1235 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.053335  1040 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.053691 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:55.054235 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.055724 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.055938  1242 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:55.016202   922 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.056708 26125 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 17.
I20260709 12:26:55.058195  1018 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:55.058866  1018 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.059433  1018 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.059525 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.061705 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.063791 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.068603 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.070641 26783 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:26:55.071265 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.022539   941 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.018821  1110 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.019770  1197 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:55.077291 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.053624  1235 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.081218 26123 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:26:55.109127 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.109805 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.079561 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.113842 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.115154 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.134049 26785 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:55.084725  1197 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.075680   519 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:55.138840 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.135393 26194 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.992286  1175 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.141624 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:55.143807 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.142482 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 203664us
W20260709 12:26:55.138060 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61318.3us
I20260709 12:26:55.145749  1019 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.162945  1019 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.163820 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.140621  1197 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:55.164933 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50841.4us
I20260709 12:26:55.146498 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.165941  1019 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.166342 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.167840  1019 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:55.168066  1214 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.647s	user 0.000s	sys 0.003s
I20260709 12:26:55.168404   990 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:55.169075   990 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.169819 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.170854 26198 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:55.170377   990 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.148770 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.172523 26051 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:55.173652 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.150693 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.129565  1087 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:55.152900 26128 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.687s	user 0.012s	sys 0.004s
I20260709 12:26:55.153034 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.153268  1180 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.154241 26050 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:26:55.154930 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.186208 26129 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.157537 26049 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:55.157958 26124 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:55.161134  1226 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.162000 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.167080 26052 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:55.172374 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:55.151921 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83425.4us
I20260709 12:26:55.149621 26046 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:55.189890 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.195837 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:55.200958  1246 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 12:26:55.201227  1252 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:55.202109   921 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.174513  1189 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.202659 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.204509  1189 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.204578   517 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.205302 26128 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:55.205991 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3820) took 1085 ms. Trace:
I20260709 12:26:55.205853   963 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.206375 26128 rpcz_store.cc:276] 0709 12:26:54.123837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.180755 (+ 56918us) service_pool.cc:224] Handling call
0709 12:26:55.205883 (+1025128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.208788 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3821) took 1084 ms. Trace:
I20260709 12:26:55.209824 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.211035   981 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:55.211653   981 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.212771 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.212890   981 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.214800 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.215543 26050 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.209318 26121 rpcz_store.cc:276] 0709 12:26:54.124133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.686015 (+561882us) service_pool.cc:224] Handling call
0709 12:26:54.686746 (+   731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.687416 (+   670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.687462 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.707754 (+ 20292us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.964558 (+256804us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:54.964600 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.992140 (+ 27540us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.208409 (+216269us) raft_consensus.cc:1685] finished
0709 12:26:55.208434 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.208600 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45736,"raft.queue_time_us":54787,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":34795,"thread_start_us":199732,"threads_started":1,"wal-append.queue_time_us":341572}
W20260709 12:26:55.222839 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.223439 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.224574  1236 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.230139 26198 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:55.231006   963 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:55.231726  1249 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 12:26:55.231968 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.232908 26047 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:55.233177 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71343.5us
I20260709 12:26:55.233963  1189 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.234364  1098 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.234961  1098 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.236414 26051 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:55.236485  1226 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.237273  1098 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:55.237658 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.237944  1184 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.238456 26051 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:55.238921  1226 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.238693  1184 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:55.239943 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 56270us
I20260709 12:26:55.240361  1180 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.242604  1180 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:55.242966 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.243409 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.243458  1038 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.243755 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.244024  1038 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:55.245523  1171 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.694s	user 0.006s	sys 0.004s
I20260709 12:26:55.245857 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.246630  1038 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.209452   992 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:55.248020   992 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:55.248418 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.239450 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.250517 26202 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:55.213595 26052 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.257742 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:55.261740 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:55.265973 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:55.267418 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:55.268245 26784 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:55.269569  1085 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.107s	user 0.015s	sys 0.003s
I20260709 12:26:55.269891  1184 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.270061  1085 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.209452   970 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.270722  1189 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:55.277451  1233 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:55.279567  1233 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:55.285179 26050 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
W20260709 12:26:55.286085 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.266319 26194 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:26:55.276088 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.286211 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:55.256889   992 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.285094 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.294566 26051 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:55.290333 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.290292 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.239494  1226 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.299492 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.303149 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.305204 26785 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:55.309103  1098 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:55.309468  1184 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:55.310382 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:55.313474 26789 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1752) took 1369 ms (client timeout 1612 ms). Trace:
W20260709 12:26:55.313951 26789 rpcz_store.cc:269] 0709 12:26:53.943611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.945508 (+  1897us) service_pool.cc:224] Handling call
0709 12:26:55.313221 (+1367713us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400715a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:55.313392 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271482752}
I20260709 12:26:55.316395 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2841) took 2277 ms. Trace:
I20260709 12:26:55.316826 26127 rpcz_store.cc:276] 0709 12:26:53.039327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.580534 (+541207us) service_pool.cc:224] Handling call
0709 12:26:53.580912 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.715359 (+134447us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:53.746896 (+ 31537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.897370 (+1150474us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.897492 (+   122us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:54.897594 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.316040 (+418446us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54001f95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:55.316091 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.316318 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13665,"prepare.queue_time_us":75437,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":6915,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":4216,"spinlock_wait_cycles":308605312,"thread_start_us":1149919,"threads_started":1,"wal-append.queue_time_us":837361}
I20260709 12:26:55.318413 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.299549  1052 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:55.319994  1052 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.320442  1052 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:55.321606  1259 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:26:55.323701 26045 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
W20260709 12:26:55.332000 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66141us
I20260709 12:26:55.323966 26049 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:26:55.337806 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.339936 26050 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:55.341312 26051 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.344487 26048 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.345304 26046 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21.
I20260709 12:26:55.344888 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.346276 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:55.350283 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:26:55.360347 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.361251 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:55.362605 26784 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:55.370903  1041 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.371747  1041 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.356037  1194 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:55.374642  1194 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.374958  1041 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.375069  1194 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.375809 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.377125 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.377854 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:55.321751  1226 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.382126  1226 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:55.383384 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 189995us
I20260709 12:26:55.384361 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:55.386436  1185 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:55.392445 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53136us
I20260709 12:26:55.352674   941 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:26:55.352946   982 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.395023   982 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.394500 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.364019 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.401216 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.403673 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:55.401039 26054 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:55.357450  1098 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:55.458652  1098 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:55.459673  1098 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.451411  1018 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.407855 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.463999 26784 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:55.409869  1185 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:55.412376  1221 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:55.465281  1221 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.465689  1221 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.413079 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:55.468487 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.416401   970 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:26:55.416949 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.470211 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.471364 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.472105 26121 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:55.417723  1180 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:55.472738 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3819) took 1352 ms. Trace:
I20260709 12:26:55.472891 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.473127  1180 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:55.473670  1180 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.418089 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.477043 26124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:26:55.477856 26124 raft_consensus.cc:606] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:55.418337 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.418793 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.417742 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.420567 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.427965 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.366837   933 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.432827 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.402067  1175 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.437438   840 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.440587 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.441419 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.442512  1085 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.442812  1217 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.449537   992 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.452615 26785 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:55.453161  1190 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:55.479882   982 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.397950  1197 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.495105 26120 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.495054  1217 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.486994  1041 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:55.496688  1041 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.457863 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.497527  1217 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.406000 26127 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:55.498584  1190 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.499444 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.500037 26127 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:55.488070 26050 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:55.501742 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.502431 26050 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:55.488256 26122 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.480011 26126 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:26:55.472887 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.489073 26129 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:26:55.498629 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:55.490262 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49778us
I20260709 12:26:55.490657 26047 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:55.473127 26123 rpcz_store.cc:276] 0709 12:26:54.119999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.677804 (+557805us) service_pool.cc:224] Handling call
0709 12:26:55.049724 (+371920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.111155 (+ 61431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.472544 (+361389us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400065ed0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:55.472663 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842407808}
I20260709 12:26:55.492091 26046 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:55.405905   941 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:26:55.500813  1190 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:55.486485 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 88444.5us
I20260709 12:26:55.504117 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.503161 26052 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:26:55.494715 26048 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:26:55.515882 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.519834 26051 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:55.526584 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.527289 26050 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:55.486595  1086 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:55.503938 26054 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22. Re-sending same reply.
I20260709 12:26:55.531751 26787 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:55.532919 26120 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:55.530977 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.533972 26045 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:55.537863 26127 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:55.493769  1235 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:55.486788   984 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.540978 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.543136 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.541009 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.544528 26045 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:55.534164   954 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:55.541086  1085 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.546540 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.547241 26046 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21.
I20260709 12:26:55.547406 26125 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 12:26:55.508515   922 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:55.547900 26125 pending_rounds.cc:85] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 16
I20260709 12:26:55.546051   954 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.550254 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.551317 26054 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21.
I20260709 12:26:55.551961 26196 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:55.554517   954 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.554852 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.555689 26050 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:55.556617 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.556797 26123 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:26:55.497604  1041 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.557950 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.559131 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.561718   984 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.562620 26195 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:55.563836 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.564057 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.564749 26049 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.564985 26051 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:55.551982 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:55.567005 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.568629 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:55.571780 26047 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:55.548262 26125 pending_rounds.cc:107] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:26:55.574285 26125 raft_consensus.cc:2889] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:55.537448  1197 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.574342 26129 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:55.577323 26052 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:55.544036  1235 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.508315   840 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:55.579700 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.578439 26048 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:55.584182 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:55.589382 26051 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:55.584834 26195 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:55.541605  1086 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:55.593379 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.527902   982 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:26:55.599954 26045 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:55.602262 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.605623 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.606189 26054 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:55.606487 26784 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:55.580027  1236 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:55.610153 26047 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:55.613345 26198 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:55.614218 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 397517us
I20260709 12:26:55.614737 26047 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:55.591602  1235 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.618287 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.588097  1018 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.618023 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:55.621984 26168 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.716s (SENT)
I20260709 12:26:55.622732 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.633100 26048 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:26:55.639309 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.615756  1087 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.612423  1236 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:55.647682 26785 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:55.654006 26048 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:55.656057   990 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:55.656983   990 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.658205   922 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.658015   990 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.658697   922 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.642794  1110 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:26:55.610288  1052 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.665009 26124 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:26:55.664680  1086 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.666033  1052 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.662441  1018 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:26:55.667932  1018 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:55.668324  1018 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:55.669680   963 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:55.669502  1018 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.670082   963 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.670143   922 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.670885  1018 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.671028   963 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.659965 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.673840  1197 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:55.651093  1236 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:55.684135 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.662381  1087 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:55.697070 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.651687 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.661890 26048 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:55.715230 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56867.2us
W20260709 12:26:55.715642 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64007.1us
I20260709 12:26:55.716581 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:55.718472 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.724581 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:55.727720   992 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.727623 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:55.739725 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.625923   840 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.717175   963 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:55.745327 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 59932.5us
I20260709 12:26:55.749030 26047 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:55.647339  1180 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.742653 26787 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:55.756304 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:55.756917 26785 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.725780  1052 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:55.725909   922 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:55.755622 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:55.737524  1040 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:55.766063  1040 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.770442  1040 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.756946 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:55.777411 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.790073 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:55.791014 26196 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:55.764117  1197 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:26:55.809139 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.811422 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.811497 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:55.811431 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.812037 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.812631 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.814384 26198 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.814407 26129 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:26:55.810840 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:55.817780 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:55.818080 26127 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:55.815163 26129 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 13
I20260709 12:26:55.819231 26129 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:26:55.819310 26195 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:55.817421 26125 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:55.819352 26124 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.819531 26129 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:55.773808  1052 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.824700 26120 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:55.831265 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:55.816424  1197 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:26:55.851352 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.823616  1087 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.826974  1190 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:55.858462 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 69095.8us
I20260709 12:26:55.872264 26196 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.885478 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.816798  1185 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 12:26:55.897163  1194 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.521s	user 0.015s	sys 0.000s
I20260709 12:26:55.897670  1194 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.900517  1087 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:55.901157  1087 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.903422  1087 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.913738   840 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:55.914800 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62129.8us
I20260709 12:26:55.857929   933 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.916882  1193 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:55.917351  1193 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:55.917719  1193 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:55.856868  1197 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.920730  1052 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.933055 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:55.949648  1085 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:55.950620  1085 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:55.953455  1085 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:55.955076 26784 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:55.955929  1190 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:55.957042  1190 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.957756   992 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:55.958981   981 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:55.959498   981 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:55.961127  1180 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.960721   981 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.962507   981 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.962800  1221 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:55.965242  1221 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.967650   954 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.969834   954 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.893025   984 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:55.976509 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60275.9us
I20260709 12:26:55.976696  1085 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:55.980201 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94789.1us
I20260709 12:26:55.981123 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:55.981997   984 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:55.982527 26020 leader_election.cc:400] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:55.982865 26020 leader_election.cc:403] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:55.983479   984 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:55.900050  1194 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:55.903949  1087 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.894090  1184 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:55.935925 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:55.999100 26124 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:55.983892 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:56.003495 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:56.018646 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.021229 26122 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:56.024135 26049 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:26:56.025650 26198 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:26:56.029719 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 406291us
W20260709 12:26:56.051829 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.020385   840 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:56.035779  1189 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:56.067126 26785 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:55.992926  1087 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.028506   984 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:26:56.062636   840 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:56.083395  1235 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:55.988768  1040 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:56.079890  1175 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.587s	user 0.017s	sys 0.000s
I20260709 12:26:56.148253 26194 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:26:56.151729 26128 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:26:56.096069   840 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:56.140179   963 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.174028 26045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.024s	sys 0.051s
W20260709 12:26:56.174521 26045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.024s	sys 0.051s
I20260709 12:26:56.259526 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.277796 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:56.319096 26122 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:56.256223 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:56.227736 26054 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:26:56.257872 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:26:56.221432 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 465872us
I20260709 12:26:56.320459 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:56.231933 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:56.231529 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.321704 26047 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:56.167838  1197 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.307576 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:56.309053 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2420) took 1049 ms. Trace:
I20260709 12:26:56.166821  1110 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:56.329207 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 984) took 1140 ms. Trace:
W20260709 12:26:56.329581 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.327353 26046 rpcz_store.cc:276] 0709 12:26:55.259393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.579152 (+319759us) service_pool.cc:224] Handling call
0709 12:26:55.579440 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.772321 (+192881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.772364 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.772440 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:55.772476 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.308823 (+536347us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540022ae54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:56.308874 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.308996 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428622208}
I20260709 12:26:56.332147 26046 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:56.326543 26054 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:56.310205 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2425) took 1016 ms. Trace:
I20260709 12:26:56.337689 26052 rpcz_store.cc:276] 0709 12:26:55.298118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.591054 (+292936us) service_pool.cc:224] Handling call
0709 12:26:55.607632 (+ 16578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.616798 (+  9166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.616821 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:55.649364 (+ 32543us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.090922 (+441558us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:56.090986 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.091091 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.310014 (+218923us) raft_consensus.cc:1685] finished
0709 12:26:56.310034 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.310151 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3115,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3159,"thread_start_us":406038,"threads_started":1,"wal-append.queue_time_us":570880}
I20260709 12:26:56.313498  1235 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:56.314926 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 379045us
W20260709 12:26:56.317584 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96968.7us
I20260709 12:26:56.229748 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.318730 26050 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:56.319895 26048 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:56.302695  1019 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:56.321980 26196 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:56.175951   963 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.329730 26049 rpcz_store.cc:276] 0709 12:26:55.182536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.405085 (+222549us) service_pool.cc:224] Handling call
0709 12:26:56.322917 (+917832us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b5400172354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:56.323035 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174003072}
I20260709 12:26:56.335304 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:56.349870 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:56.323726  1041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.010s	sys 0.013s
I20260709 12:26:56.323549  1040 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.362485 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.366492 26052 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:56.359609 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 358787us
I20260709 12:26:56.368156 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:56.369822 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.254133  1194 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.118s
W20260709 12:26:56.373286 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.353228   938 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.355998  1098 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.007s	sys 0.019s
I20260709 12:26:56.376713 26053 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:56.357123  1019 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.386090 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:56.366621  1041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.010s	sys 0.013s
W20260709 12:26:56.409565 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46371.6us
I20260709 12:26:56.414458 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.370074  1268 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:56.387441  1098 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.007s	sys 0.020s
I20260709 12:26:56.416697 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.416040 26053 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:56.418962 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:56.421463 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.423305 26196 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:56.428802 26046 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
W20260709 12:26:56.415349 26121 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.014s	sys 0.010s
W20260709 12:26:56.433604 26121 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.015s	sys 0.010s
W20260709 12:26:56.432977 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46981.9us
I20260709 12:26:56.434906 26048 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:56.435020 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:56.431907 26047 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
W20260709 12:26:56.413590 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 394017us
I20260709 12:26:56.410068  1019 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:56.439893 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.443208 26202 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:56.443526 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:56.445807 26201 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:56.440798 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.446818 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.449609  1180 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:56.450467  1274 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:56.450629   933 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:56.451094   933 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:56.451273  1197 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.451877  1272 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:26:56.453783 26203 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:56.455050 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.456540  1275 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.122s
I20260709 12:26:56.456916 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.456588 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:56.457670 26198 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:56.457695  1197 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:56.458564 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.458878  1197 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:56.459528  1197 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:56.459865 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.460549 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.415287  1184 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:56.469051 26122 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
W20260709 12:26:56.472478 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.475149 26052 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:56.376283   938 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.420926  1087 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.477283 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.481211 26050 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:56.477397  1272 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:56.487669 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.488322 26203 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:56.485823 26197 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:26:56.494383 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:56.495430 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:56.469844  1193 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.000s
W20260709 12:26:56.496999 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.499419 26203 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:56.500550 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:56.501164 26050 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:56.493074   982 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.505525 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.509244 26047 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:56.506222 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.506265 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.511893 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:56.512900 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.514263 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.516474 26203 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:56.492389  1098 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.495198  1087 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:56.469980  1268 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:26:56.525143 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:56.533149 26052 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:56.504355  1189 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:56.527325  1098 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.527941   938 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:56.505702  1193 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.511122   982 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.502190  1185 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:26:56.529428  1087 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:56.549223 26053 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:56.536347 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:56.551529 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.624912 26202 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:56.625070 26050 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:56.625136 26198 raft_consensus.cc:2376] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21. Re-sending same reply.
I20260709 12:26:56.517802  1052 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:56.607085 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.543401 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:56.628268 26049 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:56.636113 26047 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:26:56.607517  1273 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 12:26:56.607401  1236 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.925s	user 0.010s	sys 0.007s
I20260709 12:26:56.649535 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.650630 26202 raft_consensus.cc:2368] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:56.656126 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.637837   938 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:56.656620  1269 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 12:26:56.646062  1281 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:26:56.544422  1189 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.641875  1052 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.671447  1283 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:56.647965  1236 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:56.692063 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 245074us
W20260709 12:26:56.715200 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 280237us
I20260709 12:26:56.702179   963 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:56.734534 26789 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:56.652601  1184 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.735654 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.719018  1289 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:56.733726   941 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:56.748960  1289 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:56.750173  1285 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 12:26:56.749974  1289 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.750933  1285 consensus_queue.cc:1243] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:56.751441  1285 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:56.751508  1289 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:56.751852  1285 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:56.752338 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.712139  1283 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:56.753933 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.757946 26789 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:26:56.700909   938 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.739015  1288 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:56.712447  1197 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:56.767207 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.772871 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3727) took 1359 ms. Trace:
I20260709 12:26:56.773290 26045 rpcz_store.cc:276] 0709 12:26:55.413840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.617234 (+203394us) service_pool.cc:224] Handling call
0709 12:26:55.629988 (+ 12754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.638240 (+  8252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.638273 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:55.653300 (+ 15027us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.227778 (+574478us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:56.227822 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.227887 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.772669 (+544782us) raft_consensus.cc:1685] finished
0709 12:26:56.772690 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.772816 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":521286,"threads_started":1,"wal-append.queue_time_us":741515}
I20260709 12:26:56.758939  1297 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.707816  1290 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:26:56.802738  1290 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:56.803148  1290 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:56.759075  1283 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:56.804371 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:56.804930  1287 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:26:56.805079 26124 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:56.807574 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.808307 26197 raft_consensus.cc:2376] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20. Re-sending same reply.
I20260709 12:26:56.808764 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.809377 26195 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:26:56.812294  1291 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:56.812889  1291 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.814939  1291 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:56.803968  1290 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.815176  1197 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.815601 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.816072  1290 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:56.816388 26202 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:56.752736  1285 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.817158  1098 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.817826  1285 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:56.785974 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.795848   963 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:56.796830  1041 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:56.797279  1052 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.797683  1185 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:56.800271 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 367655us
I20260709 12:26:56.800468  1175 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:26:56.787827 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 360182us
I20260709 12:26:56.701730  1087 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.820662 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:56.823026 26122 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:26:56.776571  1279 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:26:56.836400 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:56.838137 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.816390   982 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:56.833577  1175 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:56.849902  1175 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.851821 26196 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:56.852468  1310 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:56.852869  1310 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:56.853858  1310 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.855347  1310 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:56.856877 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 990) took 1258 ms. Trace:
I20260709 12:26:56.857832   938 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:56.863111 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.826787 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:56.828809 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3830) took 1388 ms. Trace:
I20260709 12:26:56.825650 26200 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:56.781322   990 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.778520  1288 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:56.825461 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 517928us
I20260709 12:26:56.867050 26121 rpcz_store.cc:276] 0709 12:26:55.440849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.526176 (+ 85327us) service_pool.cc:224] Handling call
0709 12:26:55.526529 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.543049 (+ 16520us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:56.434002 (+890953us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.828413 (+394411us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.828471 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:56.828532 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.828625 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.828749 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":758867,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3286,"raft.queue_time_us":136,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":1141917,"threads_started":2}
I20260709 12:26:56.857374 26051 rpcz_store.cc:276] 0709 12:26:55.598330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.663913 (+ 65583us) service_pool.cc:224] Handling call
0709 12:26:55.665960 (+  2047us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:55.668487 (+  2527us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:55.670265 (+  1778us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.366140 (+695875us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:56.413960 (+ 47820us) log.cc:844] Serialized 52 byte log entry
0709 12:26:56.414147 (+   187us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:56.414184 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.414251 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.856563 (+442312us) raft_consensus.cc:1685] finished
0709 12:26:56.856596 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.856801 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":326,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":43478,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":215,"thread_start_us":693491,"threads_started":2,"wal-append.queue_time_us":691385}
I20260709 12:26:56.884582 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:56.890302 26201 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:56.893852 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:56.896309 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.845116  1193 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.851887  1175 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:56.846858  1286 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:56.900323 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:56.824324  1297 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:56.911535  1309 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:56.911633 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48160.9us
W20260709 12:26:56.914876 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78553.2us
I20260709 12:26:56.920802 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1776) took 1195 ms. Trace:
I20260709 12:26:56.921233 26789 rpcz_store.cc:276] 0709 12:26:55.725881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.726056 (+   175us) service_pool.cc:224] Handling call
0709 12:26:56.920646 (+1194590us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54005708d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:56.920747 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290518912}
I20260709 12:26:56.922983 26790 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:26:56.818101   933 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.880191  1284 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:56.924618 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 388365us
I20260709 12:26:56.926944 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:56.907222   840 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.003s
I20260709 12:26:56.931643 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:56.842303  1086 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:26:56.936666 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.945397 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.960637 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:56.960888 26201 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:56.933755 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:56.964643 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:56.966498 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:56.967193 26785 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
W20260709 12:26:56.937122 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 142339us
I20260709 12:26:56.973649 26202 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
W20260709 12:26:56.978499 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 152036us
W20260709 12:26:56.978920 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52117.6us
I20260709 12:26:56.983762 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:56.912674   938 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:26:56.987495 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:56.923264  1307 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:56.985201 26202 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:56.984180 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:56.891682   990 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:56.932801  1284 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.986371 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:56.968386   963 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:56.929320  1197 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:26:56.975920  1236 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:56.982638  1233 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:26:57.021229  1275 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:57.021164  1314 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:57.029726  1180 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:57.030429  1301 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:26:57.029843  1302 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:26:57.033136 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.033602 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.035424 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:56.968561   840 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.040864 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.042872 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.043689 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.044620 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.045409 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.046175 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.052688 26168 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21.
I20260709 12:26:56.983516  1085 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:56.983139  1184 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:56.987628  1304 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:57.062639 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.024644  1297 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:56.990023  1110 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:26:57.050223  1284 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.079344 26785 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:57.078264 26201 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:57.059921  1041 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:57.106721  1275 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:57.110965  1180 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:26:57.173062 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.058748   963 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:57.196206 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.218931 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.614871 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.614850 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:26:57.615692 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.238799 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 274203us
W20260709 12:26:57.309581 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.111871 26194 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.112376 26198 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:26:57.618518 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.512445 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:57.513293 26202 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:57.060979  1194 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 12:26:57.017952  1300 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.123s
I20260709 12:26:57.516994 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:57.594696   984 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.620446 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.621131 26195 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:26:57.070943  1314 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:57.594918  1291 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:57.600018 26199 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:57.591938 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.012s	sys 0.012s
W20260709 12:26:57.629601 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.012s	sys 0.012s
W20260709 12:26:57.592408 26095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.014s
W20260709 12:26:57.630671 26095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.003s	sys 0.014s
I20260709 12:26:57.516801  1189 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:57.621912 26785 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:57.620448 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.623433 26018 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:57.585326 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.001s	sys 0.009s
W20260709 12:26:57.633948 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.002s	sys 0.009s
I20260709 12:26:57.627280 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2865) took 1274 ms. Trace:
I20260709 12:26:57.634815 26120 rpcz_store.cc:276] 0709 12:26:56.352658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.352811 (+   153us) service_pool.cc:224] Handling call
0709 12:26:56.363714 (+ 10903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.364071 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.364105 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.365050 (+   945us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.661285 (+296235us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:56.661344 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.661440 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.627048 (+965608us) raft_consensus.cc:1685] finished
0709 12:26:57.627074 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.627219 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":281,"raft.queue_time_us":2011,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2622,"thread_start_us":249562,"threads_started":1,"wal-append.queue_time_us":456196}
W20260709 12:26:57.628827 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 602354us
W20260709 12:26:57.634351 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 624581us
W20260709 12:26:57.630184 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 642706us
I20260709 12:26:57.639860 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:26:57.612828  1317 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.441s	user 0.003s	sys 0.012s
I20260709 12:26:57.645169  1085 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:57.645941  1085 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:57.646616 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:57.647575  1189 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.629778  1314 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.648187  1180 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:57.623355  1311 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.566s	user 0.000s	sys 0.011s
I20260709 12:26:57.610536  1041 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.646458  1085 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.624704 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.638424 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:57.616706  1190 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:57.666287 26194 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:57.667014 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.658576  1304 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.672804  1304 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.675069  1304 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:57.675257 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:57.676923 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.677219 26786 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:57.677933  1235 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:57.678357 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.678613  1235 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:57.679567 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.682430 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:57.660413  1297 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:57.684849  1297 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.684558   963 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.685329  1297 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.687700  1297 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:57.678781  1318 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:26:57.627660   840 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.620599  1272 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:26:57.694870 26791 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:57.695557  1175 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.773s	user 0.024s	sys 0.006s
I20260709 12:26:57.696171 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.697021 26791 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:57.655059  1291 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.698028  1189 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.699795  1291 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.696344  1175 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.637976   984 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.700605  1175 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.700811  1085 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:57.702651   984 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.659180  1041 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:57.704347  1285 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.878s	user 0.009s	sys 0.014s
I20260709 12:26:57.704948  1285 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:57.705084  1175 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.705610  1175 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:26:57.695204  1190 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.706037  1175 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.708793  1190 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.677958   981 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:26:57.709484  1283 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.905s	user 0.011s	sys 0.009s
I20260709 12:26:57.709136  1314 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.709825   981 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:57.710099 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:57.709934  1283 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.710927   981 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.711902 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:57.710933 26790 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:57.712790 26203 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:26:57.689427 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.700534   990 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.679679  1235 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.718405 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.719195 26789 raft_consensus.cc:2368] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 18. Current term is 19.
I20260709 12:26:57.720162 26788 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:57.721405  1184 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:57.722118  1184 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.722595  1184 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.684600  1275 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.727169 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:57.729763 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 217311us
I20260709 12:26:57.729979 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:26:57.730976 26785 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:26:57.736963 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.739603 26785 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:57.733944 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.734077 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.735322 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:57.742750 26194 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:57.720064   990 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.740711 26790 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:26:57.746134 26195 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:26:57.746663 26195 pending_rounds.cc:85] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 13
I20260709 12:26:57.747207 26195 pending_rounds.cc:107] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:26:57.747664 26195 raft_consensus.cc:2889] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.708204  1175 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:57.737742  1277 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.772096 26791 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:57.774690 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.777760 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.778730 26791 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:57.780684  1184 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:57.782315  1288 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.783803  1288 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.784205 26195 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:57.784423  1277 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.787009  1277 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:57.770866   990 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.789937   990 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:57.775588 26790 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:57.794195 26194 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:26:57.796923   941 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:57.797508   941 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:57.798349  1041 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:57.802713 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 690640us
I20260709 12:26:57.810309 26788 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:57.807199 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:57.813207 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:57.814911  1310 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.959s	user 0.022s	sys 0.002s
I20260709 12:26:57.816336 26202 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:57.823411  1314 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.824384 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:57.825730 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.826504 26788 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:57.827760 26788 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:26:57.828321   990 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:57.828837   990 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.829345   990 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:57.837558  1310 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:57.846164 26194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:26:57.846870 26194 raft_consensus.cc:606] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:57.847239 26194 raft_consensus.cc:614] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:57.811103  1283 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:57.850713  1309 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.939s	user 0.015s	sys 0.005s
I20260709 12:26:57.864290 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.876006 26785 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:57.892685 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.892927 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.893424 26123 raft_consensus.cc:1217] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 13.11->[14.12-22.16]   Dedup: 16.13->[18.14-22.16]
I20260709 12:26:57.893556 26127 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:57.893968 26123 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [14.12-22.16]
I20260709 12:26:57.892724 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:57.865274  1290 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.047s	user 0.012s	sys 0.010s
W20260709 12:26:57.876156  1019 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.397s	user 0.025s	sys 0.002s
W20260709 12:26:57.876363  1289 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.124s	user 0.018s	sys 0.006s
I20260709 12:26:57.900996 26127 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:57.905107 26196 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:26:57.850108   941 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.907234 26785 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:26:57.911199 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 235997us
I20260709 12:26:57.875196  1314 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.912243 26120 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:57.911618 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.914698 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.914131 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:57.918015 26129 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:57.917460 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:57.913780 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:57.897107  1309 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:57.929520 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.931763 26050 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:26:57.933488 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:57.934413 26047 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:57.939061 26047 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:57.941907 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:57.942420 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 295858us
I20260709 12:26:57.943300 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:57.944068 26125 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:57.944533 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:57.945441 26045 raft_consensus.cc:2368] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
I20260709 12:26:57.927374  1041 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:57.947206 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.948354 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.931617  1284 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.948839 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:57.949194 26120 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 25 because replica has last-logged OpId of term: 23 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:26:57.950910  1284 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:57.952946 26200 raft_consensus.cc:2368] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:26:57.956328 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.957137 26195 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:26:57.962117 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:57.966601 26198 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:57.966468 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:57.967437 26050 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:57.919970   963 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.969551 26194 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:57.969601 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.970144 26194 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:26:57.961463 26756 connection.cc:256] RPC call timeout handler was delayed by -1.1134054849999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:57.981700 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:57.982807 26120 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:57.909585  1019 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:57.988700 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:57.990263 26199 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 17.
W20260709 12:26:57.989538 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 103524us
I20260709 12:26:57.989409 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:57.931103  1314 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:57.966440   984 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.010121 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.003235 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:57.988485 26194 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.017681 26129 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:26:58.022964 26018 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.505s (SENT)
I20260709 12:26:58.018455 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:57.970153  1312 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:58.032930 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.033950 26127 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:57.966344  1333 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:57.971045  1335 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.019335 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.017813  1289 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.047199 26197 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:58.025849   941 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:58.026696 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:58.027726 26194 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:58.028694 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:58.036316  1180 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:58.030987 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:26:58.038229 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.029515 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:58.041786 26198 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:26:57.988884  1189 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.062253 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.044435 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:58.049817 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 106601us
I20260709 12:26:58.062937 26788 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:58.036823 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.058995 26201 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:26:58.055811 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:57.983431   963 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.061796 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.030258  1197 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:58.070921 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.073560 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:58.074210 26129 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.077386 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:58.047212  1312 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:58.083422 25946 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 11 to 17, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 17 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:26:58.084941 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62008.9us
I20260709 12:26:58.086418  1019 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:58.086577  1327 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.054821  1290 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:58.093452 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2455) took 1157 ms (client timeout 1519 ms). Trace:
W20260709 12:26:58.106485 26196 rpcz_store.cc:269] 0709 12:26:56.936209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.936387 (+   178us) service_pool.cc:224] Handling call
0709 12:26:58.093206 (+1156819us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:58.093387 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170928512}
W20260709 12:26:58.107381 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.095515 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:58.031059   984 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.070122  1275 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:58.069150  1272 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:58.112893 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.081823   941 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:58.121932 26788 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:26:58.075664   840 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:58.133121 26129 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:58.061872  1283 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:26:58.134613 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:26:58.136842 26120 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:26:58.050282  1284 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:26:58.139424 26129 raft_consensus.cc:606] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:58.103020  1285 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:58.047417  1333 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.104133  1332 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.069342  1189 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.106279  1290 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.072705  1300 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:26:58.112028  1175 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:58.081272  1184 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:26:58.080561  1197 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.145648  1019 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:58.049993  1335 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.127699  1275 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.128995  1272 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:58.179747 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71362.4us
W20260709 12:26:58.145985 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 220044us
I20260709 12:26:58.160071 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:58.105624   990 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:58.172253 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 123152us
I20260709 12:26:58.184900 26168 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21.
I20260709 12:26:58.185341 26168 leader_election.cc:403] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:26:58.173983 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118265us
W20260709 12:26:58.187218 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.182940  1272 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.166174  1332 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.162634  1290 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.181982  1312 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.202634  1314 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.203718  1332 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.205116   963 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:58.206827  1340 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:58.209702 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.181877 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.157338   941 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:58.109331  1327 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.143460   840 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:26:58.221927 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.225557 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.226583 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:58.234359 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2459) took 1201 ms. Trace:
I20260709 12:26:58.234843 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:58.240182 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55509.2us
I20260709 12:26:58.239166 26203 rpcz_store.cc:276] 0709 12:26:57.032823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.043439 (+ 10616us) service_pool.cc:224] Handling call
0709 12:26:58.208385 (+1164946us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b5400301e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:58.234258 (+ 25873us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1381824768}
I20260709 12:26:58.181092  1275 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:58.244441 26756 leader_election.cc:400] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
W20260709 12:26:58.244570 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.247452 26756 leader_election.cc:403] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:26:58.244442 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 84490.7us
I20260709 12:26:58.244980  1314 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:58.246088 26129 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:58.253407 26121 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:26:58.239519  1355 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:26:58.261920 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:58.247025  1332 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21.
I20260709 12:26:58.249087  1304 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:58.253556  1110 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
W20260709 12:26:58.284272 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 57748.2us
I20260709 12:26:58.290102 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.295269 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:58.297664 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.298062  1360 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:58.249213  1180 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
W20260709 12:26:58.267251  1351 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:26:58.297664 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.303992 26784 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:26:58.307638 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.311015 26788 raft_consensus.cc:2376] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23. Re-sending same reply.
I20260709 12:26:58.313999 26194 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:58.312968 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:58.321662 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:58.300743  1357 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.258343  1324 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:26:58.281019  1304 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:26:58.330360 26094 connection.cc:256] RPC call timeout handler was delayed by -1.8000864229998115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 12:26:58.334158 26194 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:58.304785   963 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:58.336494 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:58.334084 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7685864230002153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
W20260709 12:26:58.342964 26094 connection.cc:256] RPC call timeout handler was delayed by -1.7370864230001644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 12:26:58.344170 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6983854069999325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 12:26:58.353818 26094 connection.cc:256] RPC call timeout handler was delayed by -1.683086423000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 12:26:58.359392 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6695854069998859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 12:26:58.368482 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.376495 26786 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:58.333211  1304 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:58.298772  1360 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.379513 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1598) took 1388 ms. Trace:
I20260709 12:26:58.382408 26783 rpcz_store.cc:276] 0709 12:26:56.990756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.026103 (+ 35347us) service_pool.cc:224] Handling call
0709 12:26:58.379240 (+1353137us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b540057a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:58.379424 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093661568}
I20260709 12:26:58.338371  1370 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:58.363246  1333 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:58.380990  1288 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.007s
I20260709 12:26:58.389971 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1600) took 1312 ms. Trace:
I20260709 12:26:58.311992   941 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.367095   963 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:58.403187 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.406273 26783 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:58.407583 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94714.8us
I20260709 12:26:58.410948 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.411090   963 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.411532 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.413023 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:58.400974  1356 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:26:58.394093  1277 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:58.376495  1361 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:58.390060  1370 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.387161   984 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:58.394973  1180 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:58.409061 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.400548  1357 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.424968 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.407280  1110 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:58.428895 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.429515 26046 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:58.428982 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.428249 26786 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:58.407444  1197 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.414145  1189 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:58.424114  1343 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.233s	user 0.000s	sys 0.002s
I20260709 12:26:58.421972  1330 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.442145 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 192010us
I20260709 12:26:58.443181 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:58.435343  1341 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.183s	user 0.001s	sys 0.001s
I20260709 12:26:58.425109  1370 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:58.429594  1180 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.427416  1368 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.435370  1110 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.422338  1333 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:26:58.437956  1335 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:58.452623 26049 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:58.435442   984 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:58.457391 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:58.471328 26124 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:26:58.455019 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 133419us
I20260709 12:26:58.442131   840 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:58.453308 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.460203  1365 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.473340 26786 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:58.462378 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.463394  1040 raft_consensus.cc:993] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:26:58.463012  1366 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 12:26:58.461175  1110 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.445263  1189 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.476807 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113254us
I20260709 12:26:58.456664  1180 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:58.455010  1327 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:58.497237 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.489971 26053 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:58.497954 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:26:58.496551 26094 leader_election.cc:336] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.540s (SENT)
I20260709 12:26:58.503762 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:58.496161  1197 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:26:58.501715 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:58.504885 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.436007  1357 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.528399 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:58.507045 26786 raft_consensus.cc:2376] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22. Re-sending same reply.
I20260709 12:26:58.529060 26049 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:58.535789 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.536888 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.483991  1333 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:26:58.447621  1330 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.501168  1347 raft_consensus.cc:1081] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
W20260709 12:26:58.540366 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.507570  1361 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:58.460852  1368 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.505721   984 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:58.527837  1288 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.527567  1375 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.550119 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:58.557840 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67984.1us
I20260709 12:26:58.501145  1377 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:58.584856 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6506854069998553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 12:26:58.560539  1283 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:58.561515  1376 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:58.539907  1327 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:58.496338  1189 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.483646  1365 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.540117   840 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:58.601452 26051 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:58.574456  1285 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.575471 26047 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:26:58.576442 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.509928  1290 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:58.594046 26094 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.605s (SENT)
W20260709 12:26:58.595914  1369 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:26:58.647397 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.573637 26049 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:58.538884  1354 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 12:26:58.653172 26054 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:58.653324 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:58.653939 26783 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:58.661994 26786 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:58.685082 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.663918 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:58.573871  1361 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:26:58.666448 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.669262  1357 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:58.671011  1365 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:58.671097  1290 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.671808  1283 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.698266 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 111951us
I20260709 12:26:58.587607  1275 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:58.698838 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6398864230000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 12:26:58.699757 26094 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.704s (SENT)
I20260709 12:26:58.599054  1375 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.599180  1288 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.679559  1370 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.646299   840 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:58.695068 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.600523  1272 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.683797  1110 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.688537  1194 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.698089  1283 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.688279 26049 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:58.713569 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.717466  1385 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:58.701388 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.684178  1368 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.700145 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:58.700369  1275 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:58.712422  1319 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.606s	user 0.008s	sys 0.007s
I20260709 12:26:58.792886 26791 raft_consensus.cc:2368] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:26:58.738500   984 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.667099  1335 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:58.749207 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.666247  1197 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.745101 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:58.705229  1371 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:26:58.700842 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.863867  1175 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:58.693900  1361 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.865525 26786 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:58.867861 26049 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:26:58.784471 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:58.791155  1312 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:58.795225 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 244337us
I20260709 12:26:58.795853 26050 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:26:58.701635  1019 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 12:26:58.712220  1304 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.844111  1330 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:58.840590  1110 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:58.842949  1370 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:26:58.831732 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 132024us
I20260709 12:26:58.841322  1375 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.829803  1335 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:58.841028  1288 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:26:58.793764  1221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.013s	sys 0.036s
I20260709 12:26:58.794569   840 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.664695  1285 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:58.889278  1285 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:58.839121  1376 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:58.872414 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:58.890097 26203 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:58.891372 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:58.892661 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:58.893213 26203 raft_consensus.cc:2376] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22. Re-sending same reply.
I20260709 12:26:58.894042 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:58.894618 26203 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:26:58.894948  1304 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:58.895406 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:58.881088 26786 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:58.897107 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.898195 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.898736 26786 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:26:58.899507 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.900053 26786 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:58.900935 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.902024 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:58.904611   840 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:58.904703 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:58.905318 26203 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:58.906116 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:58.906944 26194 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
W20260709 12:26:58.907388  1390 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:58.881330 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:58.883073 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:58.884675 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:58.398258 26787 rpcz_store.cc:276] 0709 12:26:57.077485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.077917 (+   432us) service_pool.cc:224] Handling call
0709 12:26:57.588604 (+510687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:57.609434 (+ 20830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.609465 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:57.636867 (+ 27402us) log.cc:844] Serialized 26 byte log entry
0709 12:26:57.920388 (+283521us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:58.301623 (+381235us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.389443 (+ 87820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.389670 (+   227us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.389744 (+    74us) raft_consensus.cc:1685] finished
0709 12:26:58.389764 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.389912 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":245921,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":92618,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":11478,"thread_start_us":618177,"threads_started":3,"wal-append.queue_time_us":274002}
I20260709 12:26:58.921969 26203 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:26:58.944653 26788 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 12:26:58.973205 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91930us
W20260709 12:26:59.063993 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2476) took 1987 ms (client timeout 1697 ms). Trace:
I20260709 12:26:59.045433 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:58.979167 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.047317 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.066175 26203 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:59.064589 26198 rpcz_store.cc:269] 0709 12:26:57.076435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.618255 (+541820us) service_pool.cc:224] Handling call
0709 12:26:59.063823 (+1445568us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.063925 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799096064}
I20260709 12:26:59.062897 26127 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:26:58.865053 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 120063us
W20260709 12:26:59.064898 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2468) took 2029 ms (client timeout 1626 ms). Trace:
I20260709 12:26:59.067108 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:26:58.979007 26094 leader_election.cc:336] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.779s (SENT)
I20260709 12:26:59.066406 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1330) took 1175 ms. Trace:
I20260709 12:26:59.069321 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.069520 26121 rpcz_store.cc:276] 0709 12:26:57.890556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.890671 (+   115us) service_pool.cc:224] Handling call
0709 12:26:59.066223 (+1175552us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54001d1fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.066341 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765268224}
I20260709 12:26:59.068684 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.069335 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:58.930227 26129 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 19. (term mismatch)
I20260709 12:26:59.070961 26788 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:26:59.071557 26129 pending_rounds.cc:85] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 18
I20260709 12:26:59.071936 26129 pending_rounds.cc:107] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 23.19
I20260709 12:26:58.927948 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.072270 26129 raft_consensus.cc:2889] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.072649 26047 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:58.928454 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:58.920750 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.073944 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:58.909224 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.074401 26203 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:26:59.074779 26195 raft_consensus.cc:2376] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21. Re-sending same reply.
I20260709 12:26:59.074908 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:58.908113 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:58.926350 26124 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:58.917711 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.077423 26196 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
W20260709 12:26:58.911291 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2463) took 1877 ms (client timeout 2177 ms). Trace:
W20260709 12:26:59.068107 26201 rpcz_store.cc:269] 0709 12:26:57.035738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.610472 (+574734us) service_pool.cc:224] Handling call
0709 12:26:59.064709 (+1454237us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.064834 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678663040}
I20260709 12:26:59.076105 26194 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:59.084052 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.085328 26053 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:59.085505 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:26:59.079180 26197 rpcz_store.cc:269] 0709 12:26:57.034203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.045847 (+ 11644us) service_pool.cc:224] Handling call
0709 12:26:58.911004 (+1865157us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400396554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:58.911207 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016402816}
I20260709 12:26:59.087663 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.088482 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.088964 26195 raft_consensus.cc:2376] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21. Re-sending same reply.
I20260709 12:26:59.084860 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:26:59.090229 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.090126 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.091501 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:58.911827  1189 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.092079 26203 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 16.
W20260709 12:26:58.908373  1221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.014s	sys 0.037s
I20260709 12:26:59.091516 26790 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:59.093835 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.094331 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.094988 26052 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:59.093730 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:59.096048 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.097767  1384 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:59.105137 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.105623  1384 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.105698 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.106493 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.107116 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.110476 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:59.098151 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.101239 26126 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.101316 26198 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 22 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 8.
I20260709 12:26:59.107920 26201 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:26:58.921916  1367 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.929416  1374 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:58.958204  1378 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:58.909574  1283 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.084052  1297 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:59.106042  1384 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.144584 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.145444 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:59.096107 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.147567 26194 raft_consensus.cc:2376] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23. Re-sending same reply.
I20260709 12:26:59.031960  1376 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.915814   933 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.114143 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.152639 26197 raft_consensus.cc:2376] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16. Re-sending same reply.
I20260709 12:26:59.115507 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.107919 26195 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:59.154949 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.155741 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:26:59.097941 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:26:59.156361 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.120177 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.098397  1316 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:59.120632 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.120419 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:58.980526  1370 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:58.929539  1310 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:58.917091  1272 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.103225   981 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:26:59.130514  1189 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:59.103368   954 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:26:59.084400  1314 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:59.083913  1375 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:59.136370 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.140471 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.184659 26046 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
W20260709 12:26:59.141353 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 264302us
I20260709 12:26:59.150799 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:26:59.185545 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5966674740002418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 12:26:59.185776 26045 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:26:59.172039 26196 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:59.186713 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.187206 26045 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:26:59.187611 26196 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:26:59.187844  1285 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:59.111372  1326 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:59.131857  1019 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:59.188745 26196 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:59.174088 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.189381 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.188829  1019 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.190708  1175 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:59.190665  1019 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.191401  1175 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.191951  1175 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:59.160567 26050 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:59.172911  1370 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.188284  1285 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.189580 26047 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 21 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:59.177682 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.202679 26197 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.190953 26196 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
W20260709 12:26:59.185927 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5561674740001763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 12:26:59.203081 26201 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:59.203464 26196 pending_rounds.cc:85] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 7
W20260709 12:26:59.203552 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5435864229998515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2315s for a response
I20260709 12:26:59.203896 26196 pending_rounds.cc:107] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 15.8
W20260709 12:26:59.203994 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5093674740001006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
I20260709 12:26:59.179666 26198 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:59.204216 26196 raft_consensus.cc:2889] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:59.204387 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5079387709997718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32380000000000003s for a response
W20260709 12:26:59.204707 26094 connection.cc:256] RPC call timeout handler was delayed by -1.4436228649997247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 12:26:59.205032 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.206465 26198 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:59.173058  1310 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.211705 26052 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:59.209461  1326 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:26:59.211781  1180 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.006s
I20260709 12:26:59.142393  1288 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:26:59.215366 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 79127.6us
I20260709 12:26:59.218230 26201 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:59.218431 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.142251  1378 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.211182 26198 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:26:59.220952 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.221597 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3356674740002745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
I20260709 12:26:59.221581 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.171602  1376 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.231734  1376 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.208994  1370 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.241933  1326 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.214707  1314 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:59.228452  1189 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:59.245843 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:59.222229 26094 connection.cc:256] RPC call timeout handler was delayed by -1.321942861000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
I20260709 12:26:59.210629  1285 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.250799 26201 pending_rounds.cc:85] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 12
W20260709 12:26:59.252362 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2933854069997324s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 12:26:59.253275 26094 connection.cc:256] RPC call timeout handler was delayed by -1.142186423000112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2761s for a response
W20260709 12:26:59.253649 26094 connection.cc:256] RPC call timeout handler was delayed by -1.098410819000037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
W20260709 12:26:59.253849 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0949375610002789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:26:59.254012 26094 connection.cc:256] RPC call timeout handler was delayed by -1.081074746000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 12:26:59.254164 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0612178410001434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
I20260709 12:26:59.142354  1367 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.254941 26201 pending_rounds.cc:107] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:26:59.251714  1375 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:59.255707  1312 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.256179 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.256780  1189 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:59.222600   840 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.258136 26197 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:59.258687 26201 raft_consensus.cc:2889] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.143910  1374 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.212894  1290 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:26:59.227310  1361 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.276048  1297 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:59.244902  1323 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.275970 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.213939  1180 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.265682  1375 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.272953  1189 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.288580 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:26:59.288707 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.292594 26787 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:59.283787  1297 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.295470  1052 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.219669   941 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:26:59.291183  1189 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.291306  1375 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.278234  1312 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.292694  1221 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:59.310566 26126 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:59.300447  1297 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:59.287777  1323 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.320487 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.338023  1378 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.338689  1372 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:59.339241  1372 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.344260  1384 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.345538  1312 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:26:59.346401  1398 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:59.346796  1384 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.346922 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.350122  1372 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.352095 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.354070 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.355675  1285 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.355947 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.358409  1310 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.359607  1285 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.359848  1310 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.358470  1189 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:59.361784 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.306825   954 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:26:59.315838  1221 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:59.381134  1320 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.779s	user 0.005s	sys 0.000s
I20260709 12:26:59.312469  1314 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.371896  1312 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.391034 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:59.391914 26123 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:59.393847   990 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.397938 26197 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
W20260709 12:26:59.401191 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 85020.2us
I20260709 12:26:59.404275  1361 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.406266  1283 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:59.406311  1367 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.406939  1283 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.407685  1283 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.408840  1283 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.415594 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.410123 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.416385 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:59.416534 26126 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:26:59.418223  1323 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.418519  1180 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.418251 26785 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:26:59.419370  1086 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.420470   840 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.421057   840 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.422698 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:26:59.423040 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2475) took 2346 ms (client timeout 1943 ms). Trace:
W20260709 12:26:59.423452 26202 rpcz_store.cc:269] 0709 12:26:57.076173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.665529 (+589356us) service_pool.cc:224] Handling call
0709 12:26:59.422849 (+1757320us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400302354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.422973 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339210624}
I20260709 12:26:59.419845  1086 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.424136 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.424860 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.424177   840 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.426864  1052 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:59.427460  1052 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:59.427878  1370 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.428627  1052 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.440769  1052 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.440971 26046 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:26:59.429806  1361 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:59.442370  1361 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:59.442762  1361 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.430387 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.443632  1361 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.444983  1361 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:59.442052  1175 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.444923 26202 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:26:59.446455 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.433981 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.434762 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.435456 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.435770  1221 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.436141 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.437508 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.448446 26203 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:59.449020 26197 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:26:59.448295 26201 raft_consensus.cc:2376] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:26:59.453369 26123 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 21 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:59.453960 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:59.459931 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:59.461546  1285 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:59.452418  1370 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.462869  1175 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.457907  1283 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:59.471729 26791 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1250) took 1545 ms (client timeout 1539 ms). Trace:
I20260709 12:26:59.474406 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.476337 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:26:59.475503 26791 rpcz_store.cc:269] 0709 12:26:57.925974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.046873 (+120899us) service_pool.cc:224] Handling call
0709 12:26:59.471390 (+1424517us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.471625 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1655802240}
W20260709 12:26:59.480079 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77865.3us
I20260709 12:26:59.481035  1019 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:59.481484  1019 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.481828  1019 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:59.482800  1374 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.483783  1347 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:59.484189  1347 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:26:59.479825  1290 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:59.484886  1290 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.485252  1290 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:59.485142  1347 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.486487  1347 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.489171  1086 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:26:59.478873  1283 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.496781  1283 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.478335  1310 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:59.497434  1310 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.498394  1310 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.498847 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.499346  1335 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.501333 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1249) took 1592 ms. Trace:
I20260709 12:26:59.501724 26785 rpcz_store.cc:276] 0709 12:26:57.909282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.929411 (+ 20129us) service_pool.cc:224] Handling call
0709 12:26:59.501170 (+1571759us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.501273 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3377763584}
I20260709 12:26:59.502107 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.502769 26201 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:26:59.496824 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.503799 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.505206 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.505508 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.506064 26121 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:26:59.506389 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.506942 26203 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.507486 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1608) took 1833 ms. Trace:
I20260709 12:26:59.499436 26126 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:59.509042 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:26:59.507943 26789 rpcz_store.cc:276] 0709 12:26:57.673677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.727170 (+ 53493us) service_pool.cc:224] Handling call
0709 12:26:57.738559 (+ 11389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.773921 (+ 35362us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:57.988363 (+214442us) log.cc:844] Serialized 14 byte log entry
0709 12:26:59.398313 (+1409950us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b1800230b60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:26:59.398721 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 12:26:59.507045 (+108324us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:59.507095 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:59.507139 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.507330 (+   191us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.507434 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":337527,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3773,"raft.queue_time_us":83,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":4471168,"thread_start_us":341703,"threads_started":3,"wal-append.queue_time_us":155246,"wal-append.run_cpu_time_us":3793,"wal-append.run_wall_time_us":1032128}
I20260709 12:26:59.509922 26123 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:59.510152  1272 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:59.515659  1272 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.517273  1087 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.522274 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:26:59.522769  1373 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.042s	user 0.012s	sys 0.000s
I20260709 12:26:59.522976 26126 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:26:59.524106 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1227) took 1226 ms. Trace:
I20260709 12:26:59.524510 26199 rpcz_store.cc:276] 0709 12:26:58.297625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.299033 (+  1408us) service_pool.cc:224] Handling call
0709 12:26:58.316640 (+ 17607us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:58.325344 (+  8704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.325377 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:58.325980 (+   603us) log.cc:844] Serialized 52 byte log entry
0709 12:26:58.376968 (+ 50988us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:58.377015 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.377087 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.522429 (+1145342us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b240020bb98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:26:59.523782 (+  1353us) raft_consensus.cc:1685] finished
0709 12:26:59.523802 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.524013 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8159,"raft.queue_time_us":15010,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7831,"spinlock_wait_cycles":26986240,"thread_start_us":50566,"threads_started":1,"wal-append.queue_time_us":132466}
I20260709 12:26:59.530064 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:26:59.531527 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.532804 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.538748 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.539451 26047 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:59.529909 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.540845 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.541668 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.519086  1272 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.546860 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:26:59.552872 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:26:59.554395 26053 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:59.556298 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:59.556104 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:59.556455  1235 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.557616  1283 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:59.558647 26052 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:26:59.554001  1370 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.554167  1335 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.564373 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.566809 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.546609  1382 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:59.568712  1272 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:59.569489  1370 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:59.570554  1221 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:59.570883  1376 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:59.572453  1376 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:59.572907 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.573473 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2457) took 1092 ms. Trace:
I20260709 12:26:59.574141  1376 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:59.573894  1386 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:59.574807  1332 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:59.575217  1332 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.575803 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.561535  1335 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.576136  1332 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.572387  1297 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.577144  1360 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.577390 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.577731  1360 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:59.578030  1335 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.578521  1360 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.578882  1366 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.573851 26051 rpcz_store.cc:276] 0709 12:26:58.481391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.481662 (+   271us) service_pool.cc:224] Handling call
0709 12:26:58.497999 (+ 16337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.643133 (+145134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.573219 (+930086us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400173254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:59.573408 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232711424}
I20260709 12:26:59.566455   840 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:59.582401   840 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:59.583384   840 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.566058  1314 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.585444  1283 raft_consensus.cc:2764] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:59.593101  1283 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.587345 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.593837 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.589272 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:59.595755 26198 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:26:59.591923 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.596190 26198 pending_rounds.cc:85] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 14
I20260709 12:26:59.596041 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.596844 26198 pending_rounds.cc:107] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:26:59.597131 26198 raft_consensus.cc:2889] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.604647 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:59.607349 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.608098 26195 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:59.608786 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:59.609179  1272 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:59.609633  1180 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:59.609751  1272 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:59.610118  1180 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.610183  1361 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:59.610625  1180 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:59.610994  1361 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:59.585948  1086 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:26:59.612741  1284 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:59.567963   941 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:26:59.613199  1284 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:26:59.614126  1284 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.615377  1284 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:59.616887 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:26:59.617676  1374 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:59.618075  1374 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:26:59.618494  1374 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:59.624486 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 109532us
I20260709 12:26:59.625396 26094 leader_election.cc:400] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:59.625691 26094 leader_election.cc:403] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:59.627009 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.627142  1052 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20.
I20260709 12:26:59.627756  1235 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:59.634248 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.610759  1272 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:26:59.660846 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 63588.8us
I20260709 12:26:59.661643 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.635319 26788 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:59.643890 26047 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:26:59.645752 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58681.2us
W20260709 12:26:59.646368 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 85329.9us
I20260709 12:26:59.668141 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.672062 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.673574 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.675047 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.578974  1297 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:26:59.711259 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.712486 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.695727 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.715731 26198 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:59.710992 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:59.717432 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 144594us
I20260709 12:26:59.718436 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.719895 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.720765 26203 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:26:59.717563 26785 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:26:59.722747 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:26:59.722894 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.723379 26048 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:26:59.727478 26791 raft_consensus.cc:2368] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:59.718175 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.731302 26786 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:26:59.731899 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.734812 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.734961 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.735421 26784 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.736155 26788 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 21 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:59.736542 26787 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:26:59.631072  1376 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:59.746296 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.739209 26791 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:59.739933 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.729763  1236 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.750903 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.710901  1175 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:59.743063  1310 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:59.758314 26200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2460) took 2725 ms (client timeout 3367 ms). Trace:
I20260709 12:26:59.665470   933 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 12:26:59.778415 26123 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:26:59.772919 26785 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 12:26:59.779269 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 114949us
W20260709 12:26:59.780161 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 114475us
I20260709 12:26:59.780972 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.658532  1290 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.783452 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:59.779158 26200 rpcz_store.cc:269] 0709 12:26:57.033186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.033311 (+   125us) service_pool.cc:224] Handling call
0709 12:26:59.755219 (+2721908us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b5400302d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:26:59.758242 (+  3023us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943281024}
I20260709 12:26:59.693457  1285 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:26:59.785207 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.785302 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:59.790696 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.758490  1297 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:26:59.788508 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77176.9us
I20260709 12:26:59.790040 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.798741 26784 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:26:59.779515  1221 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.834688 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.736366  1375 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.806291  1303 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:26:59.808683  1404 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.476s	user 0.000s	sys 0.004s
I20260709 12:26:59.815636  1332 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.765336  1236 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.816776  1376 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.782766  1314 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:59.820355  1180 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.824369 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.825266  1420 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:59.827821 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.830062 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:26:59.871768 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47584.8us
I20260709 12:26:59.871913 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.831606  1189 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:26:59.799609 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.833648 26199 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:26:59.834000 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:26:59.841027 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51066.3us
I20260709 12:26:59.874228 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.846938 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:26:59.850261 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:26:59.789187  1361 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.754535  1374 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.855222 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:26:59.875564 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.800891  1297 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.860033 26786 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:26:59.866771 26049 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 17: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:26:59.868671 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 83443.5us
I20260709 12:26:59.881829 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.834537  1413 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:59.876348 26784 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:26:59.785828  1310 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:26:59.872535 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.792879  1304 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:59.882649 26123 raft_consensus.cc:2368] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:26:59.884198 26788 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:26:59.887478 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87938.9us
I20260709 12:26:59.901230 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:26:59.902161 26197 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:26:59.888753 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:26:59.910552 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.910262 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.860452  1285 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:59.916277 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:26:59.917541 26126 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 12:26:59.917973 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46121.3us
I20260709 12:26:59.919072 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:26:59.919179 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.919946 26202 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:59.923292 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:26:59.933378 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.867168  1314 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:59.884516  1303 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.944416 26128 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:59.891366  1297 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:59.851353  1221 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:26:59.910243  1335 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:59.909811  1190 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:59.909714  1300 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:59.860638  1175 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:26:59.924347   990 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:59.917924  1197 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:59.998203 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3752) took 1355 ms. Trace:
I20260709 12:27:00.003824 26202 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:00.006470 26050 rpcz_store.cc:276] 0709 12:26:58.643016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.092709 (+449693us) service_pool.cc:224] Handling call
0709 12:26:59.093142 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.211274 (+118132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.997954 (+786680us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b5400227754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:26:59.998142 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154402176}
W20260709 12:27:00.012351 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 216851us
I20260709 12:27:00.013633 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.931627  1420 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.930605  1304 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:26:59.939471  1314 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:59.943065   840 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:00.028932 26787 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:26:59.892849  1361 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:00.026324 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 93019us
I20260709 12:27:00.039009 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:26:59.937565  1413 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:26:59.988997  1268 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:00.041162 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.992486  1370 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:59.992775  1386 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:59.989632  1288 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:00.007783  1366 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:26:59.994562  1335 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:00.018613  1407 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:27:00.053802 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:00.048691 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 176119us
I20260709 12:27:00.010437  1197 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:00.025529  1052 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:00.068603 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.069307 26126 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
W20260709 12:27:00.105118  1414 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:27:00.059410  1197 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:00.035748   990 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:00.042496  1421 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:00.043221  1300 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:00.049656  1370 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.125211 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:00.059015  1335 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:00.126603 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 237968us
I20260709 12:27:00.051745  1386 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:00.061523  1384 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:00.135246 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:00.139539 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81762us
I20260709 12:27:00.147382 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.127713  1421 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.058887  1366 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.133325  1370 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:00.144145  1019 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.001s
I20260709 12:27:00.140985  1386 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.134727  1425 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:00.153291  1367 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:00.163123  1347 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.000s
I20260709 12:27:00.161927 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:00.151322 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:00.159019 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:00.154078  1421 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.188830 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:00.187548 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.190940 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:00.192245 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:00.174850  1087 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.000s
I20260709 12:27:00.168285  1425 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:00.239173 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 200295us
W20260709 12:27:00.239902 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.240165 26018 leader_election.cc:400] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:27:00.242933 26782 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:27:00.246172 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55301.6us
I20260709 12:27:00.246920 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:00.190374  1268 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:00.187563  1310 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:00.193411  1236 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:00.148361  1304 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.249431 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:00.250065 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.191567  1290 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:00.193084  1052 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:00.382184 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:00.382795 26128 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.255235 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:00.245784  1235 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:00.169412  1019 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:00.324046 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77192.9us
I20260709 12:27:00.166649  1413 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.312608 25946 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 17 to 19, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 19 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.243353  1275 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.338902  1412 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.414s	user 0.001s	sys 0.011s
I20260709 12:27:00.344602 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:00.338837  1326 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:00.190717   990 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.235109  1085 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 12:27:00.356956  1366 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:00.360860 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:00.395414  1426 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:27:00.365037 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.368682 26120 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:00.446671 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.447353 26120 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:00.247912  1300 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.239616  1277 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:00.372359 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:00.458338 26789 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.235065  1303 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:00.379046 26020 leader_election.cc:400] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:27:00.240644  1374 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:00.373044   941 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:00.502746   941 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:00.503362 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.503743   941 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.413265 26126 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.504663 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:00.505065  1370 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.507172  1370 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:00.507721  1427 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.403s	user 0.005s	sys 0.000s
I20260709 12:27:00.413990 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:00.416759  1275 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:00.353183 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 113146us
I20260709 12:27:00.513525 26018 leader_election.cc:400] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18.
I20260709 12:27:00.514422 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:00.386972  1431 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:27:00.412631  1236 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:00.361388 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:00.426640 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1231) took 1513 ms. Trace:
I20260709 12:27:00.515273  1236 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.516546  1451 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:00.517786 26126 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:00.518054  1451 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:00.519474  1451 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:00.520421  1275 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:00.521394 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:00.522454  1236 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:00.522701 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:00.516144 26783 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:00.416857  1430 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.427191 26782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1610) took 2753 ms (client timeout 1819 ms). Trace:
I20260709 12:27:00.418200  1361 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:00.441665 25946 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 20, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.422472 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:00.527422 26782 rpcz_store.cc:269] 0709 12:26:57.674206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.679296 (+  5090us) service_pool.cc:224] Handling call
0709 12:27:00.426949 (+2747653us) spinlock_profiling.cc:243] Waited 2.56 s on lock 0x7b54004ba854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:00.427100 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5888970112}
I20260709 12:27:00.442867  1375 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:00.445168 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.452570 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.389770  1235 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:00.545571 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:00.413201  1439 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:27:00.458070  1277 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.459861  1435 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.497414 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.501168 26128 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:00.485878  1303 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:00.505507 26121 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:00.567179 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:00.391654  1332 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.005s
I20260709 12:27:00.528378 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:00.515861 26196 rpcz_store.cc:276] 0709 12:26:58.913017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.212908 (+299891us) service_pool.cc:224] Handling call
0709 12:26:59.215266 (+  2358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.215898 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.215935 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.222759 (+  6824us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.006634 (+783875us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b1800482f80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:27:00.159586 (+152952us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:00.159636 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.177717 (+ 18081us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.426320 (+248603us) raft_consensus.cc:1685] finished
0709 12:27:00.426352 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.426576 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1333,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":1929,"spinlock_wait_cycles":32506752,"thread_start_us":152750,"threads_started":1,"wal-append.queue_time_us":1174738}
I20260709 12:27:00.487375 26020 leader_election.cc:403] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:00.578171 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 49917.9us
I20260709 12:27:00.423348  1425 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:00.583351 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 204512us
I20260709 12:27:00.569288 26789 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:00.565510 25946 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:00.541896   933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.013s
I20260709 12:27:00.573513 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:00.592737 26203 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:00.593061 26047 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:00.490655 26790 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:00.580156 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:00.595142 26128 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:27:00.580963 26194 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.007s	sys 0.028s
I20260709 12:27:00.586205 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:00.595999 26194 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.007s	sys 0.028s
I20260709 12:27:00.541652  1366 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:00.598080 26054 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:00.561125  1235 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.565850  1277 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:00.613516 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 156409us
I20260709 12:27:00.599785 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:00.593572 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:00.569600  1372 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:00.604735 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:00.626457 26051 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:00.604923 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:00.627710 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:27:00.577258  1303 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:00.603364   933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.000s	sys 0.014s
I20260709 12:27:00.625741 26052 raft_consensus.cc:2410] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:27:00.629292 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:00.582751  1275 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:00.543180  1430 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.646248 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:00.635129 25946 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 18, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 18 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:00.630895  1423 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:27:00.600920  1357 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:00.625157  1360 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:00.630920  1376 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:27:00.590978  1375 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:00.551167  1370 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:27:00.598371 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:00.605441  1290 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.596701  1440 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20.
I20260709 12:27:00.651690 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:00.625991  1332 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.669525 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 222776us
I20260709 12:27:00.683722 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:00.702476 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69987.6us
W20260709 12:27:00.645429  1197 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.003s
W20260709 12:27:00.652173  1335 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 12:27:00.626145  1372 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.659132 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:00.671660  1275 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:00.682806  1357 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.694792  1375 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:00.631808   984 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:27:00.699518  1297 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.692s	user 0.005s	sys 0.005s
I20260709 12:27:00.581014  1435 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.605655  1433 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:27:00.703337 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:00.705535 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1241) took 1075 ms. Trace:
W20260709 12:27:00.707181 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55578.3us
I20260709 12:27:00.719570  1423 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:00.670766  1175 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:00.638626  1432 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:27:00.733688 26121 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:00.734629 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75620.7us
I20260709 12:27:00.761611 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:00.724573  1372 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:00.757172 25946 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 19, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 19 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.751317 26195 rpcz_store.cc:276] 0709 12:26:59.630622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.635240 (+  4618us) service_pool.cc:224] Handling call
0709 12:26:59.724763 (+ 89523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.943303 (+218540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.943342 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.980874 (+ 37532us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.124933 (+144059us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:00.125568 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.126330 (+   762us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.705214 (+578884us) raft_consensus.cc:1685] finished
0709 12:27:00.705244 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.705447 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33046,"raft.queue_time_us":58075,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4412,"thread_start_us":297064,"threads_started":2,"wal-append.queue_time_us":159073}
I20260709 12:27:00.737031   984 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:00.745944  1360 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:00.720176  1332 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.722690  1197 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.749251  1326 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:00.723011  1347 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:27:00.724161  1335 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.762878  1175 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.774685  1376 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:00.747872  1297 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.785837 25946 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 16 to 23, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 23 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.750310  1449 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 12:27:00.799829 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 96558.1us
I20260709 12:27:00.789096  1326 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.826160 26048 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:00.820510 25946 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 16 to 20, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 20 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.789551  1366 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:00.800467  1175 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:00.807016  1370 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.799414  1275 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.823096  1235 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:00.822013  1386 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
I20260709 12:27:00.796067  1413 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.774966  1440 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:27:00.835600 25946 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 17 to 20, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.833426  1455 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:00.828663  1285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.012s	sys 0.012s
I20260709 12:27:00.824157  1376 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:00.829772 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 296) took 60438 ms (client timeout 15000 ms). Trace:
I20260709 12:27:00.797892  1375 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.812161   984 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.837671  1019 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:00.834985  1366 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:00.920159 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1629) took 1182 ms. Trace:
I20260709 12:27:00.853989 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:00.922433 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72830us
I20260709 12:27:01.030344 26787 rpcz_store.cc:276] 0709 12:26:59.738017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.759782 (+ 21765us) service_pool.cc:224] Handling call
0709 12:27:00.919979 (+1160197us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:00.920092 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554790144}
I20260709 12:27:00.906780 26126 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:01.031700 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:00.930923 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.012362 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.928741 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:00.925992 26048 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:00.869896 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:00.882691 25946 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:00.917932 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:27:01.036514 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118671us
I20260709 12:27:01.038255 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.031327 26127 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:00.860836  1459 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:01.042436 26198 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:01.043069 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 280175us
W20260709 12:27:00.851878  1285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.013s	sys 0.012s
I20260709 12:27:00.909387  1446 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.047757 26121 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:01.085255 26121 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:00.874410  1421 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.003s
I20260709 12:27:01.075266 25946 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.889604  1361 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.866591  1453 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:27:01.086774 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 216952us
I20260709 12:27:00.914508  1420 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:00.920414  1235 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.046180  1330 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.909144  1297 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:00.922178  1335 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.064076 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:01.088158 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:01.077901 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:00.906121  1184 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:01.086244 26121 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:00.921361  1357 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:00.910176  1332 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.098353 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:00.888947  1423 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:01.100515 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:00.841758  1370 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.043277  1304 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:00.851055  1386 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.103214 25946 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 15, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 15 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.907820  1456 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.200s
I20260709 12:27:01.078882  1019 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.867931  1175 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.078471  1285 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:00.900319  1303 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.089080   990 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:01.090693   984 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:01.123965 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60036us
I20260709 12:27:01.125483 26785 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:01.117296 26782 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:00.853514  1413 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.122418 26787 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:01.118378 26126 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:01.091215  1435 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.133270 26789 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:01.098189  1420 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:01.140015 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.043582  1277 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.112674  1476 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.146194 26791 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:01.150027 25946 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 19, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 19 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:00.861042  1376 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.157601  1386 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:01.158691 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68315.2us
I20260709 12:27:01.097666  1421 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.129776  1455 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:01.102025  1361 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.167295 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:01.168802 26783 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:01.131099  1197 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.176191 26787 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:01.175655 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 73496.5us
I20260709 12:27:01.095232  1446 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.146286  1332 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:01.122442  1330 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.181686 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:01.151513  1304 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:01.183588  1476 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.184904 25946 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:01.203265  1386 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.179615   990 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:01.233069  1366 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.233438  1421 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.255492 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:01.256183 26124 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:01.213037  1474 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:27:01.224706 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.258695 26126 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:01.258162 25946 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:01.259670 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:01.220009  1464 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:01.260880  1303 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:01.228874  1478 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:01.261726 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2855) took 1364 ms. Trace:
I20260709 12:27:01.262113 26194 rpcz_store.cc:276] 0709 12:26:59.897777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.899893 (+  2116us) service_pool.cc:224] Handling call
0709 12:26:59.900463 (+   570us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:00.596581 (+696118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.596636 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:00.597329 (+   693us) log.cc:844] Serialized 52 byte log entry
0709 12:27:00.725093 (+127764us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:00.725130 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.725204 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.261496 (+536292us) raft_consensus.cc:1685] finished
0709 12:27:01.261521 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.261670 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11058,"raft.queue_time_us":136985,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8308,"thread_start_us":793821,"threads_started":2,"wal-append.queue_time_us":481086}
I20260709 12:27:01.225739  1285 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.264308 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1627) took 1570 ms. Trace:
I20260709 12:27:01.264735 26788 rpcz_store.cc:276] 0709 12:26:59.694086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.766911 (+ 72825us) service_pool.cc:224] Handling call
0709 12:26:59.776859 (+  9948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.894257 (+117398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.264069 (+1369812us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000e2c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:01.264250 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245958016}
W20260709 12:27:01.238497 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63423us
I20260709 12:27:01.265890  1326 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.268215 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:01.241636 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.270110  1471 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:01.272651  1456 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:01.272163  1471 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:27:01.255056  1386 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.274767  1471 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:01.275183  1471 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:01.276379 26094 leader_election.cc:400] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:27:01.277463  1471 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.278097  1361 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.243342 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:01.134478  1184 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.279016   941 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.281035  1184 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.276679 26094 leader_election.cc:403] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:27:01.282279  1330 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.282374  1456 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:27:01.283337  1420 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.146247  1423 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.161300  1019 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:00.853034 25944 rpcz_store.cc:269] 0709 12:26:00.246148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.246406 (+   258us) service_pool.cc:224] Handling call
0709 12:27:00.666114 (+60419708us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:48.868621 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:49.149394 (+280773us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:49.191652 (+ 42258us) write_op.cc:435] Acquired schema lock
0709 12:26:49.225556 (+ 33904us) tablet.cc:662] Decoding operations
0709 12:26:49.418458 (+192902us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:49.418610 (+   152us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:49.418652 (+    42us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:26:49.420398 (+  1746us) tablet.cc:701] Row locks acquired
0709 12:26:49.420453 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:26:49.420654 (+   201us) write_op.cc:270] Start()
0709 12:26:49.420828 (+   174us) write_op.cc:276] Timestamp: P: 1783600009420578 usec, L: 0
0709 12:26:49.420866 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:26:49.477901 (+ 57035us) log.cc:844] Serialized 5965 byte log entry
0709 12:26:50.546397 (+1068496us) op_driver.cc:464] REPLICATION: finished
0709 12:26:51.069891 (+523494us) write_op.cc:301] APPLY: starting
0709 12:26:51.097403 (+ 27512us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:51.889627 (+792224us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:51.902622 (+ 12995us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:52.568622 (+666000us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:52.569502 (+   880us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:26:52.569646 (+   144us) write_op.cc:312] APPLY: finished
0709 12:26:52.570207 (+   561us) log.cc:844] Serialized 139 byte log entry
0709 12:26:53.543088 (+972881us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:53.555654 (+ 12566us) write_op.cc:454] Released schema lock
0709 12:26:53.571586 (+ 15932us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:53.628467 (+ 56881us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:53.628505 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":357304,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":442548,"apply.run_cpu_time_us":60070,"apply.run_wall_time_us":2559731,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66917,"num_ops":15,"prepare.queue_time_us":133878,"prepare.run_cpu_time_us":15073,"prepare.run_wall_time_us":973037,"raft.queue_time_us":225422,"raft.run_cpu_time_us":23105,"raft.run_wall_time_us":560828,"replication_time_us":1080110,"thread_start_us":1666606,"threads_started":5,"wal-append.queue_time_us":1126963,"wal-append.run_cpu_time_us":13442,"wal-append.run_wall_time_us":2438959}]]}
I20260709 12:27:01.232622 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:01.308310 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75763.8us
I20260709 12:27:01.288372 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.289331 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.311532 26203 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 17: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:01.291973 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48676.7us
I20260709 12:27:01.297592 26198 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:01.247635  1304 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:01.310227 26196 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:01.278826  1471 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:01.325940 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.319033 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.276122  1386 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.337137 26203 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:01.285043  1221 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.289427  1184 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.341176 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.362107 26198 raft_consensus.cc:2368] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25.
I20260709 12:27:01.361871  1184 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.363230 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.364460 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.365588 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.366674 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:01.366967  1442 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.682s	user 0.003s	sys 0.000s
I20260709 12:27:01.367219 26198 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:01.288997  1366 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:01.354094 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:01.354318 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 279) took 15007 ms (client timeout 15000 ms). Trace:
I20260709 12:27:01.354909  1413 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.354836  1330 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:01.357291  1370 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.359360  1326 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.317373  1197 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.322269  1447 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:01.296442  1367 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:01.375761 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.381443 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.381138 26124 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:01.390374 26196 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:01.404856 26203 raft_consensus.cc:2368] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:27:01.332952  1421 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:01.336377  1420 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:01.339157  1446 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.368772  1366 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.415829 26198 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:01.417377 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.418556 26198 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:01.421638 26195 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:01.417377 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.380641  1332 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.391124  1361 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:01.403939  1447 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.404686  1175 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.428020 26196 raft_consensus.cc:2368] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:27:01.431334 26196 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:01.434584  1175 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.418048  1446 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.445933 26050 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:01.446385 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:01.444298 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.447481 26127 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:01.447340 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:01.419445  1479 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:01.450928 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.453162 26198 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:01.454227 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:01.454470 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.449015 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0284442920001311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 12:27:01.449815 26196 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:01.411949  1197 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.448361 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.432267  1447 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:01.436466  1184 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.462184 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.409178  1367 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.461565 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.466781 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:01.467258 26124 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:01.469095 26203 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:01.469516 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1244) took 1622 ms. Trace:
I20260709 12:27:01.472755 26047 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:01.456643 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:01.458945 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:01.458180 26049 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:27:01.380383 25944 rpcz_store.cc:269] 0709 12:26:46.347425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.354037 (+15006612us) service_pool.cc:211] Skipping call since client already timed out
0709 12:27:01.354224 (+   187us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:27:01.463641 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.452081  1221 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:01.465863 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.477311 26196 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.481177 26050 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:01.435442  1175 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.485122 26203 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:01.457814  1449 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.448289  1413 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:01.450965  1446 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.456552  1479 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.492869  1423 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.463949 26052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:27:01.493359  1413 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.494083  1449 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.495898  1413 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.492414  1221 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.501281  1455 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:01.502854 26052 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:01.530061 26052 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:01.493256  1446 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.501168  1277 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:01.505990 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.504890  1366 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.502076  1235 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:01.482010 26198 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:01.545118 26198 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:01.503271 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:01.547329 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44132.6us
W20260709 12:27:01.478957  1314 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.429s	user 0.016s	sys 0.000s
I20260709 12:27:01.432473  1332 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.519605 26203 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
W20260709 12:27:01.526531 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.470018 26200 rpcz_store.cc:276] 0709 12:26:59.847543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.847697 (+   154us) service_pool.cc:224] Handling call
0709 12:26:59.858829 (+ 11132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.859173 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:59.892742 (+ 33569us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.284015 (+1391273us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b1800406480. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:27:01.469185 (+185170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:01.469240 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:01.469285 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.469343 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.469456 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14375,"prepare.queue_time_us":32680,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":125439,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":9612,"spinlock_wait_cycles":43603840,"thread_start_us":185030,"threads_started":1}
I20260709 12:27:01.530795 26196 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:01.430037   990 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.556128 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.557102 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.494124  1479 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.564926 26203 pending_rounds.cc:85] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 12
I20260709 12:27:01.565383 26196 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [19.16-21.17]
I20260709 12:27:01.479190  1184 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:27:01.527835  1451 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:01.576565 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.577814 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:01.578598 26126 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:01.573539 26203 pending_rounds.cc:107] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:27:01.580063 26203 raft_consensus.cc:2889] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:01.480898  1370 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 12:27:01.574828 26196 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:27:01.561648  1446 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:01.596513  1446 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.596643  1451 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.598383 26126 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:01.598690  1446 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.599066  1451 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.599650 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:01.596546  1449 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.601034  1370 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:01.601227  1367 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.584477 26127 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:01.601647 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:01.602164  1367 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:01.561990  1413 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:01.602715  1367 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:01.602792 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:01.603048  1413 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.603471  1413 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:01.603482  1197 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.587884 26196 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:27:01.568207  1180 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.603688  1367 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.595314 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:01.605465  1485 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 12:27:01.605703 26127 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:01.569577  1314 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.602797 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.610194 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.614480 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.615912 26127 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:01.609357 26052 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.628289 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.629076 26784 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
W20260709 12:27:01.634773 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 179758us
I20260709 12:27:01.551826  1486 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
W20260709 12:27:01.639451 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.662207 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 205686us
I20260709 12:27:01.640820 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:01.538544  1455 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.601696  1370 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:01.624222  1372 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.843s	user 0.012s	sys 0.000s
I20260709 12:27:01.666718 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:01.675920 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:01.676923 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:01.680900  1330 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:01.681754  1330 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.688186 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:01.627965  1435 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:27:01.705813 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:01.684569  1446 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:01.675324  1494 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:01.663913  1184 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:27:01.669309  1372 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.705010  1330 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.736317 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1238) took 2149 ms. Trace:
I20260709 12:27:01.738052 26199 rpcz_store.cc:276] 0709 12:26:59.586998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.587108 (+   110us) service_pool.cc:224] Handling call
0709 12:27:01.735960 (+2148852us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400399254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:01.736199 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548763392}
I20260709 12:27:01.714174  1494 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.735105  1197 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.771456 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65740.1us
I20260709 12:27:01.785941  1494 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:01.743057  1451 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:01.735155  1479 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.737437  1446 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:01.795804  1492 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 12:27:01.801718  1491 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:27:01.802858  1506 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:01.715425  1370 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.804663  1375 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:01.806269  1332 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.776949  1236 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.810765 26120 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:01.810635  1493 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:01.770869  1509 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:01.812492  1455 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.816866  1221 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.733613  1304 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.824338  1446 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.822865  1479 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:01.817646  1498 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:27:01.794361  1197 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.837001  1236 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.836614  1332 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:01.842501  1493 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.862942 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1353) took 1496 ms. Trace:
W20260709 12:27:01.850487  1437 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.488s	user 0.006s	sys 0.000s
I20260709 12:27:01.863701 26786 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:01.833745  1451 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:27:01.864971 26129 rpcz_store.cc:276] 0709 12:27:00.366551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.367323 (+   772us) service_pool.cc:224] Handling call
0709 12:27:00.367643 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.450957 (+ 83314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.451020 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.337758 (+886738us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:27:01.374790 (+ 37032us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.507392 (+132602us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:01.507445 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.507539 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.860383 (+352844us) raft_consensus.cc:1685] finished
0709 12:27:01.860409 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.862873 (+  2464us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14709,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3452,"spinlock_wait_cycles":16191744,"thread_start_us":173900,"threads_started":2,"wal-append.queue_time_us":264216}
I20260709 12:27:01.852926  1479 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.848659  1314 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.834970  1375 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.880035  1430 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.883428 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:01.866775  1332 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.891644  1335 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:01.888923  1297 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.877375  1197 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.897571  1236 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:01.894526  1347 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.006s	sys 0.002s
I20260709 12:27:01.894934  1455 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:01.898370  1332 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.896202  1335 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.895383  1516 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:01.928987  1347 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.007s	sys 0.002s
I20260709 12:27:01.900406  1297 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.891673  1447 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:01.960781  1508 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:27:01.961908  1448 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.038s	user 0.001s	sys 0.000s
I20260709 12:27:01.962927 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1356) took 1585 ms. Trace:
I20260709 12:27:01.963471 26123 rpcz_store.cc:276] 0709 12:27:00.377337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.509183 (+131846us) service_pool.cc:224] Handling call
0709 12:27:00.509527 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.534613 (+ 25086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.534660 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.535262 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.962353 (+1427091us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b1800250ea0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1
0709 12:27:01.962519 (+   166us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:01.962555 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.962618 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.962655 (+    37us) raft_consensus.cc:1685] finished
0709 12:27:01.962666 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.962866 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24636,"raft.queue_time_us":176740,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":19002,"spinlock_wait_cycles":17763072,"thread_start_us":195531,"threads_started":1,"wal-append.queue_time_us":235601}
I20260709 12:27:01.965122 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1020) took 1722 ms. Trace:
I20260709 12:27:01.965651 26046 rpcz_store.cc:276] 0709 12:27:00.243258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.243414 (+   156us) service_pool.cc:224] Handling call
0709 12:27:00.246583 (+  3169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.246824 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.246849 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.247320 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.625740 (+378420us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:00.625789 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.625857 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.961198 (+1335341us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:01.964750 (+  3552us) raft_consensus.cc:1685] finished
0709 12:27:01.964780 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.965048 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127015808,"thread_start_us":270358,"threads_started":1,"wal-append.queue_time_us":521909}
I20260709 12:27:01.969475  1375 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.971338  1493 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.979185  1335 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.942412  1235 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:01.986328 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.986980  1413 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:01.987702 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.988418  1236 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.988960 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.989122  1236 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.989899  1297 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.990119 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.949051  1304 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:01.991290 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.992486  1087 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:01.993192  1087 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.994364  1447 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:01.995064  1447 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.948071  1455 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:01.996883 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:01.997664 26790 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:01.998884  1455 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.000556 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2859) took 2101 ms. Trace:
I20260709 12:27:02.000716 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:02.001628 26787 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:02.002478 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 107378us
W20260709 12:27:02.002591 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 207647us
I20260709 12:27:02.001003 26197 rpcz_store.cc:276] 0709 12:26:59.899410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.920461 (+ 21051us) service_pool.cc:224] Handling call
0709 12:26:59.920965 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:59.928698 (+  7733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.928731 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:59.949334 (+ 20603us) log.cc:844] Serialized 52 byte log entry
0709 12:27:00.150896 (+201562us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:00.163473 (+ 12577us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.163876 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.164050 (+   174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.164117 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.850063 (+1685946us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b240020ba78. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:27:02.000273 (+150210us) raft_consensus.cc:1685] finished
0709 12:27:02.000303 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.000470 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41664,"prepare.queue_time_us":46022,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":83,"raft.queue_time_us":136127,"raft.run_cpu_time_us":536,"raft.run_wall_time_us":62833,"spinlock_wait_cycles":39042944,"thread_start_us":120211,"threads_started":1,"wal-append.queue_time_us":322832}
I20260709 12:27:02.013634 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:02.014313 26050 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.014400 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.015136 26047 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:01.995590  1447 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.016616 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:02.018843 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:01.991873  1236 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:02.022117 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.021685 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:02.018225 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.015769 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:01.952749  1516 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:02.018231  1235 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:27:02.025512 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.041468 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.037395  1052 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:27:02.043450  1456 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:02.044481  1515 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:02.044227  1516 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.044931  1515 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:02.044943  1197 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:02.045913  1515 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.047608  1515 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:02.039181  1471 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.019470  1284 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.057479  1284 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.023363  1386 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:02.061601 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43480.5us
I20260709 12:27:02.037569 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:02.018344  1375 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:02.024020 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:02.054497  1361 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:02.056298  1413 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.061437  1386 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.037006  1314 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:27:02.065694  1361 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.068090  1386 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.068634  1361 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.069252  1386 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:02.065726  1375 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.070178  1494 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.073565  1447 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:02.078799 26788 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:02.059960  1284 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:01.995846  1087 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.065673 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:02.072032  1375 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.082098 26197 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:02.082625 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.086484 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 60710.1us
W20260709 12:27:02.086761 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.063270 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:02.085309 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61379.4us
I20260709 12:27:02.089536 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:01.991894 26785 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:27:02.091621 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.092321 26198 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:02.095053 26051 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:27:02.096210 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:02.097229 26046 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:02.098223 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.096314 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:02.102180 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:02.103523 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.104386 26054 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 22 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 12:27:02.104947 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.106277 26050 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:02.107208 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.108006 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.108525 26050 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:02.102682 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.109146 26200 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:27:02.111869 26197 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:02.073887  1494 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:02.114095 26049 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:02.102180  1314 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.119643 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.120894 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:02.121407 26049 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:02.121572  1375 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:02.114832 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.105580 26051 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:02.122267  1375 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:02.122748 26054 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
I20260709 12:27:02.123172 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.123253  1375 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.124013 26051 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:02.095678  1304 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:02.124998 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.125599 26051 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:02.129873  1284 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:02.130658  1284 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.117487 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.123662 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.129873  1489 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:02.124670  1304 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.135069 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.095477  1469 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:02.099488  1456 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:02.151089 26045 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:02.152557 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.116367 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.157910 26094 leader_election.cc:400] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25.
I20260709 12:27:02.113852  1297 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:02.114985 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:02.162765 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.161304 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.156455 26790 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
W20260709 12:27:02.163396 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47144.1us
I20260709 12:27:02.165694 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:02.168134 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53239.4us
I20260709 12:27:02.164400 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.159734  1519 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 12:27:02.172871  1087 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:02.160168  1469 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.216640 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.156497  1489 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.157023  1456 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:02.182770  1284 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:02.186724 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.157426  1454 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.376s	user 0.004s	sys 0.004s
I20260709 12:27:02.186794  1324 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:02.187558  1516 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:02.187637 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.188110 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:02.188323  1235 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.189527 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:02.190651  1386 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:02.203308 26787 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:02.210901 26054 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:27:02.217800 26198 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:02.217907 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.234107 26046 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:02.220821 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:02.212297 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44635.9us
I20260709 12:27:02.216348 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.231249 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:02.232489 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44933.1us
I20260709 12:27:02.235728 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.196568  1297 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:02.218401 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.239908 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.240676 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.237144 26094 leader_election.cc:400] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 16 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 16. Current term is 17.
I20260709 12:27:02.240566 26203 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:27:02.242822 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:02.229383  1456 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:02.245189 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.243003 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.233991  1447 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:02.227766  1087 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:02.241550 26196 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:02.248601 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:02.243543 26199 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.239537 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:02.250705 26198 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:02.250811 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:02.238409 26046 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.243028 26094 leader_election.cc:400] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:27:02.248611 26203 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:02.252029 26094 leader_election.cc:403] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:27:02.256667 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:02.260746 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.236754  1516 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:02.263355 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.269066 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.273162 26787 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:02.274076 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.237660  1455 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:02.274477 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.275333 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.276463 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.273303 26200 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:27:02.243062  1519 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.278404 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.275615 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:02.281071 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.287643 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:02.292829 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:02.293975 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.293982  1520 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:02.295150 26787 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:02.295248  1520 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:02.295719  1520 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:02.296155 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.296495 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:02.296434  1513 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:02.297792 26787 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:02.297863  1513 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:02.298067 26789 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:02.298369  1513 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:02.299337 26789 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:02.299258  1513 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.300490 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.304967  1455 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:02.306557  1471 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:02.307142  1455 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:02.307473  1471 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:02.306841  1297 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.307859  1471 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:02.255409  1438 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.308728  1471 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.309175  1519 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:02.309573  1519 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:02.310292  1471 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.310977  1516 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.270460  1421 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:02.312638  1421 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:02.285441  1446 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.317026 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:02.317200 26199 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:02.317741 26782 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:02.271695  1235 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22.
I20260709 12:27:02.313515  1446 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.306542  1087 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:02.350260  1485 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:27:02.282120  1332 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:02.337441  1522 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:27:02.314079  1347 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.279161  1499 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:27:02.282733  1449 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:27:02.333724  1513 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:02.352126  1446 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.364369  1332 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:02.456182 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 150000us
W20260709 12:27:02.457464 26756 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.186s (SENT) [suppressed 30 similar messages]
I20260709 12:27:02.402936  1449 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:02.356324  1519 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.460141  1469 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.466627  1489 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.488253 26782 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:02.359323  1520 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.494535 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1635) took 1396 ms. Trace:
I20260709 12:27:02.465615  1499 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:27:02.495221 26783 rpcz_store.cc:276] 0709 12:27:01.098520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.098642 (+   122us) service_pool.cc:224] Handling call
0709 12:27:01.117859 (+ 19217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.237472 (+119613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.494163 (+1256691us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:02.494441 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116461824}
W20260709 12:27:02.490430  1531 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:27:02.495512  1519 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:02.510135 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.510998 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.496937  1469 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.500305  1372 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.512025  1520 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.541005  1476 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.542716 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2863) took 1255 ms. Trace:
I20260709 12:27:02.543148 26195 rpcz_store.cc:276] 0709 12:27:01.287049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.287239 (+   190us) service_pool.cc:224] Handling call
0709 12:27:01.317816 (+ 30577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.422214 (+104398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.542424 (+1120210us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002175d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:02.542639 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237671296}
I20260709 12:27:02.545405  1456 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:02.525061 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65311.4us
I20260709 12:27:02.550675 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:02.548182  1456 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.553956  1476 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:02.554448  1476 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:02.554903  1476 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:02.561838 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:02.566485  1542 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:02.567020  1542 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:02.567476  1542 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.521380  1469 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.571841  1469 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.580973 26195 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:02.577332  1542 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.583527 26196 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.653519 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:02.657085 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.509282  1332 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.591317 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.614373  1456 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:02.618804  1235 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:02.620627 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:02.685529 26054 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:02.631878  1515 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.584s	user 0.017s	sys 0.005s
I20260709 12:27:02.635002  1509 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:27:02.636770  1314 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.517s	user 0.017s	sys 0.003s
I20260709 12:27:02.638248 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1814) took 1156 ms. Trace:
I20260709 12:27:02.639148  1304 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.642479 26052 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.642805  1446 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.644945 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1252) took 1604 ms. Trace:
I20260709 12:27:02.639190 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1354) took 2272 ms. Trace:
I20260709 12:27:02.649225 26121 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:02.688700  1449 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:02.700488  1314 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:02.701282 26051 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:02.657763 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:02.683894 26790 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:02.702101 26202 rpcz_store.cc:276] 0709 12:27:01.040551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.043712 (+  3161us) service_pool.cc:224] Handling call
0709 12:27:01.044038 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:01.411125 (+367087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.411170 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.412183 (+  1013us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.479973 (+ 67790us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:01.480025 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.644356 (+1164331us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002b3754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:02.644431 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.644508 (+    77us) raft_consensus.cc:1685] finished
0709 12:27:02.644532 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.644853 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5067,"raft.queue_time_us":42000,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":11263,"spinlock_wait_cycles":472054016,"thread_start_us":223622,"threads_started":2,"wal-append.queue_time_us":196347}
I20260709 12:27:02.699589  1449 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.685194 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:02.628091  1019 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:02.692224 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 234794us
I20260709 12:27:02.691920 26786 rpcz_store.cc:276] 0709 12:27:01.481561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.481715 (+   154us) service_pool.cc:224] Handling call
0709 12:27:01.482069 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.869698 (+387629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.637871 (+768173us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54005d20d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:02.638177 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877196160}
I20260709 12:27:02.710701 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:47804 (request call id 2462) took 1264 ms. Trace:
I20260709 12:27:02.690981  1455 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.711134 26052 rpcz_store.cc:276] 0709 12:27:01.457032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.463828 (+  6796us) service_pool.cc:224] Handling call
0709 12:27:02.641015 (+1177187us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439ca3d 00007f5b7936ccd0 00007f5b728f1254 00007f5b728f11e2 00007f5b728f1111 00007f5b728f108c 00007f5b728f0342 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:02.708308 (+ 67293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20738944}
I20260709 12:27:02.691576  1509 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:02.718017  1509 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.719480  1509 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:02.685254  1235 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:02.702167  1515 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.706705 26125 rpcz_store.cc:276] 0709 12:27:00.366817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.370495 (+  3678us) service_pool.cc:224] Handling call
0709 12:27:00.371141 (+   646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.417894 (+ 46753us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:00.670362 (+252468us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.745782 (+ 75420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.748899 (+  3117us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:00.748985 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.869964 (+120979us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b18000daaa0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:02.638909 (+1768945us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b54003d1b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:02.638975 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.639124 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5119,"prepare.queue_time_us":378761,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":58052,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":19467,"spinlock_wait_cycles":118074880,"thread_start_us":278735,"threads_started":2,"wal-append.queue_time_us":117427,"wal-append.run_cpu_time_us":1279,"wal-append.run_wall_time_us":1030287}
I20260709 12:27:02.714161  1499 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.742204 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.705539  1456 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.745759 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.752681  1451 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:02.764551 26052 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:02.753158  1175 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.754818  1446 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.756287  1449 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.758421 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.771654 26045 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:02.730763  1019 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:27:02.728062  1434 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.760887  1335 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:02.726430  1455 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:02.761971 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49059.8us
I20260709 12:27:02.765583 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.776786 26052 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:02.759675  1542 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:02.773135 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 480358us
I20260709 12:27:02.784396 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:02.773747 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.778013 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:02.779524 26198 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:27:02.779696 26196 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:27:02.779781 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.780581 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:02.781967 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:02.793380 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 548233us
I20260709 12:27:02.738983  1469 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:27:02.797967 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:02.798640 26049 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:02.776661  1517 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:02.737279  1515 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.785696  1019 consensus_queue.cc:1243] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:02.811627 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:27:02.820521 26121 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:27:02.821933 26121 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 11
I20260709 12:27:02.822817 26121 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:27:02.802413  1297 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:02.802181  1335 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.826843  1517 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:02.829049  1455 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.830279  1330 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:02.830684  1456 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:02.831697  1425 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:02.826851 26048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1021) took 2581 ms (client timeout 2186 ms). Trace:
I20260709 12:27:02.833078 26195 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:27:02.833316  1515 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.837378 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2856) took 2939 ms. Trace:
I20260709 12:27:02.823284  1357 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:27:02.805524  1197 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.841347 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.879570 26201 rpcz_store.cc:276] 0709 12:26:59.898210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.903186 (+  4976us) service_pool.cc:224] Handling call
0709 12:26:59.903630 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.177960 (+274330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.177989 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.912555 (+734566us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:27:00.912962 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.078639 (+165677us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:01.078678 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.837069 (+1758391us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5400302854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:02.837134 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.837181 (+    47us) raft_consensus.cc:1685] finished
0709 12:27:02.837192 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.837319 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53698,"raft.queue_time_us":139459,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":55744,"spinlock_wait_cycles":66247680,"thread_start_us":413696,"threads_started":2,"wal-append.queue_time_us":217048,"wal-append.run_cpu_time_us":2581,"wal-append.run_wall_time_us":1137700}
I20260709 12:27:02.842806 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:02.843266 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.884473 26787 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:02.884827 26202 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:02.776901  1434 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.844333 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.845120 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:02.844333 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:02.850405 26791 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:02.851563 26121 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:02.887360 26783 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
W20260709 12:27:02.876981 26048 rpcz_store.cc:269] 0709 12:27:00.244356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.244506 (+   150us) service_pool.cc:224] Handling call
0709 12:27:02.825654 (+2581148us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400131354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:02.825763 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025460096}
W20260709 12:27:02.855726 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.872815  1330 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:02.897044 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 97391.9us
I20260709 12:27:02.902760  1197 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:02.902055 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:02.904425  1197 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:02.897537  1357 consensus_queue.cc:1243] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:02.905777  1197 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:02.907212  1197 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.888123  1536 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.867734  1297 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.914386 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:02.865343  1552 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:27:02.939922  1434 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:02.919133 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.873024  1335 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:02.920759 26783 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:02.945648 26783 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:02.946448 26787 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:02.947988 26093 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:02.918974 26054 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:02.951459 26048 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 12:27:02.955089 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.955734 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.974191 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.920765  1536 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:02.980811 26048 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:02.979473  1476 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:03.036473 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.016896 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:02.967931  1515 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:03.077904  1335 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:02.999334 26783 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.079039 26783 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.087037 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.087970 26787 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:27:03.106751 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.118186 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:03.126528 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.127720 26052 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:03.140524 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:03.141585 26202 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:03.143814 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:03.146284 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:03.148537 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.165405 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.112978  1513 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:02.950243  1330 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.164742  1304 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:27:03.206249 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3883) took 1755 ms. Trace:
I20260709 12:27:03.121076  1519 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:03.123102  1335 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:03.217330 26120 rpcz_store.cc:276] 0709 12:27:01.450520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.450644 (+   124us) service_pool.cc:224] Handling call
0709 12:27:03.206000 (+1755356us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400186ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:03.206162 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825599744}
I20260709 12:27:03.129297  1515 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:03.221292 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.165972  1019 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.236814  1019 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.166119  1557 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.241071  1557 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.242199  1449 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:03.242638  1449 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.243666 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:03.243481  1449 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.166651  1197 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:03.245692  1197 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.246191 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.247304  1553 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:03.248214 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.248369  1515 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.248658 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:03.250489 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.241110 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:03.167930  1297 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:03.269676 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.234151 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3793) took 1115 ms. Trace:
I20260709 12:27:03.270943 26051 rpcz_store.cc:276] 0709 12:27:02.150486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.152296 (+  1810us) service_pool.cc:224] Handling call
0709 12:27:03.233907 (+1081611us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:03.234047 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249446144}
I20260709 12:27:03.258339 26197 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:03.260660 26782 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:27:03.275450  1052 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:03.276104 26785 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:03.276225  1052 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:03.281390 26789 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:03.286675 26788 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:03.288584 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:03.289422 26784 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.246222  1197 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.231036  1519 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.309437  1476 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:03.310679  1519 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:03.250794  1335 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.219909  1513 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.271438  1542 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:03.313051  1542 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.270578  1019 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:03.277978  1330 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:03.244132  1557 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:03.272361  1451 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:27:03.321339  1542 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:03.360900  1434 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:27:03.363095  1513 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:03.363436  1019 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:03.364796  1434 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:03.365312  1434 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:03.366255  1434 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.367625  1434 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:03.368924  1565 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:27:03.371724 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 102089us
I20260709 12:27:03.373078 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.343755  1553 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.371289  1332 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 12:27:03.376690  1332 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:03.376534  1553 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.377140  1332 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:03.381546 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:03.386315 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:03.386890 26202 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:03.387566 26051 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:03.404033 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.407308 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2912) took 2633 ms. Trace:
I20260709 12:27:03.407785 26121 rpcz_store.cc:276] 0709 12:27:00.773591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.773766 (+   175us) service_pool.cc:224] Handling call
0709 12:27:00.774027 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.648894 (+1874867us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b58009c26d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:27:02.887624 (+238730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.407020 (+519396us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:03.407246 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749842176}
I20260709 12:27:03.408703  1472 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:03.409101  1472 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.410071  1472 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.411222  1472 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.415256  1473 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:03.415874  1473 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.416296 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.416970 26785 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:03.375895 26783 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:03.389055 26784 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:03.389649 26054 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:03.389566 26196 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:03.418045 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.418730 26051 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:27:03.390169 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:03.380488  1425 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.004s
I20260709 12:27:03.422114 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:03.422729 26052 raft_consensus.cc:2376] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24. Re-sending same reply.
I20260709 12:27:03.392977 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.423182 26782 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:03.423660 26050 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:03.394482 26049 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:27:03.400159  1516 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.401727 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:03.414963  1542 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.391263 26788 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:03.391070  1332 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.428380 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:03.379910 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.419966 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2861) took 2134 ms. Trace:
I20260709 12:27:03.423041 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:03.420918 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.430779 26196 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.429535 26194 rpcz_store.cc:276] 0709 12:27:01.286312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.290023 (+  3711us) service_pool.cc:224] Handling call
0709 12:27:01.290444 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.336382 (+ 45938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.336430 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:01.336495 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:01.336543 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.419690 (+2083147us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:03.419753 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.419899 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104795904}
I20260709 12:27:03.416877  1335 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:03.436340 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56544.7us
I20260709 12:27:03.436967 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:03.435581 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:03.428313  1509 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.009s
I20260709 12:27:03.445166 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.423246  1494 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.451578 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:03.452564 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:03.453351 26195 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:27:03.458220 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:03.450572 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:03.458907 26196 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:03.388641  1513 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:03.416792  1473 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.617448 26049 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:03.460170 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:03.565706 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.436553  1180 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:03.614702 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.489719 26196 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:03.620709 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.454857  1332 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.619827 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.478729 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:03.447120  1366 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.074s	sys 0.139s
W20260709 12:27:03.675484 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51907.3us
W20260709 12:27:03.674510 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 195904us
I20260709 12:27:03.688488  1557 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:03.689973  1288 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:03.690434  1288 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.690776  1288 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:03.692656  1471 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.621061 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:03.697584 26785 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:03.430207  1516 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:03.699261 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.448812  1425 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:03.621933  1509 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.460287  1451 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:03.669551  1180 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.449838  1569 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:03.456727  1019 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.247s
I20260709 12:27:03.707124 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:03.774142 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.774639 26195 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:03.677598  1455 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:03.711272 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:03.694123  1366 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.075s	sys 0.140s
I20260709 12:27:03.677223  1314 raft_consensus.cc:2764] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:03.693240  1471 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.702533  1473 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.749923  1235 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.756047  1347 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:03.757563  1469 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:03.808393  1469 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.809424  1469 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.810581  1469 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:03.768395 26202 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.462106  1494 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.704515  1087 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.707859 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:03.672802  1566 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 12:27:03.812865 26201 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:03.813642  1566 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:03.814044  1566 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.814766  1288 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.770826  1451 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:03.773759  1180 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.770936  1425 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:03.828460  1425 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.724677  1499 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:27:03.831779  1499 consensus_queue.cc:1243] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 26.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:03.784011  1569 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.832968  1569 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:03.834511  1544 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.118s	user 0.000s	sys 0.017s
I20260709 12:27:03.787658  1370 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:03.836822  1516 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:03.837420  1542 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:03.839357  1527 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.629s	user 0.012s	sys 0.005s
I20260709 12:27:03.812996 26197 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:03.840741 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3778) took 1750 ms. Trace:
I20260709 12:27:03.776896 26195 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:03.841151 26047 rpcz_store.cc:276] 0709 12:27:02.090481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.091990 (+  1509us) service_pool.cc:224] Handling call
0709 12:27:02.092421 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.092872 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.092915 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.100158 (+  7243us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.210442 (+110284us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:02.210495 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.210567 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.838797 (+1628230us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:03.840380 (+  1583us) raft_consensus.cc:1685] finished
0709 12:27:03.840402 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.840672 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":613,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5594,"spinlock_wait_cycles":16266112,"thread_start_us":81367,"threads_started":1,"wal-append.queue_time_us":80755}
I20260709 12:27:03.842978  1471 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:03.843258  1438 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:03.843734  1438 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:03.771127  1509 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.844681  1438 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.845973  1438 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.846807  1509 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:03.849860  1288 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:03.850409 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:03.852330 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.856596  1310 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:03.857376  1310 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:03.857715  1310 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.858050  1310 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.860842 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:03.862625 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.818924  1235 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.825009  1451 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:03.814903  1566 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:03.820489 26196 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:27:03.861127  1567 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:27:03.865060 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:03.823969 26202 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:03.878252 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:03.879510 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.782182  1455 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:03.799093  1347 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:03.885565 26053 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:27:03.886019 26053 pending_rounds.cc:85] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 15
I20260709 12:27:03.885774 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:03.886307 26053 pending_rounds.cc:107] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:27:03.886641 26053 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:03.882047  1332 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:03.874513  1288 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:03.898581 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:03.929733 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:03.931910 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:03.946419 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 486328us
I20260709 12:27:03.946220 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.889705  1560 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:03.947220 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 492084us
I20260709 12:27:03.891637  1470 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:03.896499 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.897204  1511 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:03.958022 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:03.900471  1288 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.970288 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:03.949395 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.966423 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:03.977419 26121 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.962512 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:03.887910  1285 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:27:03.942334  1375 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:03.957006  1516 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:03.992399 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.949117  1515 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.014s	sys 0.019s
I20260709 12:27:04.026855 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.027431 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.028228 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.026928  1314 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:04.028923 26127 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:04.029412  1314 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.029850  1314 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.950012  1335 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:04.030619 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:04.031406 26127 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:04.033116 26129 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:04.034554 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.036171 26126 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:04.037582 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:04.038221  1314 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.039377 26122 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:03.950875  1370 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:04.079869 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.027650 26123 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:03.960538  1542 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:03.976632  1288 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:03.968106  1511 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:04.012501 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.011598 26121 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:27:04.013933 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51533.5us
W20260709 12:27:04.015641  1434 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.648s	user 0.013s	sys 0.001s
I20260709 12:27:03.988906  1423 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:04.097707 26127 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:03.998075  1516 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.099104 26125 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 12:27:04.099707 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.003s	sys 0.019s
I20260709 12:27:03.968700  1347 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.209753 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1822) took 2034 ms. Trace:
I20260709 12:27:04.210599  1347 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:03.960911  1560 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:04.133271  1515 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.016s	sys 0.031s
I20260709 12:27:04.143946 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.213073 26121 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:27:04.213992 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:04.214550 26121 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:04.215217  1560 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:04.085142  1470 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:04.210314 26790 rpcz_store.cc:276] 0709 12:27:02.175251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.199653 (+ 24402us) service_pool.cc:224] Handling call
0709 12:27:02.200083 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.703997 (+503914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.209428 (+1505431us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540069e6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:04.209670 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112084864}
I20260709 12:27:04.133304 26122 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
W20260709 12:27:04.178566 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.004s	sys 0.020s
W20260709 12:27:04.220899 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 762688us
I20260709 12:27:04.221092 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.222714 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.224165 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:04.225400  1554 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:04.225868  1554 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:04.098896  1471 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:04.226764  1554 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.135946  1555 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.227813  1554 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:04.228605 26121 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:04.228148  1555 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.229611 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.230137 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2475) took 1279 ms. Trace:
I20260709 12:27:04.230624 26054 rpcz_store.cc:276] 0709 12:27:02.950995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.003168 (+ 52173us) service_pool.cc:224] Handling call
0709 12:27:03.003616 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.418501 (+414885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.229900 (+811399us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400173c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:04.230081 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887205120}
W20260709 12:27:04.234967  1330 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.890s	user 0.009s	sys 0.003s
I20260709 12:27:04.236055 26200 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:04.237042  1435 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:04.237687 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1258) took 1105 ms. Trace:
I20260709 12:27:04.238205 26200 rpcz_store.cc:276] 0709 12:27:03.136116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.136318 (+   202us) service_pool.cc:224] Handling call
0709 12:27:03.177277 (+ 40959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.236535 (+1059258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.237353 (+   818us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400396f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:04.237605 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2433756032}
I20260709 12:27:04.185745  1314 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.192422  1516 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:04.196355  1288 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.145694  1285 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.144752  1473 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:04.203575  1455 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.205950  1494 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:04.208416  1536 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.208979 26194 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:04.184504  1472 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.242983 26121 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:04.179561 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.011946  1180 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:04.250560 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.018s	sys 0.018s
W20260709 12:27:04.285280 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.019s	sys 0.018s
I20260709 12:27:04.225888 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:04.208027  1197 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.879s	user 0.006s	sys 0.005s
W20260709 12:27:04.256342  1483 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.003s	sys 0.020s
W20260709 12:27:04.276306 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.253603  1470 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:04.145948  1511 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:04.227455  1471 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:04.285710 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 592381us
W20260709 12:27:04.320528 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74411.6us
I20260709 12:27:04.280102  1432 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:04.237710  1435 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.143280  1326 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.145378  1542 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.232033  1555 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:04.177646  1375 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 12:27:04.313596 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.345537  1432 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:04.346092  1432 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:04.349884 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.157140 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.350871 26123 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:04.351310 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.351824 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.352066 26125 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:27:04.352185 26129 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:04.352463 26123 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:04.353000 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.353289 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.353717 26125 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 16.
I20260709 12:27:04.354600 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.355228 26123 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:04.286381 26122 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:04.242991  1423 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.356874 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.358045 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.358371  1423 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:04.359117 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.359627  1470 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.361104 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:04.361582  1469 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.002s
I20260709 12:27:04.361959 26122 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:27:04.322804  1580 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:27:04.362816 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:04.363350 26122 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:04.305436  1086 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:27:04.356069 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.325654 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:04.311416  1483 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.004s	sys 0.020s
I20260709 12:27:04.326327  1473 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:27:04.326509   984 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:04.264546  1285 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.349978  1542 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.315894  1335 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:27:04.365778 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:04.324447  1366 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.005s	sys 0.014s
I20260709 12:27:04.309065 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.335305 26198 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:04.366814 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:04.405082 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:04.405835  1516 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.408748  1553 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:04.409173  1553 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:04.279618  1304 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:27:04.387789  1566 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:04.311650 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 462368us
I20260709 12:27:04.366608 26123 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:27:04.417498 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:04.356900  1288 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:04.401717 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:04.304718  1472 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.403996 26121 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:04.422247 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:04.412026 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.376026  1310 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.001s
I20260709 12:27:04.413249 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:04.386381  1438 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.002s
I20260709 12:27:04.325769  1413 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:04.317189  1197 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.418047 26123 raft_consensus.cc:606] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:04.449708 26123 raft_consensus.cc:614] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:04.375360  1469 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:04.402935 26120 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:27:04.320537  1574 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:27:04.387728   984 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:04.416523  1366 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.019s
I20260709 12:27:04.413129  1473 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:04.420912 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.415135  1516 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.473134 26122 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:04.474124 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:04.458740 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.476460 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:04.409948  1553 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.466538 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:04.477053 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.387180  1326 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.477907 26125 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:04.478489 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:04.426992  1573 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:04.446233  1547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.005s	sys 0.026s
I20260709 12:27:04.448549  1542 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:04.488168 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.024s
W20260709 12:27:04.488659 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.009s	sys 0.024s
W20260709 12:27:04.461705  1361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.009s	sys 0.022s
I20260709 12:27:04.502462 26129 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:27:04.489306 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:04.504838 26129 pending_rounds.cc:85] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:27:04.442303  1332 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:04.485093 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1032) took 2458 ms. Trace:
I20260709 12:27:04.484256 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:04.445735  1310 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.505268 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:04.507179 26053 rpcz_store.cc:276] 0709 12:27:02.017028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.017149 (+   121us) service_pool.cc:224] Handling call
0709 12:27:02.017445 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.887464 (+1870019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.474675 (+587211us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54002d0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:04.474877 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4292472576}
I20260709 12:27:04.513118 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:04.492583 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:04.514113 26196 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:04.505183 26129 pending_rounds.cc:107] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 22.16
W20260709 12:27:04.487043  1547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.006s	sys 0.026s
I20260709 12:27:04.515164 26129 raft_consensus.cc:2889] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:04.505118 26194 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:04.491428 26199 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:04.457098  1413 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.487265  1297 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:04.514602 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:04.488555  1423 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:04.487691  1438 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.484952  1557 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:04.549175 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.492817 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:04.554868 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.492261 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:04.594023 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:04.594385 26202 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
W20260709 12:27:04.485849 26789 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.016s	sys 0.034s
W20260709 12:27:04.595398 26789 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.017s	sys 0.034s
W20260709 12:27:04.481400 26203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.006s	sys 0.046s
I20260709 12:27:04.504215  1488 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:04.557300 26756 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.012s	sys 0.026s
W20260709 12:27:04.610092 26756 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.012s	sys 0.026s
W20260709 12:27:04.610419 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 587126us
W20260709 12:27:04.489053 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 592625us
I20260709 12:27:04.612735 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:04.556874 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:04.487687  1513 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.501453  1582 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.519392 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:04.487879  1542 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:04.575222 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.503213  1361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.010s	sys 0.023s
I20260709 12:27:04.501472  1494 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.502122  1509 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:04.595911 26203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.006s	sys 0.046s
I20260709 12:27:04.526484 26125 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:04.506268  1553 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:04.597844 26199 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:04.615221 26129 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:04.585534   990 consensus_queue.cc:1175] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a981c9a7972c4e99a0903924c64fbb5c" responder_term: 19 status { last_received { term: 18 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:04.502118  1573 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.607820 26194 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:04.521032  1556 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:04.523588  1456 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:04.523478  1447 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:04.525177  1471 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:04.517647  1516 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:04.608639  1235 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:04.621320 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 102042us
I20260709 12:27:04.556872  1366 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:04.833613 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.800576 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.692499 26169 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.013s	sys 0.009s
W20260709 12:27:04.851102 26169 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.018s	sys 0.009s
I20260709 12:27:04.853451 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:04.658738 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43881.7us
I20260709 12:27:04.851152 26129 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:04.754588 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:04.856438 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:04.729012 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 116385us
W20260709 12:27:04.852013 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 709162us
I20260709 12:27:04.858513 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:04.860693 26053 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:04.861668 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1273) took 1154 ms. Trace:
I20260709 12:27:04.860780 26125 raft_consensus.cc:606] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:04.597815  1557 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:04.609194  1564 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:27:04.862067 26203 rpcz_store.cc:276] 0709 12:27:03.707713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.708392 (+   679us) service_pool.cc:224] Handling call
0709 12:27:03.708683 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.708896 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:03.709189 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.856769 (+1147580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.856825 (+    56us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:04.856871 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.856937 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.861596 (+  4659us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6334,"prepare.queue_time_us":2236,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":45904,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":20818,"thread_start_us":1147157,"threads_started":2,"wal-append.queue_time_us":515581}
W20260709 12:27:04.872882 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.002s	sys 0.015s
W20260709 12:27:04.873848 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.003s	sys 0.015s
I20260709 12:27:04.876338 26047 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:27:04.876530 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 918569us
I20260709 12:27:04.877416 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:04.878803 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:04.879706 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:04.872143 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:04.880467 26050 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:04.880559 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:04.880718 26053 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:04.879662 26047 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:04.875394 26129 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:27:04.886260 26129 pending_rounds.cc:85] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 14
I20260709 12:27:04.886590 26129 pending_rounds.cc:107] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 23.15
W20260709 12:27:04.803063  1451 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.921s	user 0.008s	sys 0.004s
W20260709 12:27:04.906572 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48505.9us
I20260709 12:27:04.932821 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:04.938484 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:04.938701 26054 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:27:04.933187 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:04.934257 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80936.9us
I20260709 12:27:04.767652  1347 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:04.667232  1554 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:04.886889 26129 raft_consensus.cc:2889] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:04.939790 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:04.872119  1470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.000s
W20260709 12:27:04.937963 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.851828  1556 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:04.841063  1511 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 12:27:04.767081  1572 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:04.941041 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:04.941255  1304 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:04.801815  1180 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:04.997210  1304 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:04.998692  1587 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:27:04.999625  1304 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:04.940418  1482 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:04.882742  1494 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.000730  1482 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:05.001868  1482 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.002844  1494 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:04.939692  1471 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.004586  1471 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.617193  1580 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:04.975369  1470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.018s	sys 0.005s
I20260709 12:27:05.006773  1599 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.007597  1470 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.007510  1599 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.009415 26196 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:05.008111  1470 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.830978  1285 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:04.752971  1314 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.884013  1542 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:04.969746 26046 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:04.660957  1375 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.969007  1554 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.864934  1447 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:04.883236  1557 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:04.971897 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:04.609710  1423 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.018034  1285 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.017642  1536 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:04.997673  1589 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.441s	user 0.001s	sys 0.000s
I20260709 12:27:05.002360  1573 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.965862 26053 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:05.022327 26123 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 12:27:05.091358 26123 pending_rounds.cc:85] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 12
I20260709 12:27:05.091639 26123 pending_rounds.cc:107] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 12:27:05.092026 26123 raft_consensus.cc:2889] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.093650 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.020715 26050 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:04.663568  1513 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.787937  1593 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:04.608831  1332 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.616866  1499 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.317s
I20260709 12:27:04.618600  1488 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.868553  1366 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.989840  1556 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:04.667698  1582 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.045786  1314 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.028155  1511 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.988634  1483 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:05.019441  1197 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.072973 26129 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 12:27:05.142486 26129 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 13
I20260709 12:27:05.142797 26129 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:27:05.143059 26129 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:05.077170  1598 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.064s	user 0.002s	sys 0.001s
W20260709 12:27:05.078809  1600 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:27:05.080315  1310 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:05.081588  1413 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.012318  1470 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:05.045451  1375 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:04.883061  1435 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:04.854463  1235 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.012435  1572 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:04.958786  1451 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.051992  1330 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:05.095607 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 239284us
I20260709 12:27:05.194689 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:05.055840  1554 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:05.079999   984 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.056694  1052 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:05.095485  1601 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:27:04.886868  1509 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.217788  1488 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.219398  1509 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:04.610651  1438 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.220950  1572 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:05.222435  1584 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:27:05.222525  1438 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:05.096364  1516 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:05.103405  1593 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.224169  1593 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:05.225098  1423 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:05.229671  1297 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 12:27:05.230389  1297 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:05.146710  1483 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.204056  1330 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.234333  1483 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.235217  1330 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:05.121444  1347 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:05.084797  1285 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.095214  1536 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.159247  1335 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:05.156644  1197 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:05.186939  1566 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.000s
W20260709 12:27:05.207754  1432 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.861s	user 0.010s	sys 0.002s
I20260709 12:27:05.210018  1366 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.212273  1582 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.214448  1314 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.214643  1556 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.216460  1513 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.187729  1235 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:05.158193  1456 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:05.244184  1335 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.151309  1447 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.282888  1542 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:05.318367  1595 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:27:05.320850  1556 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.338050  1511 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.322466  1451 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.330564  1472 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.332060  1542 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.332459  1536 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.333747  1456 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:05.353835  1592 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.464s	user 0.000s	sys 0.008s
I20260709 12:27:05.362869  1599 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:05.123541 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 265088us
W20260709 12:27:05.412680 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.135352 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 257995us
I20260709 12:27:05.104287 26123 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:05.417520 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 476298us
W20260709 12:27:05.418190 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.128489 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.422251 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.421499 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:05.425231 26123 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20. Re-sending same reply.
I20260709 12:27:05.426451 26122 raft_consensus.cc:2410] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:27:05.419622 26093 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:05.428296 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.430326 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.150314 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1275) took 1333 ms. Trace:
I20260709 12:27:05.125725 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.476672 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:05.157709 26201 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:05.174779 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.484964 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.486265 26200 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:27:05.487648 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:05.490748  1542 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.492653  1482 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:05.492255  1542 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.324258  1375 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:05.505117 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.011s	sys 0.020s
W20260709 12:27:05.505833 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.011s	sys 0.020s
I20260709 12:27:05.508839  1447 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:05.436194 26128 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
W20260709 12:27:05.437199 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.437235 26127 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:05.432641 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2926) took 1413 ms. Trace:
I20260709 12:27:05.513494 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.512426 26121 rpcz_store.cc:276] 0709 12:27:04.018637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.444911 (+426274us) service_pool.cc:224] Handling call
0709 12:27:04.445244 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.450536 (+  5292us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:04.839685 (+389149us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.103066 (+263381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.103120 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:05.103163 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.423370 (+320207us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:05.423524 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.432390 (+  8866us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":489547,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":380,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4544,"spinlock_wait_cycles":11662720,"thread_start_us":637535,"threads_started":2,"wal-append.queue_time_us":311124}
I20260709 12:27:05.515244 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:05.442988 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.313010  1335 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.367185  1557 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.436831 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1370) took 1477 ms. Trace:
I20260709 12:27:05.432355 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:05.468113  1599 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:05.473727 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 279030us
I20260709 12:27:05.516227 26202 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
W20260709 12:27:05.503648 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70519.3us
I20260709 12:27:05.504010   984 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:05.549274 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:05.300619  1432 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.517175 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1297) took 2080 ms. Trace:
I20260709 12:27:05.552510 26789 rpcz_store.cc:276] 0709 12:27:03.436892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.437008 (+   116us) service_pool.cc:224] Handling call
0709 12:27:03.458296 (+ 21288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.485323 (+ 27027us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:03.891802 (+406479us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.596905 (+705103us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:04.612462 (+ 15557us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.612610 (+   148us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:04.612649 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.612730 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.352956 (+740226us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:05.516368 (+163412us) raft_consensus.cc:1685] finished
0709 12:27:05.516554 (+   186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.516867 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64341,"prepare.queue_time_us":377627,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":217849,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":22777,"spinlock_wait_cycles":16563072,"thread_start_us":828606,"threads_started":2,"wal-append.queue_time_us":844690}
I20260709 12:27:05.518929 26197 rpcz_store.cc:276] 0709 12:27:03.816850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.840834 (+ 23984us) service_pool.cc:224] Handling call
0709 12:27:03.841212 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.841584 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.841613 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.864750 (+ 23137us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.265039 (+400289us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:04.265091 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.265179 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.150070 (+884891us) raft_consensus.cc:1685] finished
0709 12:27:05.150092 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.150246 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":367246,"threads_started":1,"wal-append.queue_time_us":597317}
I20260709 12:27:05.557224 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.558135 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3888) took 1368 ms. Trace:
I20260709 12:27:05.524545  1297 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.559309 26120 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:05.559691  1297 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.474920 26203 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:27:05.522068  1197 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:05.550052 26202 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.561292 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2937) took 1459 ms. Trace:
I20260709 12:27:05.526077 26194 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:05.527410 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1277) took 1562 ms. Trace:
I20260709 12:27:05.530956 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:05.537918 26200 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:05.517236 26121 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: [20.11-22.12]
I20260709 12:27:05.489631  1285 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.521672 26124 rpcz_store.cc:276] 0709 12:27:03.959428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.960213 (+   785us) service_pool.cc:224] Handling call
0709 12:27:03.984841 (+ 24628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.445099 (+460258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.445143 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:04.445721 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.750717 (+304996us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:04.750761 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.750983 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.435427 (+684444us) raft_consensus.cc:1685] finished
0709 12:27:05.435805 (+   378us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.436360 (+   555us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150008,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":15336,"thread_start_us":764394,"threads_started":2,"wal-append.queue_time_us":490846}
I20260709 12:27:05.558722 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.536695 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:05.535888 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:05.528601 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105007us
I20260709 12:27:05.558588 26127 rpcz_store.cc:276] 0709 12:27:04.189773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.312135 (+122362us) service_pool.cc:224] Handling call
0709 12:27:05.557877 (+1245742us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001f6dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:05.558070 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2519955840}
I20260709 12:27:05.529322 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2949) took 1259 ms. Trace:
I20260709 12:27:05.542835 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58316 (request call id 35) took 1018 ms. Trace:
I20260709 12:27:05.432163 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:05.580296 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.535666 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:05.529525  1599 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:05.578759 26129 rpcz_store.cc:276] 0709 12:27:04.188788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.438126 (+1249338us) service_pool.cc:224] Handling call
0709 12:27:05.439375 (+  1249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.440033 (+   658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.440058 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.440120 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:05.440149 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.440223 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.448113 (+  7890us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.583912 26050 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:05.584801  1197 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.580961  1326 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.586675  1326 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.587486  1326 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.588402 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3889) took 1388 ms. Trace:
I20260709 12:27:05.588546 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:05.588848 26121 rpcz_store.cc:276] 0709 12:27:04.190082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.515941 (+1325859us) service_pool.cc:224] Handling call
0709 12:27:05.516643 (+   702us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:05.574379 (+ 57736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.578905 (+  4526us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.589336 26194 raft_consensus.cc:2376] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:27:05.589833  1449 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.517839  1288 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:05.592679 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62161.2us
I20260709 12:27:05.593022  1288 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.593631  1556 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:05.593961   984 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:05.533732  1542 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.595064  1413 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:05.596042  1413 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:05.596341  1086 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:05.597214  1086 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:05.597596  1413 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.583855 26053 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:05.599836 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:05.529379 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1378) took 1512 ms (client timeout 1935 ms). Trace:
I20260709 12:27:05.600832 26203 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:05.596354  1335 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.601315  1560 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:05.583820  1599 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.603833  1599 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:05.579286 26125 rpcz_store.cc:276] 0709 12:27:04.524699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.526326 (+  1627us) service_pool.cc:224] Handling call
0709 12:27:05.541929 (+1015603us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54000e9320. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b74322520 00007f5b7439f652 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:05.542552 (+   623us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621401344}
I20260709 12:27:05.602990  1611 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:05.601265  1366 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:05.524760 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.570533 26120 rpcz_store.cc:276] 0709 12:27:04.101951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.610473 (+508522us) service_pool.cc:224] Handling call
0709 12:27:05.558946 (+948473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.559813 (+   867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.560953 (+  1140us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54001f6dd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:05.561204 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180284672}
I20260709 12:27:05.571563 26201 rpcz_store.cc:276] 0709 12:27:03.964666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.964883 (+   217us) service_pool.cc:224] Handling call
0709 12:27:03.965273 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.480184 (+1514911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.526835 (+ 46651us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002b4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:05.527153 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2741662080}
I20260709 12:27:05.599160  1086 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.595453   984 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:05.600013  1413 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:05.585592 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.524224  1557 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:05.601086 26128 rpcz_store.cc:269] 0709 12:27:04.017444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.017627 (+   183us) service_pool.cc:224] Handling call
0709 12:27:05.529065 (+1511438us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002ca6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:05.529283 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3234493824}
I20260709 12:27:05.595654 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:05.620433 26199 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:05.533380 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:05.624151 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.036s	sys 0.036s
I20260709 12:27:05.621110 26197 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:27:05.627633 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.036s	sys 0.037s
I20260709 12:27:05.612507  1297 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.612296   984 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:05.636380  1472 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:05.638036  1472 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:05.638590  1472 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:05.534654  1593 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.640600  1542 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.641382  1542 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.643000  1593 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.643781  1542 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.604869  1087 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:27:05.650876 26123 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:05.660617 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.666029 26124 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:05.673147 26202 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:05.675392  1456 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.648905   984 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.676318  1456 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.677781 26194 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:05.666626 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:05.662246  1297 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.690871 26198 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:27:05.693392  1483 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:05.693845  1197 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.694125  1483 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:05.695390  1483 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.678786 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:05.699934 26200 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:05.701210 26201 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:27:05.639983  1472 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.702463 26197 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:05.707891 26054 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:05.705332 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.716022 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:27:05.720268 26048 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:27:05.720929 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 187986us
I20260709 12:27:05.722334 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:05.723093  1197 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.723979  1197 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.720333 26120 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:05.728163 26202 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [18.13-21.14]
I20260709 12:27:05.729715 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3899) took 1533 ms. Trace:
I20260709 12:27:05.730264 26120 rpcz_store.cc:276] 0709 12:27:04.196792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.719423 (+1522631us) service_pool.cc:224] Handling call
0709 12:27:05.720007 (+   584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.729378 (+  9371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.729640 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:05.732414 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78529.6us
I20260709 12:27:05.732820  1536 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:05.733314  1536 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:05.733724  1536 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.733944 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:05.735872  1052 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:05.736400  1052 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.736815  1052 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:05.723382 26129 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:05.741796 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3895) took 1550 ms. Trace:
I20260709 12:27:05.746518  1288 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.757977  1456 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.758576 26123 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:05.742707 26129 rpcz_store.cc:276] 0709 12:27:04.191578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.695245 (+1503667us) service_pool.cc:224] Handling call
0709 12:27:05.722866 (+ 27621us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:27:05.723128 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.741010 (+ 17882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.741672 (+   662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7522176}
I20260709 12:27:05.803216 26128 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:05.804068 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3893) took 1612 ms. Trace:
I20260709 12:27:05.758970  1573 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:05.766232  1617 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:27:05.770206 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:05.770582 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:05.770843  1554 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.014s
I20260709 12:27:05.773612 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:05.774451 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:05.776170  1615 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.198s	user 0.005s	sys 0.000s
I20260709 12:27:05.776561  1432 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.776660 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2530) took 1288 ms. Trace:
I20260709 12:27:05.777700  1326 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.780958   984 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:05.790827 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3902) took 1593 ms. Trace:
I20260709 12:27:05.796362 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1380) took 1546 ms. Trace:
I20260709 12:27:05.821440 26120 rpcz_store.cc:276] 0709 12:27:04.252030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.795145 (+1543115us) service_pool.cc:224] Handling call
0709 12:27:05.795589 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.795909 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.795938 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.796016 (+    78us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:05.796053 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.796123 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.796270 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.798496 26125 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:05.781140  1297 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.781464  1335 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.804507 26128 rpcz_store.cc:276] 0709 12:27:04.191106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.692938 (+1501832us) service_pool.cc:224] Handling call
0709 12:27:05.802312 (+109374us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:27:05.802665 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.803719 (+  1054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.803980 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17460096}
I20260709 12:27:05.814498 26198 rpcz_store.cc:276] 0709 12:27:04.488426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.488565 (+   139us) service_pool.cc:224] Handling call
0709 12:27:04.488901 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.691520 (+1202619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.776269 (+ 84749us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:05.776561 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2753369984}
I20260709 12:27:05.826772 26124 rpcz_store.cc:276] 0709 12:27:04.197598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.789024 (+1591426us) service_pool.cc:224] Handling call
0709 12:27:05.789880 (+   856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.790307 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.790346 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.790438 (+    92us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:05.790485 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.790563 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.790723 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.821453 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:05.826464 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.784279  1482 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:05.830163 26195 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:05.830816 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 235313us
I20260709 12:27:05.831247 26051 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:05.824659  1554 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.834828 26049 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:05.816033  1574 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:05.837383 26048 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:05.837541  1482 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:05.839218  1482 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:05.841476 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.828809 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1381) took 1505 ms. Trace:
I20260709 12:27:05.860685 26125 rpcz_store.cc:276] 0709 12:27:04.319019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.797646 (+1478627us) service_pool.cc:224] Handling call
0709 12:27:05.798083 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.822888 (+ 24805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.824502 (+  1614us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.870628  1235 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.872150 26122 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:27:05.862349 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:05.875370 26120 raft_consensus.cc:2376] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23. Re-sending same reply.
I20260709 12:27:05.876909 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:05.878391 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:05.880770 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.883955 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:05.884369 26790 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:05.884919 26782 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:05.886193 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:05.889576 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:05.890216  1447 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:05.890694  1087 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:05.891505 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:05.892333  1574 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:05.898407 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:05.899642 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53654.3us
I20260709 12:27:05.900033 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.901573 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:05.902787  1432 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:05.903954  1593 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:05.904564  1593 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.904969  1471 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.906527 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:05.914263 26782 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:27:05.916575  1554 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:05.918371 26789 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:05.918789  1482 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:05.920673 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:05.921162 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:05.921479 26790 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:05.923139 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:05.923882 26791 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:05.903265 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:05.903436  1618 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:27:05.919776  1197 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.932542 26051 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:05.848099  1599 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.937772 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:05.865862 26169 connection.cc:256] RPC call timeout handler was delayed by -1.4401199300000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
I20260709 12:27:05.939119 26195 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:05.936714  1573 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:05.904992  1593 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.941044 26757 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:05.947046 26124 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 12:27:05.947465 26124 pending_rounds.cc:85] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 12
I20260709 12:27:05.762912  1052 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.946651 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:05.937287  1574 raft_consensus.cc:2764] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:27:05.947708 26124 pending_rounds.cc:107] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 12:27:05.960649 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:05.961129 26124 raft_consensus.cc:2889] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.961665 26195 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:05.962380 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2953) took 1515 ms. Trace:
I20260709 12:27:05.962785 26124 rpcz_store.cc:276] 0709 12:27:04.446772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.829427 (+1382655us) service_pool.cc:224] Handling call
0709 12:27:05.936675 (+107248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.962078 (+ 25403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.962301 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.964726  1332 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.965329  1332 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:05.965880  1052 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:05.967954 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2948) took 1779 ms. Trace:
I20260709 12:27:05.968415 26122 rpcz_store.cc:276] 0709 12:27:04.188546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.432850 (+1244304us) service_pool.cc:224] Handling call
0709 12:27:05.434017 (+  1167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.872805 (+438788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.967639 (+ 94834us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:05.967881 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006588672}
I20260709 12:27:05.969869 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.971091 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:05.972586  1288 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:05.973416  1288 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:05.975698 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3807) took 1098 ms. Trace:
I20260709 12:27:05.976563 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:05.977489 26790 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:05.976222 26049 rpcz_store.cc:276] 0709 12:27:04.877217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.877666 (+   449us) service_pool.cc:224] Handling call
0709 12:27:04.939517 (+ 61851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.835267 (+895750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.975392 (+140125us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54000f8f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:05.975617 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057967360}
W20260709 12:27:05.975687 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84361.6us
W20260709 12:27:05.981139 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.963893 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.982497 26791 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:05.983398 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:05.985453 26122 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:05.986056 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 275108us
I20260709 12:27:05.998164 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:05.944509  1573 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:06.006599 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:05.981528  1432 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:05.975752  1288 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:05.984433  1332 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.012466 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.010257  1573 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:06.022029 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.024086 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.025108  1449 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:06.025473 26791 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:06.025898  1449 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.029554 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:06.031589  1472 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:06.035588  1574 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:06.011721  1432 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.037820 26124 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:06.040690 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:06.053258 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.051965 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.057894 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.043921  1432 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:06.038980  1574 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:06.060287 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.061201 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.037453  1471 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:06.057739 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.060109 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.061196 26195 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:06.070600 26125 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:06.065235  1086 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:06.075160 26126 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:06.066121 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.044888  1235 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.068840 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.071159 26122 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:06.081099 26790 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:06.075377 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.083894 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.085335 26046 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:06.058037  1326 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:06.083607 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.068956  1574 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:06.062115  1449 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.073611 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.088984 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:06.094121 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.094687 26048 raft_consensus.cc:2368] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23.
I20260709 12:27:06.096343 26049 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:06.097303 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 162520us
I20260709 12:27:06.046738  1052 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:27:06.099128 26052 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:27:06.096488 26196 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
W20260709 12:27:06.106194 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44232.8us
I20260709 12:27:06.106326  1332 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:06.106429  1455 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.107702  1620 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:06.108286  1572 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:06.110917 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 69480.9us
I20260709 12:27:06.112205 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.120198 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.124157 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.121218 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.124897 26198 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:06.113363  1332 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:06.120412 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 121825us
I20260709 12:27:06.131160 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.132735 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:06.132853 26126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 2928) took 2113 ms (client timeout 2175 ms). Trace:
W20260709 12:27:06.133298 26126 rpcz_store.cc:269] 0709 12:27:04.019287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.037277 (+ 17990us) service_pool.cc:224] Handling call
0709 12:27:06.132639 (+2095362us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:06.132797 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4212729728}
I20260709 12:27:06.134272 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:06.134958 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.136323 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:06.136611 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.136494 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.144083 26198 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:06.145375 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.146730 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.120463  1599 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.138597 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.139576 26050 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:27:06.120383  1235 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:06.140326 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:06.134958 26125 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:06.121464  1572 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:06.149351 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.141544 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.165706 26790 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:06.143191 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.144775  1332 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:06.138501 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.130318 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.147586 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.148388 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:06.168923 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.195151 26051 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:06.193396 26047 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 25 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:27:06.192976 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71785.4us
I20260709 12:27:06.198428 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.197530 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:06.152563 26053 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:06.203426 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1385) took 1796 ms. Trace:
I20260709 12:27:06.194221 26124 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:06.211244 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.120891  1326 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.194994  1447 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:06.158480  1449 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:06.213323 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1382) took 1894 ms. Trace:
I20260709 12:27:06.218324 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:06.221131 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59053.3us
I20260709 12:27:06.156965  1297 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.223742 26195 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:06.224927 26048 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 24 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:06.224273 26121 rpcz_store.cc:276] 0709 12:27:04.319360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.800098 (+1480738us) service_pool.cc:224] Handling call
0709 12:27:05.800648 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.801158 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:06.000410 (+199252us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.212944 (+212534us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.213002 (+    58us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:06.213058 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.213123 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.213259 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":307789,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":6712,"thread_start_us":351973,"threads_started":2}
I20260709 12:27:06.129665  1455 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.206878 26128 rpcz_store.cc:276] 0709 12:27:04.366782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.848988 (+1482206us) service_pool.cc:224] Handling call
0709 12:27:05.872692 (+ 23704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.873177 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.873209 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:05.874019 (+   810us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.076683 (+202664us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:06.076728 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.076796 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.163312 (+ 86516us) raft_consensus.cc:1685] finished
0709 12:27:06.163332 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.163486 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":229,"thread_start_us":197369,"threads_started":1,"wal-append.queue_time_us":266186}
I20260709 12:27:06.245867 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:06.253204 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:06.231995 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.193991  1542 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:06.157425  1235 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:06.250995 26169 connection.cc:256] RPC call timeout handler was delayed by -1.3257768339999529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 12:27:06.253012 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 223874us
I20260709 12:27:06.263862 26195 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:06.206214  1573 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.268208  1573 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.268908  1573 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:06.269641  1573 consensus_queue.cc:1243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:06.270671 26786 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:06.265801 26169 connection.cc:256] RPC call timeout handler was delayed by -1.299211552000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2667s for a response
W20260709 12:27:06.271420 26169 connection.cc:256] RPC call timeout handler was delayed by -1.2304907349998757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 12:27:06.271754 26169 connection.cc:256] RPC call timeout handler was delayed by -1.1494611120001537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27480000000000004s for a response
W20260709 12:27:06.272018 26169 connection.cc:256] RPC call timeout handler was delayed by -1.1275819939996836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
I20260709 12:27:06.272749  1288 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.273317  1288 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.275518  1288 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.213104  1197 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
W20260709 12:27:06.279553 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 164287us
I20260709 12:27:06.252722  1554 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:06.223243  1449 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.287617 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.288388 26121 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:06.292671 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:27:06.277585  1456 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:06.290464  1087 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.341094  1087 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:06.352406 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62640us
I20260709 12:27:06.353336 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:06.353794 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 156388us
I20260709 12:27:06.354079 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.354262 26126 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:06.356426  1556 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:06.358064  1556 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:06.359279  1052 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:06.359905  1052 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.360368  1052 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:06.361147 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:06.362381 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68985.5us
I20260709 12:27:06.363282 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:06.256922  1574 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.365805  1288 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:06.366236  1288 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:06.366580  1288 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:06.360396 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.372984 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1384) took 2006 ms. Trace:
I20260709 12:27:06.355652 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.387893 26126 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:06.391690 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:06.392671 26125 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:06.385321 26123 rpcz_store.cc:276] 0709 12:27:04.366427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.057036 (+1690609us) service_pool.cc:224] Handling call
0709 12:27:06.057567 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.095318 (+ 37751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.095350 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.096049 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.257238 (+161189us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:06.257296 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.257380 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.372773 (+115393us) raft_consensus.cc:1685] finished
0709 12:27:06.372794 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.372918 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1566,"raft.queue_time_us":11056,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":13841,"thread_start_us":160761,"threads_started":1,"wal-append.queue_time_us":237765}
I20260709 12:27:06.383920 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.395830 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1383) took 2054 ms. Trace:
I20260709 12:27:06.396853 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.290670  1449 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:06.353361  1625 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:27:06.368172  1574 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.396361 26129 rpcz_store.cc:276] 0709 12:27:04.341088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.803646 (+1462558us) service_pool.cc:224] Handling call
0709 12:27:05.804152 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.804737 (+   585us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:06.110312 (+305575us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.395440 (+285128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.395507 (+    67us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:06.395570 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.395640 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.395772 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":387808,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":519466,"threads_started":2}
I20260709 12:27:06.418071 26787 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:27:06.428224 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.432837 26124 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:06.295774  1554 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:06.334290  1456 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:06.438212 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.394843  1087 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.352406  1235 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.376423  1593 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.359362  1556 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:27:06.449234 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53321.7us
I20260709 12:27:06.417852  1599 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:06.440677  1456 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:06.430662  1560 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:06.396090  1482 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:27:06.457818 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:06.464388 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 110499us
I20260709 12:27:06.450717 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.408875  1326 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:06.474149 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.466707 26128 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:06.294024  1572 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.461258  1560 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.502836  1560 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:06.508530 26126 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
W20260709 12:27:06.509683 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 219045us
W20260709 12:27:06.464665  1536 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.730s	user 0.018s	sys 0.000s
I20260709 12:27:06.510728 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:06.511384 26126 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:06.511765 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 86791us
W20260709 12:27:06.491163  1629 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:27:06.493675 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:06.501816  1593 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.484479  1574 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:27:06.465193  1599 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:06.491709 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 194) took 22815 ms (client timeout 15000 ms). Trace:
I20260709 12:27:06.513679 26128 raft_consensus.cc:2376] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:27:06.479699 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:06.453227  1556 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:06.513532  1447 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:06.539363 26124 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:06.540165 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3907) took 1019 ms. Trace:
I20260709 12:27:06.540606 26124 rpcz_store.cc:276] 0709 12:27:05.521264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.538620 (+1017356us) service_pool.cc:224] Handling call
0709 12:27:06.539024 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.539865 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.540072 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:06.541743 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:06.529601  1235 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:06.511030  1536 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.545720 26125 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:06.546540 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1387) took 2016 ms. Trace:
I20260709 12:27:06.546921 26125 rpcz_store.cc:276] 0709 12:27:04.530489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.534070 (+2003581us) service_pool.cc:224] Handling call
0709 12:27:06.542364 (+  8294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.546257 (+  3893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.546481 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:06.543996  1451 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.553586  1451 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.558404 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:06.562794 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 97460.5us
I20260709 12:27:06.564285 26129 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:27:06.566545  1449 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.568444  1638 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.568714  1556 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.569077  1638 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.570207  1556 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:06.571597  1638 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.572610  1447 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.574393 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.575837 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.576689 26195 raft_consensus.cc:2368] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23.
I20260709 12:27:06.576397  1599 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.577637 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.578398 26199 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:06.579309 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.531700 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:06.580189 26128 raft_consensus.cc:606] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:06.580708 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.581544 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:06.581913 26127 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:06.582928  1560 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.544643  1297 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:06.584661  1297 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:06.585731  1297 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.587231  1297 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:06.546478  1572 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:06.546124  1235 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:06.588740  1235 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:06.553573  1471 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
W20260709 12:27:06.555598  1472 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.005s
I20260709 12:27:06.582134 26128 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:06.594741 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:06.596863 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.599502 26126 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:06.555289  1593 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:06.551673  1288 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.582087  1536 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.601080 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:06.593210  1432 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
I20260709 12:27:06.612535 26120 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:06.611791 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.614580 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.614687 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.615191 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.615372 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:06.614573  1413 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:06.616413 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.617347  1542 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.617892 26047 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:06.619081 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.619757 26045 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:06.617902  1542 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:06.622759 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.623193  1635 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:27:06.624584  1640 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:06.625633  1640 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:06.598067  1197 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:06.563334  1455 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.630873  1639 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.631141  1086 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:06.631831  1086 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.623203 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:06.633554 26127 raft_consensus.cc:606] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:06.633203 25947 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:06.634418  1086 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:06.562996  1554 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.641197  1052 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.641690 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.643750  1052 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.613250  1332 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.647645 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:27:06.649011 26053 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:06.652347  1432 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.670789 26789 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:06.671686 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:27:06.672356 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.673116 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.673831 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.675014 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:06.673116 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.664341 26787 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:06.677168  1456 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.675663 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.678449 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.679586  1332 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:06.680855 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 139212us
I20260709 12:27:06.684052 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1388) took 2153 ms. Trace:
I20260709 12:27:06.720521 26789 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:06.720970 26129 rpcz_store.cc:276] 0709 12:27:04.530834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.536963 (+2006129us) service_pool.cc:224] Handling call
0709 12:27:06.543477 (+  6514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.593544 (+ 50067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.683770 (+ 90226us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:06.683984 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47073152}
I20260709 12:27:06.684984  1449 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.608556 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.687741  1451 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.688437 26125 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:06.631598  1639 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.699359 26127 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:27:06.700181 26128 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
W20260709 12:27:06.720620 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109099us
I20260709 12:27:06.684542 26123 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:06.751365 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1386) took 2243 ms. Trace:
I20260709 12:27:06.751873 26123 rpcz_store.cc:276] 0709 12:27:04.507529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.532072 (+2024543us) service_pool.cc:224] Handling call
0709 12:27:06.552821 (+ 20749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.684362 (+131541us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b54000e9320. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:27:06.750993 (+ 66631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.751286 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93278080}
W20260709 12:27:06.516805 25946 rpcz_store.cc:269] 0709 12:26:43.676649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.952651 (+1276002us) service_pool.cc:224] Handling call
0709 12:27:06.489091 (+21536440us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:01.783335 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:01.797393 (+ 14058us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:01.797467 (+    74us) write_op.cc:435] Acquired schema lock
0709 12:27:01.797488 (+    21us) tablet.cc:662] Decoding operations
0709 12:27:01.799110 (+  1622us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:01.799259 (+   149us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:01.799295 (+    36us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:27:01.867444 (+ 68149us) tablet.cc:701] Row locks acquired
0709 12:27:01.874290 (+  6846us) write_op.cc:260] PREPARE: finished
0709 12:27:01.891958 (+ 17668us) write_op.cc:270] Start()
0709 12:27:01.933350 (+ 41392us) write_op.cc:276] Timestamp: P: 1783600021877630 usec, L: 0
0709 12:27:01.943015 (+  9665us) op_driver.cc:348] REPLICATION: starting
0709 12:27:01.985012 (+ 41997us) log.cc:844] Serialized 5958 byte log entry
0709 12:27:02.502656 (+517644us) op_driver.cc:464] REPLICATION: finished
0709 12:27:02.626653 (+123997us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b1800116d40. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:02.785250 (+158597us) write_op.cc:301] APPLY: starting
0709 12:27:02.799144 (+ 13894us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:03.308764 (+509620us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:03.308811 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:03.836810 (+527999us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b1800116d40. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:03.840429 (+  3619us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:03.840814 (+   385us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:27:03.840875 (+    61us) write_op.cc:312] APPLY: finished
0709 12:27:03.841290 (+   415us) log.cc:844] Serialized 139 byte log entry
0709 12:27:04.598783 (+757493us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.660610 (+ 61827us) write_op.cc:454] Released schema lock
0709 12:27:04.740832 (+ 80222us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:04.741423 (+   591us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.741472 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3828402,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":229395,"apply.run_cpu_time_us":59704,"apply.run_wall_time_us":1969399,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66917,"num_ops":15,"prepare.queue_time_us":217849,"prepare.run_cpu_time_us":24226,"prepare.run_wall_time_us":426329,"raft.queue_time_us":157005,"raft.run_cpu_time_us":10749,"raft.run_wall_time_us":222897,"replication_time_us":541847,"spinlock_wait_cycles":49787008,"thread_start_us":1135204,"threads_started":5,"wal-append.queue_time_us":1354063,"wal-append.run_cpu_time_us":14996,"wal-append.run_wall_time_us":2498813}]]}
I20260709 12:27:06.690853 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.735219  1646 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:27:06.735013 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 181374us
I20260709 12:27:06.749660 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.747567 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:06.753064 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 144588us
I20260709 12:27:06.752561 26782 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:06.755272 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.758407 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:06.758670 26790 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:06.756052 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:06.760387 26195 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:06.764461 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.757440  1646 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:06.767735 26048 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:06.767450  1646 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.769148 26046 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:06.769451  1332 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:06.769975  1332 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:06.770450 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.769085 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.670874  1432 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:06.627909  1542 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.774106 26128 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:27:06.775043 26128 pending_rounds.cc:85] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:27:06.775337 26128 pending_rounds.cc:107] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:27:06.775629 26128 raft_consensus.cc:2889] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.777014 26121 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:06.727252  1413 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:06.778445 26784 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:06.779155 26125 raft_consensus.cc:1217] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 13.11->[14.12-24.17]   Dedup: 16.13->[18.14-24.17]
I20260709 12:27:06.739814  1556 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:06.779345 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.739101  1197 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:06.790789 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50424.4us
W20260709 12:27:06.792037 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 253168us
I20260709 12:27:06.793581 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.794970 26790 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:06.782217 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.793380  1052 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:06.800473  1556 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:06.801487  1556 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.803081 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.803877 26790 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:06.804783  1560 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.781430 26125 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 13
I20260709 12:27:06.806891 26125 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:27:06.791570 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
W20260709 12:27:06.807572 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.804852 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.807204 26125 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.808472 26125 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:27:06.808758 26125 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.809876 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.815017 26045 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:06.822072 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.841997 26127 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
W20260709 12:27:06.847678 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56594.7us
W20260709 12:27:06.854944 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45129.3us
I20260709 12:27:06.855183 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.858206 26783 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:06.859344 26785 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:06.790249  1332 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:06.859963 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.860787 26053 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:06.853863 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:06.864336 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:06.866113 26195 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:06.868430 26784 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:27:06.801525  1456 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:06.869261 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.870831 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.863941  1326 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:06.873554 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.874363 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:06.882058 26127 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:27:06.887745 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:06.873232 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62598.5us
I20260709 12:27:06.874185 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.891705 26788 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:06.893186 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:06.901077 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 292188us
I20260709 12:27:06.903843 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:06.903679 26203 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
W20260709 12:27:06.905186 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.905848 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.869611  1572 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:06.927506 26047 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
W20260709 12:27:06.903057 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.912700 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:06.913348 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:06.932181 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:06.932674 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.880762  1638 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:06.931649 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.916565 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:06.916939 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:06.860509  1640 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:27:06.917868 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:06.962915 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:06.963821 26195 raft_consensus.cc:2368] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20.
I20260709 12:27:06.855683  1086 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:27:06.918753 26785 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:06.908923  1332 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:06.921968 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:06.967698 26053 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:06.922624 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:06.913877  1472 raft_consensus.cc:993] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:27:06.915536  1413 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:06.876413  1449 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.934963 26202 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 18: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:06.874049  1288 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:06.936349 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:06.876513  1326 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:06.966941  1455 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:06.960772 26200 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
W20260709 12:27:06.969336 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78415.7us
W20260709 12:27:06.969671 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52905.7us
I20260709 12:27:06.909992  1593 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:07.039002 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:07.004561 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:07.039880 26203 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:06.969069  1332 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.999465 26198 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
W20260709 12:27:07.009878 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148917us
I20260709 12:27:07.020124 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:07.042800 26195 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:06.959589 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:06.923272  1087 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.096s
W20260709 12:27:07.039245 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.050962 26194 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:07.050868 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:07.053130 26785 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:06.919945  1611 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:27:07.006934  1432 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:07.065176 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:07.066138 26196 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:06.966914   984 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:27:07.065426 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:07.077616 26202 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:07.068724 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:07.075126 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.076118 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48644us
I20260709 12:27:07.039500  1451 raft_consensus.cc:1081] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Signalling peer a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:27:06.913376  1197 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.039490  1472 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:07.057587  1332 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.020157  1449 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.113178 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:07.114732 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:07.068755  1572 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:07.123394 26202 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:07.126650 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:07.127588 26202 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:07.091863  1432 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:07.191612 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:07.201277 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 111793us
I20260709 12:27:07.204133 26203 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:07.207824 26202 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:07.169271  1611 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.112651  1472 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:07.223526 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.211200  1662 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:27:07.221227  1643 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:27:07.214370  1644 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:07.212306  1639 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:07.234432  1235 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.001s
I20260709 12:27:07.238554 26783 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:07.245782 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:07.242987 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 162127us
I20260709 12:27:07.257268 26053 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:27:07.229889  1662 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:07.244216  1628 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:27:07.262246 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 149580us
W20260709 12:27:07.262192 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.227999  1611 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:07.265133  1654 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:27:07.267410 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 180487us
W20260709 12:27:07.268911  1648 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:27:07.268378  1432 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.253288  1288 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:07.270946  1650 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:27:07.221973  1655 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 12:27:07.293267 26789 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.003s
I20260709 12:27:07.208714  1326 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:07.293903  1649 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 12:27:07.335589 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:07.337932 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2965) took 1472 ms. Trace:
I20260709 12:27:07.343089  1472 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.259378 26127 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:07.273770  1572 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.223780  1593 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.225533  1670 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 12:27:07.320026 26789 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:07.320441 26129 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:07.249212  1235 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.338622 26124 rpcz_store.cc:276] 0709 12:27:05.865175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.544138 (+678963us) service_pool.cc:224] Handling call
0709 12:27:06.560545 (+ 16407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.608412 (+ 47867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.608447 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.643470 (+ 35023us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.961672 (+318202us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:06.961732 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.961814 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.337589 (+375775us) raft_consensus.cc:1685] finished
0709 12:27:07.337622 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.337842 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6600,"raft.queue_time_us":17245,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":317752,"threads_started":1,"wal-append.queue_time_us":438774}
I20260709 12:27:07.344859 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2967) took 1226 ms. Trace:
I20260709 12:27:07.314128  1667 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.354979 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 313270us
I20260709 12:27:07.317209  1288 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:07.359227 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:07.359863 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:07.347870  1297 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.757s	user 0.009s	sys 0.006s
I20260709 12:27:07.339820  1638 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:07.378114 26123 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:07.351148 26120 rpcz_store.cc:276] 0709 12:27:06.118119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.689065 (+570946us) service_pool.cc:224] Handling call
0709 12:27:06.689502 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.689923 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.689955 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.690597 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.869112 (+178515us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:06.869156 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.869266 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.344621 (+475355us) raft_consensus.cc:1685] finished
0709 12:27:07.344655 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.344797 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16997,"raft.queue_time_us":30647,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":19676,"thread_start_us":171922,"threads_started":1,"wal-append.queue_time_us":278478}
I20260709 12:27:07.376190 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.383052 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:07.383922 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:07.385035 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:07.385696 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:07.386183 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:07.387315 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:07.388278 26046 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:07.389689 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:07.416821  1662 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.392118 26050 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
W20260709 12:27:07.449024 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.419600  1667 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.383594  1472 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.421579  1456 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:07.380033  1638 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:07.422510 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.422454  1235 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.383257  1197 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.459960 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:07.372699  1288 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:07.415185  1676 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.412696  1678 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:07.470686  1668 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:07.474812  1676 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.476799  1662 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:07.484866 26048 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
W20260709 12:27:07.497344 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 121372us
I20260709 12:27:07.494505  1676 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.468034  1472 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.503646  1472 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.508965  1052 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.509486  1052 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.510353  1678 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.511749  1052 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.512799  1678 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.513089  1483 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:07.513550  1483 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:07.514539  1483 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.515628  1483 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:07.492420  1666 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:27:07.522809 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.522938 26121 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:27:07.524605 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:07.525558 26120 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:07.526962 26094 leader_election.cc:400] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23.
I20260709 12:27:07.527297 26094 leader_election.cc:403] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:07.528653  1455 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:07.529220  1455 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:07.530865 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.539203 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:07.546062 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 160453us
I20260709 12:27:07.548197 26196 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:07.549866 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.550449 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.551940 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:07.549027 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43705us
W20260709 12:27:07.562366 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.565202 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:07.569026  1482 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:07.570574 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:07.573055  1332 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.574820 26201 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:07.576519  1086 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:07.577806  1670 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:07.578325  1670 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:07.480391  1197 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.582340  1197 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.582520  1678 raft_consensus.cc:993] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:27:07.564734  1456 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23.
I20260709 12:27:07.583304  1678 raft_consensus.cc:1081] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
W20260709 12:27:07.587819  1679 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 12:27:07.480952  1449 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.579202  1670 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.601153 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:07.607808 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:07.599205  1087 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:07.609201  1087 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.609663  1087 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:07.621361 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:07.623768  1681 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:27:07.624388 26046 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:07.627166  1667 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.610435  1670 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:07.611055  1449 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:07.597765  1332 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:07.619112  1482 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:27:07.633172  1482 consensus_queue.cc:1243] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:07.618330  1447 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.021s
I20260709 12:27:07.645716 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:07.646492 26782 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:07.651273 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:07.655081 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89960.9us
I20260709 12:27:07.658177 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:07.646749 26120 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:07.661319 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:07.664261 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.669983  1677 raft_consensus.cc:2764] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:27:07.671103 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.672345 26046 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:27:07.674118 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:07.618705  1556 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:07.680125 26051 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:27:07.683671 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:07.684969 26047 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:07.677388  1667 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:07.703105 26046 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:07.677598  1449 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:07.678995  1326 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:07.704331  1288 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.709885  1288 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:07.713459 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.698828  1556 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.714152  1455 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.716317  1556 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.716293  1455 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.700994  1677 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.724068 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:07.734452 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.791s (SENT) [suppressed 38 similar messages]
I20260709 12:27:07.704440  1662 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:07.703197  1667 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.705965  1326 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.744804 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.707369  1554 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:07.744135  1667 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:07.747493  1599 raft_consensus.cc:993] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
I20260709 12:27:07.745872  1326 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:07.752655 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:07.761397 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.758250  1668 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:07.784583 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 2963) took 1940 ms. Trace:
I20260709 12:27:07.785857 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1401) took 1226 ms. Trace:
I20260709 12:27:07.786595 26125 rpcz_store.cc:276] 0709 12:27:06.559494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.771036 (+211542us) service_pool.cc:224] Handling call
0709 12:27:06.771416 (+   380us) raft_consensus.cc:1471] Updating replica for 6 ops
0709 12:27:07.345363 (+573947us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:07.380748 (+ 35385us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.692832 (+312084us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.693084 (+   252us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:27:07.694065 (+   981us) log.cc:844] Serialized 104 byte log entry
0709 12:27:07.694229 (+   164us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:07.694257 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.694331 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.785589 (+ 91258us) raft_consensus.cc:1685] finished
0709 12:27:07.785612 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.785800 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13351,"prepare.queue_time_us":39650,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":2063,"raft.queue_time_us":19228,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":13787,"spinlock_wait_cycles":12962048,"thread_start_us":696830,"threads_started":2,"wal-append.queue_time_us":344505}
I20260709 12:27:07.790119 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:07.791102  1556 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:07.785748 26122 rpcz_store.cc:276] 0709 12:27:05.843969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.538533 (+694564us) service_pool.cc:224] Handling call
0709 12:27:06.559091 (+ 20558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.616517 (+ 57426us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:06.616826 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.854440 (+237614us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.871613 (+ 17173us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.871752 (+   139us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:06.871781 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.871842 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.719232 (+847390us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b180048d720. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:07.776836 (+ 57604us) raft_consensus.cc:1685] finished
0709 12:27:07.784357 (+  7521us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.784513 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42648,"prepare.queue_time_us":1315,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":53222,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6988,"spinlock_wait_cycles":6240000,"thread_start_us":196912,"threads_started":1,"wal-append.queue_time_us":251786}
I20260709 12:27:07.750512  1235 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:27:07.793146 26049 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:27:07.748126  1662 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.795447  1662 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.774461  1472 raft_consensus.cc:1081] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Signalling peer 8fd5643e7b494fa79c3d0aaf1e6e8152 to start an election
W20260709 12:27:07.804674  1642 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.844s	user 0.009s	sys 0.004s
I20260709 12:27:07.785799  1297 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:27:07.805550  1611 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:07.806584  1656 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:07.806998  1656 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:07.795874  1691 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:07.804558  1644 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:07.799454 26787 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:07.739638  1677 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.760804  1087 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.808032 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1285) took 1155 ms. Trace:
I20260709 12:27:07.809194  1542 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:07.754971  1554 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:07.812867 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64935.4us
W20260709 12:27:07.813406 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69987.6us
W20260709 12:27:07.803035  1686 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.178s	user 0.000s	sys 0.002s
I20260709 12:27:07.816488 26199 rpcz_store.cc:276] 0709 12:27:06.652625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.676317 (+ 23692us) service_pool.cc:224] Handling call
0709 12:27:06.676710 (+   393us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:06.739902 (+ 63192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.739935 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:06.815962 (+ 76027us) log.cc:844] Serialized 78 byte log entry
0709 12:27:07.080758 (+264796us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:07.097308 (+ 16550us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.202052 (+104744us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.796537 (+594485us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b1800083080. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:27:07.796715 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.796793 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.807804 (+ 11011us) raft_consensus.cc:1685] finished
0709 12:27:07.807829 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.807973 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":676932,"prepare.queue_time_us":47897,"prepare.run_cpu_time_us":18141,"prepare.run_wall_time_us":68227,"raft.queue_time_us":794192,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":10849,"spinlock_wait_cycles":5430656,"thread_start_us":262822,"threads_started":2,"wal-append.queue_time_us":413777}
I20260709 12:27:07.848539 26756 leader_election.cc:400] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23.
I20260709 12:27:07.849788 26756 leader_election.cc:403] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:07.835862  1653 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.621s	user 0.000s	sys 0.001s
I20260709 12:27:07.816305  1638 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.852051  1677 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.863266 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.856441  1572 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:07.866359  1542 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:07.881596 26121 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:07.882100 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:07.816208  1656 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.908445  1656 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:07.881279  1676 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.887068  1542 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:07.889919  1554 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.912567  1676 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.872653  1644 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.895130  1611 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.896133  1087 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:07.897061  1413 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:07.897178  1663 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:07.921798  1572 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23.
I20260709 12:27:07.905160  1646 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:07.905012 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65836.3us
I20260709 12:27:07.887374 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.899210  1638 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.941713  1638 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:07.942327  1638 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:07.943388  1638 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.940515  1676 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:07.940864 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55600.3us
I20260709 12:27:07.936632 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.931556  1663 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:07.953279  1663 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.955507  1663 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:07.955717 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.956524  1087 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:07.956936 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.957823  1087 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:07.950095  1611 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.962658  1611 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:07.963371  1572 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:27:07.964391 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 303382us
I20260709 12:27:07.967406  1667 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.951320 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:07.969120  1593 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:07.969503 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:07.970274  1288 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:07.970332  1456 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:07.970810  1288 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:07.971071  1456 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:07.971186  1599 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:07.972116  1456 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:07.973767 26123 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:07.970625 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:07.974596 26124 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:07.975178 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 639733us
I20260709 12:27:07.971900  1288 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:07.985939 26125 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:07.986882  1640 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:07.987296  1640 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:07.987679  1640 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:07.964026 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 757081us
I20260709 12:27:07.984333 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:07.990409 26756 leader_election.cc:400] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 19. Current term is 20.
I20260709 12:27:07.991421 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:07.976387 26094 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
W20260709 12:27:07.977164 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.982285 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.967582  1432 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:07.983013  1574 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:07.983736 26129 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:07.984313 26123 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 12:27:07.984746  1670 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:08.005506 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.006397 26048 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:08.006830 26046 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:08.007031 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:07.993122 26094 leader_election.cc:403] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:08.010021 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:08.011314 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:08.017560  1536 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:08.018064  1536 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:08.018450  1536 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:08.021265 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.014655 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.024554 26046 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:08.034446 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.037231 26051 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.037906 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:08.038396 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.038506 26791 raft_consensus.cc:2376] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21. Re-sending same reply.
I20260709 12:27:08.039335 26787 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:08.040985 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:08.041258 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.041663 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.008178  1707 raft_consensus.cc:2764] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:08.043718 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.046581 26791 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:08.044113 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.023504  1052 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.047972 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:08.054669 26787 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:27:08.049618 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.008994  1670 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:08.049983 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.050720 26052 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [18.10-20.11]
I20260709 12:27:08.051741 26789 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:08.056133 26046 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:27:08.052986 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44347.1us
W20260709 12:27:08.059446 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.059368 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.059986 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.060370 26049 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:07.936435  1413 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.062348 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.072415 26196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:48540
I20260709 12:27:08.072755 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.073861 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.074733 26784 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:08.075850 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.076853 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.078023 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:27:08.080720 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.081358 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.082551 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.034327  1599 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:08.088933 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.091761 26790 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:27:08.092873 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.097715 26790 pending_rounds.cc:85] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 15
I20260709 12:27:08.095359 26194 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:08.034972  1677 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:08.097805 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.098040 26790 pending_rounds.cc:107] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:27:08.103029 26790 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.103497 26046 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:08.105608 26051 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:08.099594 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:08.100440 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52371.4us
I20260709 12:27:08.045876  1655 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:08.098304 26048 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
W20260709 12:27:08.109530 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.115886 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.117666 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.122102 26050 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:08.122696 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46407.6us
I20260709 12:27:08.124573 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.134928 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.145223 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.146880 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.147927 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.148643 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.149253 26051 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:08.152508 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.153563 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.148627 26046 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:27:08.083998  1644 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.156909 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.159507 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.161089 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.161798 26203 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:08.163272 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.164126 26200 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:08.166872 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.168056 26048 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:08.168787 26046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:27:08.169579 26046 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:08.043378  1668 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 12:27:08.176532 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.177174 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:08.124863  1655 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.165078 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.170468 26046 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:08.042404  1326 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.176131  1449 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:08.110733  1599 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:08.185627 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.188344 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:08.190019 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.214061 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.176637  1554 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:08.195649 26124 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
W20260709 12:27:08.196595 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 160656us
W20260709 12:27:08.176297  1701 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:27:08.218367 26123 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:08.218521 26094 leader_election.cc:400] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:08.219002 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.223037 26051 raft_consensus.cc:2368] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:08.220269 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:08.224186 26124 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:27:08.152758  1455 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:08.222910 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.196404  1655 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:08.227136 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.228741 26196 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:08.228860 26047 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:08.229215 26196 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:08.238320 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:08.238922 26047 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:08.220659  1297 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:08.230559 26054 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:47804
I20260709 12:27:08.230775 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:08.240947 26054 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:08.241066 26048 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:08.241365 26054 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:08.248860 26047 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
W20260709 12:27:08.251435 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 86498.5us
I20260709 12:27:08.138535  1611 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 12:27:08.212752  1449 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:08.253078 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 86763.7us
I20260709 12:27:08.217689  1456 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:08.259373 26203 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:08.257277 26054 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.251103 26196 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.225476  1682 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:08.264478 26049 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:08.270323 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.271050 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.272043 26199 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:27:08.250828  1678 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:08.272401 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.277145 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:08.191831  1644 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.274646 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.282634 26203 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:08.275207 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.251498  1235 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:08.290377  1235 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:08.290874  1235 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.292754 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.298101  1554 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:08.261226  1449 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:08.176474  1670 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.260762  1432 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:08.288399  1715 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:08.232432  1455 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.289757  1288 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:08.298501 26129 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:27:08.292017 26019 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.600s (SENT)
I20260709 12:27:08.315801 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.316061 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.268922  1682 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:08.281395 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.318765 26199 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:08.319092 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.320710 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.320791 26197 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:08.321376 26195 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
I20260709 12:27:08.322889 26201 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:08.299839  1554 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:27:08.315032 26054 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:08.276763  1678 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:08.326426 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:08.331847 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.342190 26123 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:08.359278 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 130608us
I20260709 12:27:08.323308  1682 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.342794  1678 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.399789 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.343119  1542 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.339362  1611 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:08.402956 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.405381 26201 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19. Re-sending same reply.
W20260709 12:27:08.417707 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 125732us
I20260709 12:27:08.422371 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.417135 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.423540 26203 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
W20260709 12:27:08.422574 26019 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.622s (SENT)
I20260709 12:27:08.428843 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.430511 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.317994  1432 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.451071 26203 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:08.314278  1668 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:08.417146  1697 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:08.454090 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.455922  1052 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:08.456452  1712 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:08.456804  1052 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:08.457376  1052 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:08.458308  1052 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.421468  1542 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.460646  1676 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:08.461422  1455 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:08.462044 26019 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.635s (SENT)
I20260709 12:27:08.462730 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:08.463930 26019 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.643s (SENT)
I20260709 12:27:08.464762 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.465571 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:08.465938  1646 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.003s
I20260709 12:27:08.466944  1644 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:08.453439 26194 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:08.461346  1542 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:08.487321 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.462116  1455 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:08.455830 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:08.477070  1663 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 12:27:08.488344 26200 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:08.489112 26050 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:08.492311 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.508134 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.508790 26195 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:08.514796 26123 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:08.518033 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.518718 26200 raft_consensus.cc:2376] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24. Re-sending same reply.
I20260709 12:27:08.519695 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.523540 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.526105 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.529320 26201 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:08.394328  1599 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.455852  1715 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
I20260709 12:27:08.486472  1676 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.553696 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:08.543613 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 87868.9us
I20260709 12:27:08.515842  1560 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:08.529583  1455 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:08.573374 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:08.541831  1536 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.002s
I20260709 12:27:08.574388  1536 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.575053 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.575892  1644 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:08.576478  1644 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:08.576565 26196 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:08.576910  1644 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.528620  1326 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:08.577322 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.547209 26050 raft_consensus.cc:1217] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 19.14->[21.15-24.17]   Dedup: 21.15->[22.16-24.17]
I20260709 12:27:08.579105 26050 pending_rounds.cc:85] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 15
I20260709 12:27:08.579430 26050 pending_rounds.cc:107] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:27:08.579731 26050 raft_consensus.cc:2889] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.564596 26195 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 12:27:08.545766 26203 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:08.570518  1689 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.945s	user 0.000s	sys 0.005s
I20260709 12:27:08.584133 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.581248 26046 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:08.543292  1717 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:27:08.543341  1432 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:08.587952 26053 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:08.588918 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 270976us
I20260709 12:27:08.536346  1646 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:08.586755 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 138435us
I20260709 12:27:08.594983 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.598948 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.582417  1288 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:08.613905 26046 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:08.615187 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 284309us
I20260709 12:27:08.623211 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:08.626081 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.629599 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.551891  1235 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.634927 26201 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:08.613849  1432 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:08.640329 26201 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:08.636624 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:08.644398 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:08.637974 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.644938 26789 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:08.635378 26046 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:08.642241 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 134724us
I20260709 12:27:08.642131 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:08.614693  1656 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:08.581323  1676 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:08.643744 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66494.5us
W20260709 12:27:08.646653 26019 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.575s (SENT)
I20260709 12:27:08.644140 26201 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.652405 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.652439 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.673273  1661 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:08.673949  1711 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:27:08.591656  1449 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.676597 26201 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.627250  1668 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:27:08.681107  1667 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:08.681674 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.681921  1646 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:08.691394 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.614998  1647 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
W20260709 12:27:08.691907 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68875us
W20260709 12:27:08.727015 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.649523  1656 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:08.692920 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.676476  1542 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:27:08.708765 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62140.9us
I20260709 12:27:08.709407 26784 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:08.631673  1326 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.709503 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:08.710407 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80870.6us
I20260709 12:27:08.653831  1288 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:08.712635  1449 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:08.713045  1611 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:08.717226  1676 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:08.717703  1718 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:08.718101  1593 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:08.718727 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66416.3us
I20260709 12:27:08.758458 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.649631  1554 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:08.721385  1711 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:08.722559  1332 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:08.722690 26199 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:08.584872  1712 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:27:08.730346 26201 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:08.733516 26124 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:08.747603 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 266586us
I20260709 12:27:08.734424  1656 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:08.727270  1646 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:08.787920  1646 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.733534 26789 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:08.741125  1638 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:08.795886 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.796294 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:08.796669 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.775177 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.752732 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:08.777292 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47746.1us
I20260709 12:27:08.744180  1661 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:08.808674 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:08.809845 26789 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:08.811658 26046 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:08.756722  1718 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.818814 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:08.842190 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.709112  1455 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:08.849278 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43816.8us
I20260709 12:27:08.853482  1288 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:08.866182 26789 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:08.866789  1656 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:08.781841  1536 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:08.849320  1718 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.831696  1332 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:08.831456  1554 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:08.881721  1432 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:08.913693 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 157106us
W20260709 12:27:08.899828 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.899094 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 159117us
W20260709 12:27:08.884842  1696 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:27:08.871907  1711 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:08.910830 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:08.898770  1560 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.919178 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.901108  1647 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.901407  1676 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:08.877036  1086 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:27:08.928354  1647 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:08.927421 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:08.930769 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.963235 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.908386  1638 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:08.925702 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:08.882268  1682 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.923223  1560 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.970920 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:08.931582  1726 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:08.958881  1733 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:27:08.959708  1720 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:27:08.866285  1288 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:08.957683  1640 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.970s	user 0.015s	sys 0.000s
I20260709 12:27:08.968401 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.924795  1235 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:08.973488 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47839.8us
W20260709 12:27:08.978698 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51438.6us
W20260709 12:27:08.977273 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 101045us
I20260709 12:27:08.981684 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:08.982349 26789 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:08.982255 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.981704 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.967100  1647 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.965174  1086 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:08.987732 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:08.974772  1560 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:08.988305 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 146220us
I20260709 12:27:08.977533  1742 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:08.995083 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:08.932830  1722 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 12:27:08.991513 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.997946 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:08.991325  1086 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:08.988646 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:09.000746 26202 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:09.001358  1593 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:09.001781  1593 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:09.001757  1638 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.002409  1593 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:09.003371  1655 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.749s	user 0.009s	sys 0.002s
I20260709 12:27:09.003803  1655 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.008364  1742 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:09.009429  1742 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.000396  1536 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:09.009430 26200 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:08.975278  1676 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 12:27:09.016826  1710 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.610s	user 0.003s	sys 0.000s
I20260709 12:27:08.992398  1332 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.018661  1593 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.021139  1413 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:08.985865  1640 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:08.992105  1739 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:09.024820  1739 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:09.025218  1739 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:09.025651 26195 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:09.025815  1593 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.026510 26199 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:09.029946 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:09.030690 26195 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:27:09.027997 26199 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
W20260709 12:27:09.036803 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 278438us
I20260709 12:27:09.037859 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:09.039789 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52108.5us
I20260709 12:27:09.040830 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:09.047646 26195 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:27:09.048205 26195 pending_rounds.cc:85] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 10
I20260709 12:27:09.048533 26195 pending_rounds.cc:107] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:27:09.048884 26195 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:09.049834  1593 raft_consensus.cc:2764] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:09.052675  1656 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:09.053807  1678 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.005s
I20260709 12:27:09.054271  1678 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.055075  1447 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:09.055506  1447 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:09.000005  1711 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.062466  1670 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:09.064585  1655 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.072906 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:09.028949  1640 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.029361  1554 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:09.077963 26202 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:09.081624  1747 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.082332  1747 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.086949 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:09.087571 26788 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:09.088155 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:09.094786 26788 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:09.095889  1678 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.032794  1482 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.099009 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61278.9us
I20260709 12:27:09.102810 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:09.044670  1726 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:09.071846  1647 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.098639  1747 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.071889  1682 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.146044 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:09.117023 26123 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:09.127785 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 132761us
I20260709 12:27:09.105350  1483 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:27:09.051921  1644 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.099444  1413 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.154630 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:09.099460  1740 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:09.103672  1722 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:09.150770  1655 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:09.171142 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 130415us
I20260709 12:27:09.147486  1676 consensus_queue.cc:1243] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:09.165931 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:09.155929  1554 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:09.127329  1086 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.182340 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:09.156978  1747 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.114319  1482 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.184639 26200 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:09.159410  1726 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:09.191001 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.186316 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:27:09.199745 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 97044.5us
I20260709 12:27:09.202076 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:09.179072  1655 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:09.165558  1413 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.145509  1745 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.078s
I20260709 12:27:09.187770  1288 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:09.183472  1554 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.165220  1235 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:27:09.128793  1447 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.225354 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:09.226258 26202 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:09.227080 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:09.165292  1449 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:27:09.234463  1449 consensus_queue.cc:1243] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:09.220888  1750 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:09.221962  1413 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.222898  1288 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:09.192008  1747 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.225526  1447 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:09.200544  1086 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:09.202785  1755 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:27:09.229161 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:09.169580  1722 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.233901  1326 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:09.234050 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2583) took 1113 ms. Trace:
I20260709 12:27:09.258870 26197 rpcz_store.cc:276] 0709 12:27:08.121146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.121277 (+   131us) service_pool.cc:224] Handling call
0709 12:27:08.155861 (+ 34584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.321399 (+165538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.233774 (+912375us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:09.233964 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378144128}
I20260709 12:27:09.261888 26197 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:09.263413 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.204442  1749 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:09.265075  1754 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:27:09.266492  1727 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.163s
W20260709 12:27:09.267794  1751 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:09.238970 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:09.207403  1655 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.251856 26195 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:09.254917  1413 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:09.217164  1554 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.269505 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:09.256596  1740 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:09.285292  1739 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:09.296317 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 94612.5us
I20260709 12:27:09.297631 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:09.298712  1739 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:09.301432 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 121906us
I20260709 12:27:09.305742 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:09.268006  1560 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.291944  1413 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:09.321221  1413 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.322064  1560 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.279946  1455 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.334874 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 65461.1us
I20260709 12:27:09.337778 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:09.338430 26124 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.338696 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:09.339314 26129 raft_consensus.cc:2376] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23. Re-sending same reply.
I20260709 12:27:09.342090 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:09.342139 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:09.342015 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:09.343370 26050 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:09.345289 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:09.346196 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 117138us
I20260709 12:27:09.348424 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:09.350409 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:09.283183  1472 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 12:27:09.366473  1766 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.304013  1740 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:09.353626 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.353204 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:09.283731  1745 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:27:09.295006  1726 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.340569  1455 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.355026  1482 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.355116  1722 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:09.361177  1770 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:09.362200 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:09.347649  1536 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.349823  1288 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.388563  1740 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.364398  1763 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:09.364872  1761 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:09.366585 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:09.339766  1647 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:09.346761  1727 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:09.398641 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:09.402959 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:09.391984  1766 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:09.415560 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:09.420567  1752 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:27:09.423269  1655 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.426574 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2525) took 1278 ms. Trace:
I20260709 12:27:09.427171 26049 rpcz_store.cc:276] 0709 12:27:08.148558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.148738 (+   180us) service_pool.cc:224] Handling call
0709 12:27:09.426248 (+1277510us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b540035a2d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:09.426448 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176383360}
I20260709 12:27:09.429538  1455 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.430447 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 2908) took 1174 ms. Trace:
I20260709 12:27:09.394963  1761 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:09.430998 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.430981 26194 rpcz_store.cc:276] 0709 12:27:08.255557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.256208 (+   651us) service_pool.cc:224] Handling call
0709 12:27:09.430155 (+1173947us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400216bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:09.430325 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419441024}
I20260709 12:27:09.404531  1727 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:09.394644  1759 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:27:09.429564  1747 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:09.436671  1761 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.518347 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2530) took 1346 ms. Trace:
I20260709 12:27:09.518805 26053 rpcz_store.cc:276] 0709 12:27:08.171780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.191335 (+ 19555us) service_pool.cc:224] Handling call
0709 12:27:08.191679 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.591244 (+399565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.518024 (+926780us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:09.518245 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910810880}
I20260709 12:27:09.439985  1766 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:09.479866  1722 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.586612 26197 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:27:09.435346  1749 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:27:09.437849  1560 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
W20260709 12:27:09.589807 26094 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.571s (SENT)
I20260709 12:27:09.488442  1771 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:09.584784  1689 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 12:27:09.592445  1718 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.683s	user 0.015s	sys 0.007s
W20260709 12:27:09.623370 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.627054  1747 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.627900  1747 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.631614  1747 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:09.641259  1768 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:27:09.650565  1769 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:09.651280  1769 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:09.652369  1769 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.608754  1749 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:27:09.676592 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 263557us
I20260709 12:27:09.654631 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:09.659335 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1331) took 1614 ms. Trace:
I20260709 12:27:09.677825 26790 rpcz_store.cc:276] 0709 12:27:08.044725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.044833 (+   108us) service_pool.cc:224] Handling call
0709 12:27:08.085946 (+ 41113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.121494 (+ 35548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.659034 (+1537540us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b54005cdad0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:09.659242 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28491648}
W20260709 12:27:09.686153 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:09.656522  1767 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:27:09.686790 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1322) took 1640 ms. Trace:
I20260709 12:27:09.656405  1087 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:09.708925 26782 rpcz_store.cc:276] 0709 12:27:08.037897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.040385 (+  2488us) service_pool.cc:224] Handling call
0709 12:27:09.654456 (+1614071us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:27:09.677910 (+ 23454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6138240}
I20260709 12:27:09.599576  1722 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:09.656224  1775 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:27:09.707408 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1302) took 1075 ms. Trace:
I20260709 12:27:09.714061 26201 rpcz_store.cc:276] 0709 12:27:08.631622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.781658 (+150036us) service_pool.cc:224] Handling call
0709 12:27:09.431760 (+650102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:09.432021 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.432050 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:09.432114 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:09.432164 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.432260 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.707141 (+274881us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400216bd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:09.707315 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494241152}
W20260709 12:27:09.711372 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.655951  1682 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:09.657747  1655 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:09.654090  1769 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.720389  1682 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:09.721328  1682 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.722587  1682 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:09.684514  1749 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:09.685882  1759 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.624310  1718 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.609661  1771 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.685981  1456 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:09.733809  1765 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 12:27:09.734848  1774 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:09.607873  1727 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.732640  1297 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.736397 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:09.709942  1087 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:09.732381  1663 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.747236  1732 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.818s	user 0.004s	sys 0.012s
I20260709 12:27:09.765295  1749 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.767000  1482 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.772547  1456 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.774791  1718 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:09.826393 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:09.788820 25938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 77813.2us
I20260709 12:27:09.788574  1297 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.830566  1668 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.841557  1765 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:27:09.853744 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 556187us
I20260709 12:27:09.788664  1663 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.861769  1663 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.861914  1297 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.866063 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1067) took 1652 ms. Trace:
I20260709 12:27:09.726297  1766 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.866540 26054 rpcz_store.cc:276] 0709 12:27:08.214127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.405888 (+191761us) service_pool.cc:224] Handling call
0709 12:27:08.406325 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.421471 (+ 15146us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:08.421518 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.454203 (+ 32685us) log.cc:844] Serialized 26 byte log entry
0709 12:27:08.752256 (+298053us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:08.752318 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.752417 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.746921 (+994504us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b2400136fe8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:27:09.847430 (+100509us) raft_consensus.cc:1685] finished
0709 12:27:09.847463 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.865981 (+ 18518us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2303,"raft.queue_time_us":13647,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":22410,"spinlock_wait_cycles":64285056,"thread_start_us":226266,"threads_started":1,"wal-append.queue_time_us":395847}
I20260709 12:27:09.872861 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2907) took 1617 ms. Trace:
I20260709 12:27:09.868327  1668 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.874296  1087 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.875520  1087 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.876400  1761 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.874351  1738 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.865496  1332 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:09.887102 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1297) took 1267 ms. Trace:
I20260709 12:27:09.745014  1759 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.873230 26198 rpcz_store.cc:276] 0709 12:27:08.255273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.259817 (+  4544us) service_pool.cc:224] Handling call
0709 12:27:08.279322 (+ 19505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.310873 (+ 31551us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:08.341581 (+ 30708us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.236019 (+894438us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b18002e6cc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:27:09.872548 (+636529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:09.872622 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:09.872665 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.872725 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.872817 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":469857,"prepare.queue_time_us":30497,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":13434,"spinlock_wait_cycles":5741952,"thread_start_us":163419,"threads_started":1,"wal-append.queue_time_us":1030693}
I20260709 12:27:09.888846 26203 rpcz_store.cc:276] 0709 12:27:08.619891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.620031 (+   140us) service_pool.cc:224] Handling call
0709 12:27:08.634443 (+ 14412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.634676 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:08.649059 (+ 14383us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.519133 (+870074us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b1800149aa0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:27:09.886588 (+367455us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:09.886678 (+    90us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:09.886755 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.886845 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.887020 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":168889,"prepare.queue_time_us":16686,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"spinlock_wait_cycles":21921536,"thread_start_us":198276,"threads_started":1,"wal-append.queue_time_us":1068582}
W20260709 12:27:09.887696 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 86492.2us
I20260709 12:27:09.893268  1748 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.843771  1482 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.891957  1640 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:09.885363  1722 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:09.905187 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.929373  1771 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.931915  1759 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.902189  1656 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.943944 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.900681  1748 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.903740  1611 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:09.901015  1761 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.950343 26789 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:09.899302  1668 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.956707 26788 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:09.941479  1745 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.961900 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:09.966825 26790 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:09.967686 26785 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:09.963112 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:09.967784 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:09.964779 26788 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:09.951618  1611 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:09.949801  1718 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 12:27:09.941797  1640 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.965502  1769 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.975919  1726 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:27:10.006397  1769 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.974498  1745 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.007393  1640 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:09.992851 26049 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:09.974560  1748 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:09.981489  1668 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.009491 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:10.009400  1668 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.011374 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:10.012295  1611 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.013728  1611 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:10.013826  1760 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:10.014336  1760 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:10.015174  1760 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.969669 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.016273  1760 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.017544 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:09.974184  1682 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:10.018844  1682 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:09.980401 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.024276 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:09.982709 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.025815 26051 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:10.026257 26782 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:09.990967 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.027613 26053 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.025128 26201 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:09.989686 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.035605 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:09.954658  1740 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.055375 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.055393 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.058082 26194 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:10.058350 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.061558 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.059312 26203 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:10.068869 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.075296 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:10.069937 26203 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:27:10.005123  1644 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:10.011498  1668 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.977681  1482 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:09.950187  1656 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:09.979525  1456 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:10.095376  1554 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.812s	user 0.010s	sys 0.003s
I20260709 12:27:10.101923 26046 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:10.087646  1771 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.107138  1656 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:10.111951 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:10.094197  1722 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:10.109120  1554 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.119941 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:10.131515 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.131652 26047 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:10.120867 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.123309  1766 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:10.124048 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.124419 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.129702  1745 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:10.155397 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 200589us
I20260709 12:27:10.155586 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.156909 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.153091 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:10.133910 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:10.136962 26203 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:27:10.140110 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.159574 26202 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:10.159777 26201 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:10.141763 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:10.144429 26195 raft_consensus.cc:1217] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Deduplicated request from leader. Original: 18.10->[20.11-23.12]   Dedup: 20.11->[23.12-23.12]
I20260709 12:27:10.063608  1740 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.148080 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.148927 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.167285 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.167686 26049 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:27:10.101680  1456 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.102306  1769 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.131333  1722 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:10.161541 26196 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:10.162140 26200 pending_rounds.cc:85] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 9
I20260709 12:27:10.175993 26200 pending_rounds.cc:107] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:27:10.163117 26203 pending_rounds.cc:85] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 13
I20260709 12:27:10.166862 26051 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 12:27:10.176601 26203 pending_rounds.cc:107] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:27:10.168038 26047 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:10.176898 26203 raft_consensus.cc:2889] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:10.169114 26195 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: [20.11-23.12]
I20260709 12:27:10.170482 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.173879 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:10.184506 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.135000 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:10.151466  1718 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.187686 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.176211 26200 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:10.151716  1235 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:10.188452 26197 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:27:10.194906 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:10.196923 26785 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:10.187691 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:10.189422 26048 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:10.193243 26788 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:10.162330  1670 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:10.204056  1682 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:10.205135  1723 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:10.204892 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.330144 26196 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:10.172712  1721 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:10.181710  1722 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:10.214007  1787 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:27:10.216720  1554 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.216360  1235 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.220798  1769 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:10.224505 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:10.226374  1740 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:10.228777 26047 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:10.230767  1745 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:10.188868  1456 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.114231  1771 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:10.233194 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.309368 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.332621 26789 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:27:10.213194  1792 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:10.187186 26053 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:10.333868 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.417732 26197 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:10.345384 26194 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:10.346179 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:10.419293 26790 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:10.329489  1723 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:10.352756  1721 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:10.354192 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 200096us
I20260709 12:27:10.355865 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.421690 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:10.356877 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:10.422127 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.361263  1656 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.200798  1718 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.384810 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.384961 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.386004 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:10.244815 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109999us
I20260709 12:27:10.388856 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.389604 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.432839 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.433126 26123 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:10.390831 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.433423 26123 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:10.394447 26125 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:10.200560  1087 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:10.441515 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:10.395128 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.195384  1715 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:10.233568  1670 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.246229  1759 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.408655 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 238707us
I20260709 12:27:10.457888 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.458019 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.414700 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.415015 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.423166 26126 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:10.383139  1747 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.378449 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.429067 26127 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:10.431696 26018 connection.cc:256] RPC call timeout handler was delayed by -1.0022884370000611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24100000000000002s for a response
I20260709 12:27:10.442389 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:10.462108 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.445998 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.446043 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.461458 26127 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:10.446766 26195 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:10.463953 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.464955 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:10.446689 26123 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:10.420434  1661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.614s	user 0.020s	sys 0.033s
I20260709 12:27:10.365432  1644 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:27:10.455084 26129 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:10.458159 26120 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.458877 26121 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:10.458732 26125 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:10.459022 26196 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:10.425546  1721 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:10.435719  1755 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:10.457510  1670 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.481184  1718 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.468425 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.484736 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.485180 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.485390 26125 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:10.485647  1456 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.485838 26128 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:10.486295 26128 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:10.486228  1456 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:10.487385  1456 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:10.488057 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.488725 26126 raft_consensus.cc:2376] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22. Re-sending same reply.
W20260709 12:27:10.474021  1661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.664s	user 0.020s	sys 0.033s
I20260709 12:27:10.464113 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:27:10.441465  1087 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.489681 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.451843  1554 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.479642 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58046.5us
I20260709 12:27:10.489925  1087 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:10.453485  1795 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:10.498441 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:10.499923 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.440274  1656 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.486330 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.496824 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.496908 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.508301 26124 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.470378  1560 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:10.509362 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.474576 26127 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.510756 26127 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.469555  1677 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:10.452260  1766 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:10.489316  1661 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:10.515949 26202 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:27:10.491017  1770 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:10.519483 26129 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:10.456904  1715 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.515090 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:10.452430  1740 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:27:10.493422  1760 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:27:10.535436  1235 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:10.536202  1235 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.536784  1235 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.539278  1235 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.490463  1554 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.545953  1661 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.494083  1755 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:10.512215  1718 consensus_queue.cc:1175] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 19 status { last_received { term: 18 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:10.501400  1723 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.549571  1755 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:10.550273  1482 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:10.550772  1482 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.551174  1482 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.551895  1656 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:10.514307  1745 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:10.513592  1677 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:10.552661  1745 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.555154  1745 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:10.476346  1759 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:10.557230  1611 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.004s
I20260709 12:27:10.550287 26194 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:10.564837 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.566265 26125 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 18: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:10.567324 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.546960  1771 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:10.519389  1795 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.572153 26194 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:10.579573 26126 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:10.779592 26120 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
W20260709 12:27:10.765399 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.753542 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.753252  1771 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:10.749131 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 234101us
I20260709 12:27:10.755614 26124 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:10.757035 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:10.772298  1803 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.778128 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 315217us
I20260709 12:27:10.784125 26757 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:10.472860  1747 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.759300  1759 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:10.546980  1554 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.700290 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.790416 26125 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:10.778673 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 297775us
I20260709 12:27:10.531558  1770 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:10.780886 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.764945 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.785395 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.773236  1670 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.787086 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.790808  1803 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.807814  1235 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:10.808724  1803 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.809207 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.805433 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:10.788614 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.792095 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.792459 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:10.785658  1668 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.007s
I20260709 12:27:10.793031 26050 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:27:10.793519 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:10.793329  1812 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:10.793946  1772 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:10.795490  1745 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:10.822600 26125 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:10.823971 26791 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:10.799844 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:10.801244 26129 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.798812 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:10.808163 26124 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:10.825662 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:10.827020 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.788035 26122 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:10.813936 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:10.813148 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:10.814956 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:10.815037 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:10.806468  1770 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.828619 26789 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:10.829360 26127 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:10.829873 26093 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:10.831677 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:10.832517 26125 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:10.789987  1771 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.829463  1483 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:10.833595  1560 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.883526  1560 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.824265  1755 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:10.820953  1811 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:10.836798  1235 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:10.885917  1755 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:10.886195  1235 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:10.886283  1811 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:10.886664  1235 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.886976  1755 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.888167  1755 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:10.887782  1811 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.838038  1769 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:10.820959  1772 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.890650  1772 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.894829  1747 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.831660  1812 consensus_queue.cc:1243] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 22.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:10.896822  1812 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:10.825922  1611 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:27:10.866935  1795 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:10.909739 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.910377  1721 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.877528  1759 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.880043  1670 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:10.926524 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.870501 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:10.950589 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 150770us
I20260709 12:27:10.964761 26122 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
W20260709 12:27:10.957758  1796 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.527s	user 0.002s	sys 0.000s
W20260709 12:27:10.960682 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 147570us
I20260709 12:27:10.970233 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:10.971582 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3839) took 1645 ms. Trace:
I20260709 12:27:10.971771  1809 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:10.972854 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.976049 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:10.962411  1722 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.002s
I20260709 12:27:10.972275  1809 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.977656  1809 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.977605  1722 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.979625  1723 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.980342  1676 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:10.962913  1747 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.980706  1087 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.981549  1676 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:10.981355  1747 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.954207  1770 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:10.981977  1676 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:10.982925  1087 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.982998  1676 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:10.961967  1681 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:10.986460  1681 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.988727  1681 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:10.977907  1805 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 12:27:10.989181  1482 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.991346  1749 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:10.991523  1482 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:10.991766  1721 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:10.973096  1715 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:10.992512 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 606562us
I20260709 12:27:10.997681  1791 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:27:10.997967  1797 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.550s	user 0.005s	sys 0.002s
I20260709 12:27:10.998493  1791 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:10.999322  1288 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:11.004161 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:10.953150  1769 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.011520 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.013746 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.014645 26053 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:11.015404 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.014732 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.017246 26047 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.990352  1789 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.020398  1676 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.024848  1554 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:11.025377  1554 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.025719  1554 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:11.029141  1800 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:11.029395  1640 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:11.029722  1800 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:11.030421  1791 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:11.030962  1747 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.030720  1800 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.030974  1791 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:10.979893  1722 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.032146  1791 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:11.032603  1800 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:11.033111  1791 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.034412  1791 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:11.036474 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.037186  1740 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:11.014432  1777 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:11.037843  1769 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.029306  1656 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.039958  1777 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:11.041232  1656 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.042788 26050 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:11.043144  1788 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.043684  1788 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:10.958454  1772 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.049719 26790 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:11.053887  1677 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.055325  1677 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:10.972075 26045 rpcz_store.cc:276] 0709 12:27:09.326516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.326703 (+   187us) service_pool.cc:224] Handling call
0709 12:27:09.347039 (+ 20336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:09.438079 (+ 91040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.438108 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:09.447755 (+  9647us) log.cc:844] Serialized 52 byte log entry
0709 12:27:09.661055 (+213300us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:09.661124 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.954840 (+1293716us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:10.954910 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.954979 (+    69us) raft_consensus.cc:1685] finished
0709 12:27:10.954997 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.971503 (+ 16506us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":375,"raft.queue_time_us":1051,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":197085056,"thread_start_us":212701,"threads_started":1,"wal-append.queue_time_us":270810,"wal-append.run_cpu_time_us":8352,"wal-append.run_wall_time_us":1032043}
I20260709 12:27:11.053929  1086 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.059275  1086 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.058344 26050 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:11.061422 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.061929  1086 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.064190 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1074) took 1004 ms. Trace:
I20260709 12:27:11.076581  1087 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:11.077401  1087 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.080651 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1073) took 1033 ms. Trace:
I20260709 12:27:11.081096 26051 rpcz_store.cc:276] 0709 12:27:10.047694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.047955 (+   261us) service_pool.cc:224] Handling call
0709 12:27:10.166588 (+118633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.176732 (+ 10144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.080404 (+903672us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b54000f8550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:11.080575 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210985344}
I20260709 12:27:11.089480  1769 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.090291  1740 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:11.091204  1740 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:11.092286 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55915.4us
I20260709 12:27:11.092648  1644 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.093218  1644 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.094882 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.095366  1644 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.095652 26199 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:11.096539  1661 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.097041  1661 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.097610 26194 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:11.099510  1661 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:11.100081 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84746.4us
I20260709 12:27:11.100898 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.104310 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.104455 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.105119 26194 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:11.108784 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.109521 26194 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:11.110908 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.111577 26194 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:11.114249 26194 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:11.114363 26203 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:11.115536 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.116056 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.116135 26200 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:27:11.116355 26194 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:11.118724 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:10.966243  1560 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.116823 26202 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:11.156037 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.064667 26054 rpcz_store.cc:276] 0709 12:27:10.060101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.060342 (+   241us) service_pool.cc:224] Handling call
0709 12:27:10.086490 (+ 26148us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.094663 (+  8173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.094696 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.098963 (+  4267us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.418374 (+319411us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:10.518544 (+100170us) log.cc:844] Serialized 14 byte log entry
0709 12:27:10.519065 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.519142 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.804287 (+285145us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b18001a2cc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:11.061850 (+257563us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b18001a2cc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:11.063929 (+  2079us) raft_consensus.cc:1685] finished
0709 12:27:11.063956 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.064114 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23350,"prepare.queue_time_us":409353,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":9426,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":7241,"spinlock_wait_cycles":115052160,"thread_start_us":354056,"threads_started":2,"wal-append.queue_time_us":401910}
I20260709 12:27:11.006868 26129 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:11.129951 26203 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:11.157977 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.158748 26203 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:11.005085  1771 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.140956 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.168689 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.169092 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.168735 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.170588 26045 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:11.173273 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.174280 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.175094 26203 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:11.148706 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 315567us
I20260709 12:27:11.149183 26050 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:11.153682 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.162223 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.174156  1789 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.081s
I20260709 12:27:11.202630 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.266534 26048 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:11.267551 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.202389 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.211741  1766 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:27:11.207710  1800 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.052s
I20260709 12:27:11.272063 26790 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:27:11.139390  1769 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:11.211536 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.212039  1790 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:27:11.180116 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.223915 26202 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:11.173997  1644 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:11.248502 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87344.2us
I20260709 12:27:11.221535 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.247140 26203 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:27:11.251911 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.303612 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.305212 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.306648 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.141949  1682 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:11.263964  1759 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.139792  1723 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.182966 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.226524  1772 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.263695 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.207834  1288 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:27:11.208979  1745 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:27:11.201916  1815 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 12:27:11.325227 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.265300 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.265398 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.268679 26052 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:11.177597 26784 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:11.200812 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:11.219448 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.298164 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.100646  1809 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.310603 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.340144 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.340423 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.341166 26048 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:11.311619 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 820600us
I20260709 12:27:11.250265  1748 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:27:11.314332 26199 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:27:11.318176 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.259433 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2602) took 1091 ms. Trace:
W20260709 12:27:11.349978 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.351289 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 196552us
I20260709 12:27:11.352201 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.330001 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.332692 26789 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:11.357911 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.333915 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.335196 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.293325  1718 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.349905 26201 rpcz_store.cc:276] 0709 12:27:10.167911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.175024 (+  7113us) service_pool.cc:224] Handling call
0709 12:27:10.175400 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.182915 (+  7515us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:10.228628 (+ 45713us) log.cc:844] Serialized 14 byte log entry
0709 12:27:10.991054 (+762426us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.991562 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.991715 (+   153us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:11.252328 (+260613us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.259061 (+  6733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.259164 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.259223 (+    59us) raft_consensus.cc:1685] finished
0709 12:27:11.259241 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.259377 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":600700,"prepare.queue_time_us":33949,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":583347,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":38306,"thread_start_us":457498,"threads_started":2,"wal-append.queue_time_us":265228}
I20260709 12:27:11.342620 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.344208 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.304934  1749 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:11.348546 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:11.314064  1788 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.310457  1644 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:11.359640 26200 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.401168 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.426687 26783 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:11.406543 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.324767  1747 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:11.370141 26786 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:11.313568  1670 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:11.439770 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.336813  1682 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:11.442365 26791 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:11.456308 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.459129 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.460033 26045 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:27:11.460500 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:11.400897  1772 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.465969 26053 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:27:11.466573 26048 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:11.467712 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.368427  1661 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:27:11.321938  1656 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:11.313316  1722 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:11.472585 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.322232  1755 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.129s
I20260709 12:27:11.479054 26200 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:11.368798 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.435834  1644 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:11.433281  1743 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:11.442171  1235 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.001s
I20260709 12:27:11.433604  1087 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.428427  1749 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.440135  1747 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.487203 26048 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21. Re-sending same reply.
I20260709 12:27:11.443830  1676 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:11.486795 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 150177us
I20260709 12:27:11.554765 26093 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.514917 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.405573  1718 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:11.520710 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 337739us
W20260709 12:27:11.528546 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.336582  1723 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.322326  1759 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.556180 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:11.535008 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 166278us
I20260709 12:27:11.533839 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:11.533806  1807 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:11.558513 26045 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:11.559053 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.560081 26054 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:11.560936 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.511066  1789 raft_consensus.cc:2764] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:27:11.558383 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.472494  1670 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.517135  1743 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:11.552297  1805 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:11.570106 26050 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:11.550365  1791 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.001s
I20260709 12:27:11.534078  1788 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:11.482590  1554 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.554800  1747 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:11.574666 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 230540us
I20260709 12:27:11.551132  1777 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.480644  1772 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.576694 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.569684  1743 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.581681 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.569990  1670 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:11.584818  1821 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:27:11.587356  1676 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:11.534065  1235 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.593760  1759 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.568290  1794 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:11.595337 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:11.600716 26046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1077) took 1482 ms (client timeout 1529 ms). Trace:
W20260709 12:27:11.624773 26046 rpcz_store.cc:269] 0709 12:27:10.118990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.124131 (+  5141us) service_pool.cc:224] Handling call
0709 12:27:11.600503 (+1476372us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b5400172354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:11.600643 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167831936}
I20260709 12:27:11.628398 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.629009 26046 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:11.629823 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.630394 26046 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.597820 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.644323 26050 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:27:11.619560 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.646771 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.603194  1761 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:11.648519 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.596581  1830 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:11.596727  1826 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:11.612581 26045 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:11.653201  1723 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:11.655336  1554 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.656452  1809 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.656708  1718 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.623310  1777 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:11.657502  1817 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:27:11.621448  1676 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.571265  1682 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.664988 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:11.669648 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 88000.1us
I20260709 12:27:11.674482 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.673157 26054 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:11.677806 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.663671  1830 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.679366 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.680799 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.683118 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.684273 26054 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:11.685246 26048 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:11.687354 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:11.691780 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:11.694636 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:11.697898 26049 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:11.698355 26169 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.675161  1676 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:11.677621  1809 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.704741 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.706828 26045 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:11.705379 26054 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:11.693734  1840 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:11.707320 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:11.705963 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 484465us
I20260709 12:27:11.705428 26052 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:11.714010 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:11.725661 26046 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:11.685941  1830 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:11.722960 26054 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:11.712129 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2610) took 1236 ms. Trace:
I20260709 12:27:11.679154  1761 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:27:11.688503  1841 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:11.737540 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.672768  1826 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.731734 26195 rpcz_store.cc:276] 0709 12:27:10.475737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.475956 (+   219us) service_pool.cc:224] Handling call
0709 12:27:10.476436 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.009783 (+533347us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:11.009812 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.010279 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.100747 (+ 90468us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b18003bd2e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:27:11.424029 (+323282us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:11.424078 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.424180 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.711736 (+287556us) raft_consensus.cc:1685] finished
0709 12:27:11.711765 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.712049 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":532820,"raft.queue_time_us":537528,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":37671296,"thread_start_us":304058,"threads_started":1,"wal-append.queue_time_us":689619}
I20260709 12:27:11.747953 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.749785 26045 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:27:11.700467  1836 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:11.705981  1656 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:11.709438  1235 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.763819 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.728747  1759 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:27:11.764930 26200 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 20: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:11.761696 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.766291 26202 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:11.702519  1838 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:11.773402 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.745831  1640 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:11.719087  1809 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.779461 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.737442  1840 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:27:11.778391  1554 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.788165  1554 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:11.788725  1827 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:27:11.790585  1554 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.771529  1836 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:11.793036  1829 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:27:11.793879  1836 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.778626  1838 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:11.797142  1838 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.797974  1826 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.768203  1841 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.798866  1826 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:11.799397  1826 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.799257  1841 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.801594  1823 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:11.802158  1823 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:11.783564 26045 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:11.802975  1823 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.803781 26045 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:11.806697  1670 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.808980  1670 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.801712  1841 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:11.813586 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 258861us
W20260709 12:27:11.816601 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 255694us
I20260709 12:27:11.817962 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:11.817826  1835 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:11.819180  1815 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:11.819684  1815 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.820106  1815 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:11.820547  1837 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:11.822176  1456 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:11.822723  1456 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:11.827365  1727 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:11.829682  1842 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:11.832767  1839 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:27:11.837491 26785 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:11.838745 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.846367  1772 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.816702  1795 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.779110 26198 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:11.809691 26128 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.855127 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:11.764712  1790 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.767771  1332 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:11.864804 26045 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:11.799962  1826 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:11.864991 26128 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:11.783097  1788 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.887359 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:11.891068 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.891851 26197 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:11.893059 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.902374 26128 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
W20260709 12:27:11.903154  1798 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
W20260709 12:27:11.903100 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 3940) took 1419 ms (client timeout 1546 ms). Trace:
I20260709 12:27:11.791134  1554 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
W20260709 12:27:11.904147 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 86242.1us
I20260709 12:27:11.876510  1837 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.906124 26123 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:11.906574 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 170873us
W20260709 12:27:11.909389 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.932539 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:11.804157  1823 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.936461 26049 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:11.919085  1815 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.908466  1644 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.920775 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:11.940299 26194 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.920368  1676 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.921337  1848 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:27:11.884958  1640 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:27:11.924167 26785 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:27:11.914404 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:11.925637  1809 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.930697  1800 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:11.933844 26128 rpcz_store.cc:269] 0709 12:27:10.483845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.484890 (+  1045us) service_pool.cc:224] Handling call
0709 12:27:11.809319 (+1324429us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b54000e9320. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7432306f 00007f5b7439bf60 00007f5b7439d7de 00007f5b743a927c 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2
0709 12:27:11.903026 (+ 93707us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12048640}
W20260709 12:27:11.935374 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.823789  1456 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.885088  1772 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.938535 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:11.876847  1790 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.976442 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:11.977270  1456 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:11.977403  1790 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.977713  1844 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:11.979240  1810 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:11.979662  1810 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:11.980369  1772 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.980706  1810 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.982844  1772 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:11.983785 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45380.3us
I20260709 12:27:11.968418  1794 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.984458  1841 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:11.946936 26123 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:11.926410  1743 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.951067  1773 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:11.942116  1837 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.958634  1856 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:11.996757  1856 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.998358  1837 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.998956  1856 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:11.999179  1826 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:11.939945  1789 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:11.999765  1773 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.000679  1747 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.002138 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.002539  1773 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.002803 26788 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:12.003404 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.004463  1815 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.004670 26201 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:12.005208  1830 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.005648 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.006773 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.007360  1830 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.911793  1455 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.024s
I20260709 12:27:12.019491 26052 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 12:27:11.972968 26046 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:11.968737  1432 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:11.972373  1800 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.984572  1794 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:11.985607 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3002) took 1600 ms (client timeout 1890 ms). Trace:
W20260709 12:27:12.032779 26169 connection.cc:256] RPC call timeout handler was delayed by -1.2815373210000871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:27:12.036564 26123 rpcz_store.cc:269] 0709 12:27:10.384919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.388616 (+  3697us) service_pool.cc:224] Handling call
0709 12:27:10.763406 (+374790us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b18002d1bc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743e0646 00007f5b7431a58e 00007f5b7439bf9e 00007f5b7439d7de 00007f5b743a927c 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c
0709 12:27:11.905697 (+1142291us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b58004265d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a927c 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c
0709 12:27:11.985546 (+ 79849us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3302464128}
W20260709 12:27:11.941816 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.988576  1542 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 12:27:11.944324  1449 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:11.969527 26196 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:11.910797  1788 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:11.985100  1841 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:12.009976  1847 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:27:12.023527  1644 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:12.024649 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.026819  1795 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.027855 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.029910  1743 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:11.985009  1640 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:27:12.032014  1656 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:12.059173 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.060132 26049 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.016018 26194 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:27:12.037847 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.035771  1432 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.046415  1800 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.050458  1611 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:12.019027  1773 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.053285 26047 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:12.003445  1747 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.060971  1640 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:12.063968 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.054095  1472 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:27:12.062000 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 166566us
I20260709 12:27:12.072620 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:12.066967  1432 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.068928  1788 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.072806  1472 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:12.083053  1861 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.004s
I20260709 12:27:12.073593  1841 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.083369  1472 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.084508  1472 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.085150  1777 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.075253 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.086586  1791 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:12.087038  1791 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:12.087788 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.076020  1830 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.062990  1656 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.092669 26093 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:12.092697  1449 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:12.094004  1812 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:12.094408  1812 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.094803  1812 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:12.072919  1795 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:12.095420  1554 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.096094  1554 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.074973 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:12.097492  1851 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.097980  1851 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.073374  1773 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.099716  1770 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.067739  1611 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:12.100250  1770 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.101141  1560 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:12.100593 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:12.102447  1770 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.101590  1560 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.101272  1611 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:12.114645 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.118099 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45575.6us
W20260709 12:27:12.122484 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.129581 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.128091  1811 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:12.141397 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.146767 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.079391  1826 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.123072  1760 raft_consensus.cc:476] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Not starting pre-election -- already a leader
I20260709 12:27:12.094821  1656 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.148204 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.122242  1560 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.121954  1777 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:12.136634  1611 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:12.151957 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.154469 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:12.106231 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:12.158514 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.092739  1830 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.149785  1456 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.150876  1726 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:12.148972  1744 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:12.163132 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.164472 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.156729  1472 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.087883  1791 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.166249 26199 raft_consensus.cc:2410] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:12.163298  1798 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:12.162094  1777 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:12.205330 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.209969 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.198207  1743 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:12.215739 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.212419  1456 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:12.218628  1456 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.219799  1456 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.223613  1432 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.224645 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1446) took 1744 ms. Trace:
I20260709 12:27:12.225064 26121 rpcz_store.cc:276] 0709 12:27:10.480028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.488548 (+  8520us) service_pool.cc:224] Handling call
0709 12:27:10.488879 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.777960 (+289081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.777991 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:10.778031 (+    40us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:10.778062 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.224392 (+1446330us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540010e0d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:12.224445 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:12.224587 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664251648}
I20260709 12:27:12.227022  1432 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.230756 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2558) took 1064 ms. Trace:
I20260709 12:27:12.231218 26050 rpcz_store.cc:276] 0709 12:27:11.166289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.271341 (+105052us) service_pool.cc:224] Handling call
0709 12:27:12.230539 (+959198us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:12.230690 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531407232}
I20260709 12:27:12.232436  1743 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.233489  1810 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:12.233963  1743 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:12.235255  1815 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:12.235636 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:12.236306  1449 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:12.237093  1235 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.237241  1815 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:12.235510  1743 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:12.238202  1743 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:12.238349 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.239032  1856 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:12.239401  1235 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:12.240362  1235 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.243163  1718 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:12.239401  1856 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.244577  1773 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.244781  1682 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:12.245291  1682 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.244933  1856 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.246366  1856 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.246181  1682 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.247671  1682 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:12.248324 26051 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:12.238119  1815 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:12.249363  1815 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.250236  1815 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.251609  1815 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:12.252342 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.252746  1235 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.255070  1811 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:12.212591  1554 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.267925  1812 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.266296 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 1892) took 1452 ms. Trace:
I20260709 12:27:12.168618  1744 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.268319  1777 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.215906  1798 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.270677 26123 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:12.271287 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:12.263936  1560 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.270530 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.272505 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.272843 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.274063 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.300395 26129 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:12.275446 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.287465 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:12.301548 26120 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:12.302917 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.280856 26122 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:12.279812 26784 rpcz_store.cc:276] 0709 12:27:10.814236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.814709 (+   473us) service_pool.cc:224] Handling call
0709 12:27:12.266104 (+1451395us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:12.266231 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802168704}
W20260709 12:27:12.280864 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148881us
I20260709 12:27:12.168419  1795 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.288743 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.290184 26126 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:12.290907 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.278925  1830 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.306428 26124 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.312829 26121 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:12.314759 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.300956  1773 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:12.315657 26202 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:12.316258 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.316562 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.317354 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.315896 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.322340 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.280264  1743 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.327730 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.328769 26199 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:12.328763 26053 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:27:12.330566 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.332145 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.333817 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.339962 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.340844 26126 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.342633 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.343797 26124 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:12.345794 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.346041 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.346041 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.308498  1856 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:12.315879  1681 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:12.355823 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.315593  1773 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:12.356701 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.356835 26126 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:27:12.359819 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.367137 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.367743 26123 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:12.368546 26195 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:12.280476  1794 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.365208 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.366762 26048 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:12.330703  1816 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:12.373858 26126 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:12.353818 26124 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: [23.13-25.14]
I20260709 12:27:12.376704 26120 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:27:12.366127  1681 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.371850  1773 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:12.378638  1235 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:27:12.350713  1743 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.404786 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.405486 26125 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:12.404337 26126 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:12.408380 26120 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.393954  1745 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:12.401871  1681 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.421600 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.425271 26120 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.427950 26123 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:12.428182 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.383440  1816 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.450680  1816 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:12.456271 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.408397  1747 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:12.461814 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98510.3us
I20260709 12:27:12.467242 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.481428 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.482736 26126 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:27:12.483842 26124 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:12.494189 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.504025 26195 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:12.520411 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.499279  1816 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:12.502713  1812 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.532644 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.534400 26198 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:12.472486  1747 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.515030  1727 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.535399 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:12.583222 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 324991us
I20260709 12:27:12.541972  1743 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.546891  1851 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.553551  1676 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.558079 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.566231 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.567605  1644 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:27:12.568775  1877 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:27:12.577133  1794 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.528847  1560 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.511935  1837 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:27:12.599897 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.588133 26124 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:12.601442 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.602124 26124 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:12.601442 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.603032 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.603500 26128 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:12.603780 26124 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 12:27:12.587661  1843 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:12.607345 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.588737 26120 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.605068 26128 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:12.592505  1644 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:12.630720 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.587152  1727 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.631776 26128 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:12.649501  1856 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.621632  1681 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.675335 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.676950 26128 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:12.682442 26120 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:12.682893 26122 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:12.657965  1800 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.687009 26120 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:12.683230  1611 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.000s
W20260709 12:27:12.689071  1883 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:27:12.693609  1884 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:12.695132 26129 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:12.697162 26120 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:12.698845 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.700276 26120 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
W20260709 12:27:12.701118  1874 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:27:12.681062  1644 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:12.701851 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.702855  1644 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.702262 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.704071 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.704214 26203 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.704741 26120 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:12.705628 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.706616 26120 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:12.707608 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.708525 26120 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:27:12.708917 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.709482 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:12.711326 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.713156 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.717666 26128 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 23 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 12:27:12.722913 26052 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:12.724225  1759 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.725036  1681 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:12.725378 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:12.726622 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.726756  1749 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:12.727082 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.724953  1759 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:12.728631 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.731029 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.736712 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.737501 26126 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:12.744747  1759 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.690987  1800 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.751317 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.751706 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.752764 26787 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:27:12.754931 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:12.755224 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:12.756460 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.757345 26128 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.760496 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:12.761590 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.775017 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:12.778120  1727 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:12.780575 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:12.781164  1791 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.783059  1472 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:12.783528  1472 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.783953  1472 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:12.784142 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.785404 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:12.786155  1794 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.786935  1882 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:12.787365  1882 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.787693  1882 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.783077 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:12.793167 26124 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:12.793638  1773 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.774109  1800 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:12.797015 26045 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:12.797485  1773 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:12.803454 26124 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:27:12.808080 26757 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.805s (SENT) [suppressed 23 similar messages]
I20260709 12:27:12.811259 26784 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:12.814796 26203 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:12.817917 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:12.744803  1747 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:12.822808 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.824286 26120 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:12.816128  1640 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:12.826614 26093 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.834460 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:12.837695  1682 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:27:12.840793 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.818266  1879 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:12.818106  1726 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:27:12.826064  1743 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:12.848012 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:12.819088  1678 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.026s
W20260709 12:27:12.852243 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 268241us
I20260709 12:27:12.857817 26201 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:12.859405 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:12.850970  1682 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.861462 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:12.850208  1879 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:12.864579  1747 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:12.868043 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.868153 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3861) took 1117 ms. Trace:
I20260709 12:27:12.836689  1640 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.871245 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.873773 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.853770  1743 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:12.872857 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.873821 26051 rpcz_store.cc:276] 0709 12:27:11.750649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.795712 (+ 45063us) service_pool.cc:224] Handling call
0709 12:27:12.248124 (+452412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.248730 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.867960 (+619230us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b540008e8d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:12.868099 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1039478528}
I20260709 12:27:12.855897  1793 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:12.880303 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.878752 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.878578 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:12.879494 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.875764  1743 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:12.881018 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.885062 26195 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:12.887027 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.883158 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.888332 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.882514 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.890784 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.891115 26054 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:12.892987 26196 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:12.895332 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.895967 26045 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:12.875560  1879 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:12.881491  1895 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:12.904780 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:12.904011  1749 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.880151  1682 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.887874  1747 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:12.892969  1759 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:12.914177 26201 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:12.921500 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.921371  1882 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:12.887948  1800 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:27:12.902268  1793 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:12.939628 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2569) took 1699 ms. Trace:
I20260709 12:27:12.940052 26046 rpcz_store.cc:276] 0709 12:27:11.239972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.737620 (+497648us) service_pool.cc:224] Handling call
0709 12:27:11.737948 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.021181 (+283233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.939395 (+918214us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400173c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:12.939569 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539917056}
I20260709 12:27:12.945065 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.941180 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.946658 26046 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:12.947580 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:12.948109 26046 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:12.949368 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.921607  1759 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.956446 26791 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:12.956862 26789 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:12.959384 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.964553 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.936730  1811 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:27:12.972390 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.974924 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:12.978109 26789 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:12.979692 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.980044 26783 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:27:12.957813  1759 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:12.977352 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:12.988103 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.988838 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:12.996518 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:12.997236 26046 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
W20260709 12:27:13.001966 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.004554 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.008621 26048 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:13.009644 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.010834  1900 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.010913 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:13.011459  1900 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.012092  1795 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:13.012989  1795 consensus_queue.cc:1243] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 26.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:13.013528  1840 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:13.014212  1900 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:13.014183  1840 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.014943  1898 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:27:13.008540 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:13.019189 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.020007 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:13.021154 26199 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:13.023967 26045 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:13.024865 26200 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:12.966831  1793 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.027125 26054 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:13.029348 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.030277 26200 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:13.016601  1899 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:13.041997 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:13.046206 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:13.041671 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.048173 26194 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:13.049629 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:12.982918  1472 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.059619 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:13.061342  1905 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:13.068575 26045 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:13.070912 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:13.066062 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:13.074645 26052 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:27:13.075083 26052 pending_rounds.cc:85] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 11
I20260709 12:27:13.075320 26052 pending_rounds.cc:107] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:27:13.075541 26052 raft_consensus.cc:2889] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:13.078611 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.049072  1899 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:13.043728  1895 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:13.053900  1773 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:13.100800 26120 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:13.036590  1726 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:27:13.062952  1900 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:13.077503  1760 raft_consensus.cc:2764] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:13.077103  1803 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.074815  1882 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.077526  1727 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:13.101979  1899 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:13.008075  1747 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.104895  1749 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.074908  1791 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:27:13.103847  1773 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.077337  1611 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:27:13.112984  1840 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:13.147656  1901 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:27:13.147049  1773 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:13.146631  1611 consensus_queue.cc:1243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:13.146847  1749 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.174511  1749 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.175737  1472 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.156576  1904 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:13.176478  1904 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:13.177502  1904 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.178357 25944 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 16 to 18, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 18 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:13.181483 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.184180 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.185190 26200 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 18.
I20260709 12:27:13.153980  1903 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:13.127355  1803 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.199607 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:13.201925 26198 pending_rounds.cc:85] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 10
I20260709 12:27:13.202334 26198 pending_rounds.cc:107] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:27:13.202651 26198 raft_consensus.cc:2889] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:13.211644  1905 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:13.212193  1905 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:13.215854  1744 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:13.209823 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 100659us
I20260709 12:27:13.200342  1903 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.168776  1895 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.213281  1905 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.220534 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.312126  1640 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.276434 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.312803  1905 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:13.313091 26790 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
W20260709 12:27:13.296885  1907 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:27:13.314400  1842 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.314916  1842 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.276942 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:13.315752  1727 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:13.316311  1727 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:13.298310  1900 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:13.317346  1842 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.317552  1900 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.219050  1803 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.207801 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:13.263592  1472 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.319417  1472 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.321496  1472 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.283617  1918 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.322383  1918 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.323928 26202 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:13.324282 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.326054  1918 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.255744  1879 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.326894  1773 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.328709  1879 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.259840 25944 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 15 to 23, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 23 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.238806  1749 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:13.329624  1749 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.265720 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:13.332566 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.257004  1743 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.300227  1902 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.289s	user 0.003s	sys 0.000s
I20260709 12:27:13.335140  1743 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.335690 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:13.336018 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.341956 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.344519  1640 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:13.349246 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.349962 26199 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:13.350854 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:13.351348 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.351962 26197 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:13.353294 26203 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:13.353165  1899 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.355468  1899 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.351356 26199 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:13.360028 26785 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:13.360684 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.362597 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:13.363415 26791 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:13.360821 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.206293  1611 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:13.369889 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:13.364683 26787 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:13.283733  1840 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.418741  1840 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.419757 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49966.6us
I20260709 12:27:13.420351 26195 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:27:13.420586 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:13.422122  1840 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.417902 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:13.416894 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45581.3us
I20260709 12:27:13.425066 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.426563 26199 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
W20260709 12:27:13.374812 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.432678 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74206.6us
I20260709 12:27:13.452594 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:13.363986  1842 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:13.423767  1644 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:13.458288  1644 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.458804  1644 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.453620  1640 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:13.461046  1640 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:13.461472  1640 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:13.450712  1760 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:13.374593  1661 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 12:27:13.451148  1830 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:27:13.470989 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:13.472805 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:13.473361  1726 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:13.473786  1726 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.474270  1726 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:13.474334  1743 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:13.476122  1743 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:13.476639  1743 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:13.477520  1743 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.473714 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:13.478849 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.372874  1611 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.370376  1903 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:13.483345 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:13.483376 26791 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:13.482366  1749 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.486011  1916 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:13.486496  1916 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.484731  1769 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:27:13.487151  1916 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:13.490031 26757 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.496570 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.497225 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:13.372128  1844 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:13.492983 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:13.255910  1759 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.482016 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.371656  1744 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.112s
W20260709 12:27:13.532148 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 98698.2us
I20260709 12:27:13.538982 26045 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:13.545032 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:13.545241 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:13.546017 26046 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
W20260709 12:27:13.549019 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.551237 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54082.8us
I20260709 12:27:13.484376  1611 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:13.452387  1899 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:13.557510 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75572.3us
I20260709 12:27:13.489727  1879 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:13.564028 26048 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:27:13.494877  1682 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:13.488873  1681 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:27:13.498584  1917 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:13.567013 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.498412  1915 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:13.567621 26124 raft_consensus.cc:2376] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22. Re-sending same reply.
I20260709 12:27:13.544791  1744 consensus_queue.cc:1243] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:13.574007 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.566268 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:13.569478 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.571264 26121 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:13.572955 26128 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:13.578022 26045 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 21 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:13.580945 26128 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:13.581781 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.585005 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:13.561455  1795 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:13.588007 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.562022  1749 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:13.590118 26127 raft_consensus.cc:2368] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:13.530788  1560 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:27:13.576292 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:13.608161 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.063s
I20260709 12:27:13.606741 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:13.608700 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.005s	sys 0.063s
W20260709 12:27:13.609025 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 542995us
I20260709 12:27:13.609251 26126 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:13.610265 26791 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:13.610713 26125 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:13.611724 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.611886 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:13.573688  1840 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:13.566407  1682 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.564193  1879 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:13.632902 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.637419 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.568454  1773 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.551753  1727 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.636479 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.648942 26791 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
W20260709 12:27:13.650983 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.652673 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.637507  1919 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.657137  1919 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.657708 26123 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:13.659550  1919 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:13.653720 26127 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:13.566759  1899 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.663679 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:13.667073 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 246549us
I20260709 12:27:13.670176 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.633191  1795 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:13.633733  1682 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:13.637751  1749 raft_consensus.cc:2764] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:13.672621 26791 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:13.686508 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:13.687183 26045 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:13.688364 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:13.661736 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 89181.5us
I20260709 12:27:13.633538  1769 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:13.674283 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.689998 26126 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:13.690080 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:13.690954 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.629333 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.647166  1644 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.694708 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:13.702708 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.655263  1842 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.696837 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.705721 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:13.706933 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.707717 26126 raft_consensus.cc:2376] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23. Re-sending same reply.
I20260709 12:27:13.709110 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.693380 26128 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 23 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 12:27:13.695441 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.711095 26127 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:13.716513 26791 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:27:13.695415  1919 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:13.647756  1879 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.665251  1678 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:27:13.722785 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:13.740342 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51941us
I20260709 12:27:13.742802 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.734867 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.751315  1759 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.752107  1925 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:13.754134  1744 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:13.720896  1803 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:13.743080 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:13.758775 26200 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:13.755261 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.759809 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.764891  1830 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.728854 26785 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:13.765666 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:13.765944  1560 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.766469  1560 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.766522  1640 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.729162 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:13.768182 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.726219 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.743822 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:13.729526 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.727979 26195 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:13.694945  1661 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:27:13.745131  1727 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:13.745517  1922 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 12:27:13.745847  1918 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:13.746130  1889 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.097s	user 0.008s	sys 0.000s
I20260709 12:27:13.718961  1842 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.746974  1919 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:13.790365  1919 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:13.782752  1897 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.822s	user 0.005s	sys 0.000s
I20260709 12:27:13.792330  1842 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.792479  1919 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:13.741953 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.729846 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:13.725926 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.795279 26122 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:27:13.795591  1899 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.726405 26201 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:13.756400  1455 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.801573 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.721711  1844 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:13.775498  1681 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:13.775136 26196 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:13.783627 26128 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:27:13.729117  1472 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:27:13.797538 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.795969 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:13.765337  1830 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:13.768641  1795 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.804432 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62544.8us
I20260709 12:27:13.829480 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:13.831283 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.802382 26201 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:13.783017  1744 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.819259 26195 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:13.772294 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.833760 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:13.837265 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:13.853555 26787 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
W20260709 12:27:13.854880 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.819136  1727 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:13.764875 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:13.868822 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 93910.6us
W20260709 12:27:13.879621 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48394.5us
I20260709 12:27:13.881484 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.815860  1455 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.860605  1681 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:13.861583  1749 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:13.901901  1749 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.902329  1749 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:13.904275  1727 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:13.910043  1455 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.912461  1640 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.864660  1560 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.859546  1882 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.834537  1726 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.916714  1882 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.918666  1882 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.859737  1472 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:13.920502 26129 raft_consensus.cc:2368] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:13.818184  1844 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.861775  1744 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.922479  1844 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.919891  1472 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.894160 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.893340  1235 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:13.924711  1235 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.918678  1726 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.927488  1235 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:13.897614  1935 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:27:13.901396  1681 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:13.889701 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 125943us
I20260709 12:27:13.923825  1744 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:13.893989 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.930208  1611 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.890588  1923 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:13.931143  1900 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:13.929513  1681 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:13.932665  1727 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:13.932984  1903 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.933591  1727 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:13.934741 26783 raft_consensus.cc:2368] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27.
I20260709 12:27:13.935148  1903 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:13.935388  1900 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:13.884338  1759 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:13.936640 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:13.936462  1759 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.937716 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46764.5us
I20260709 12:27:13.935773  1900 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:13.939275 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:13.868016  1932 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:13.939657  1900 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:13.940222  1932 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.940073 26123 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:13.940968  1900 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:13.939482  1759 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:13.942502  1932 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:13.943408  1899 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:13.936800  1923 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:13.950906  1235 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.966825  1235 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.938709 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:13.953174  1932 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:13.864364  1830 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:13.951843 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:13.947759  1823 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:13.911840  1682 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:13.957903  1931 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.093s	user 0.001s	sys 0.001s
I20260709 12:27:13.960410  1611 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:13.974653 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 264731us
I20260709 12:27:13.983048 26122 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:13.976029 26123 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:13.982414 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:13.984269  1235 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:13.996273 26050 raft_consensus.cc:1217] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 18.10->[20.11-24.13]   Dedup: 20.11->[22.12-24.13]
I20260709 12:27:13.997035 26050 pending_rounds.cc:85] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 11
I20260709 12:27:13.997316 26050 pending_rounds.cc:107] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 12:27:13.997627 26050 raft_consensus.cc:2889] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.009105 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:14.013058 26124 raft_consensus.cc:2368] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25.
I20260709 12:27:14.012836 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.013934 26784 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:14.014717 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.015466 26787 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:14.016248 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:14.016548 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.016006 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.017863 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:14.018522 26788 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:14.019229 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:14.020076 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.021095 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.021415 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:14.021136 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:14.023470 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.023756 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.024158 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:14.025214 26047 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:14.024650 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:13.976783  1903 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:13.985298  1744 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:13.982066  1726 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.030249 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.024070 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:14.027395  1918 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:27:14.032949  1744 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:14.033491  1744 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.033788  1840 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:14.033960  1744 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.027585  1790 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.007s
W20260709 12:27:14.035321  1929 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:27:14.017674  1830 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:14.034550  1933 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:14.032393 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:14.005307  1895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.011s	sys 0.020s
I20260709 12:27:14.028642 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:14.021477  1661 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:13.976646  1917 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:27:13.976585  1773 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
W20260709 12:27:14.041764  1895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.012s	sys 0.021s
I20260709 12:27:14.068372 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:14.017508  1925 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:27:14.020495  1611 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:27:14.034309  1840 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:14.028736  1899 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:14.031661  1903 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:14.037813  1916 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 12:27:14.014038  1640 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:27:14.021723  1923 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
W20260709 12:27:14.038180 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 426334us
I20260709 12:27:14.044028  1747 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:14.067011  1749 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.028496  1919 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:14.030650  1844 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:14.036760  1644 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:27:14.038882  1726 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.046525  1661 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:14.046229  1790 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:14.106020 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.082883 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:14.108055 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:14.118573 26789 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:14.099305  1903 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.099465  1743 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.094869  1932 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:14.117100 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.124763 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.135881 26788 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:14.099939  1795 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:14.112782  1455 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.136478 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 118739us
I20260709 12:27:14.144856 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.146886  1749 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.148926  1816 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:14.123831  1661 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:14.153148  1905 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:14.171864 26789 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:14.095607  1842 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
W20260709 12:27:14.192358 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 168344us
I20260709 12:27:14.298298 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.128134  1644 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:14.108881  1916 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.140961  1844 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:14.192871  1455 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.331671  1844 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:14.250694 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 105912us
I20260709 12:27:14.239419 26126 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:14.153594  1795 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.333307  1844 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.335425  1844 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.336557 26782 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:14.336719  1795 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.221131  1943 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:14.337821  1943 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:14.338560  1943 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:14.338606 26123 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:14.231524  1944 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:14.141703  1743 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:14.234300 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.333370 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.342181 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:14.232895  1749 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:27:14.288503 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.408459  1795 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.409065  1795 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.143592  1790 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.131s
I20260709 12:27:14.411451  1795 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:14.412860  1954 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:27:14.413502  1895 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.414522  1895 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.295658  1905 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:14.261817  1809 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:14.417335  1809 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:14.417239  1905 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.418597  1905 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.418515  1809 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.419751  1809 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.299048 26122 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:14.420688  1949 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.421139 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.421446  1958 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.421943 26122 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:14.423184 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:14.284579  1937 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
I20260709 12:27:14.282701  1856 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.425908  1957 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:14.426265  1856 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.302380  1932 raft_consensus.cc:476] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Not starting pre-election -- already a leader
I20260709 12:27:14.319033 25944 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 15 to 21, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 21 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:14.428726  1749 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.303038  1816 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.429294  1749 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.431049  1816 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:14.425979  1941 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:14.331396  1920 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.569s	user 0.001s	sys 0.002s
I20260709 12:27:14.340142 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.283517  1682 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:14.343405 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:14.399432 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55279.5us
I20260709 12:27:14.444314 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:14.516965 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 117090us
I20260709 12:27:14.537720 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:14.538403 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 195344us
I20260709 12:27:14.539753 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.412379  1919 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:14.260576  1611 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:14.284364  1942 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 12:27:14.515954 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:14.547339 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 411216us
I20260709 12:27:14.534749 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.474177  1918 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:14.541630 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 419246us
I20260709 12:27:14.533198 26123 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:14.544363 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.537726 26169 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:14.337839  1842 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:14.451829 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.456166  1895 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:14.569144 26121 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
W20260709 12:27:14.456102  1953 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:14.541317  1932 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.431877  1681 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.341207  1944 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:14.595642  1944 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.597402  1944 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.599251  1943 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:14.603482  1744 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 12:27:14.435240 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.606480 26125 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:14.607339  1744 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:14.560729 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.432072  1749 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:14.561424 26122 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:14.458092  1743 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.563222 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:14.614012  1744 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:14.614586 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51454.4us
I20260709 12:27:14.614666  1744 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.615301  1744 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.454818  1726 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.466193  1455 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.449640  1856 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:14.477902  1916 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.450351  1879 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:14.567332 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:14.309083  1640 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.246s
I20260709 12:27:14.547997  1759 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:14.460724  1900 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.001s
I20260709 12:27:14.550568  1682 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.624091 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:14.589740 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45615.5us
I20260709 12:27:14.624912 26785 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:14.625190 26018 leader_election.cc:400] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27.
I20260709 12:27:14.625578 26018 leader_election.cc:403] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:14.627161 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:14.627640 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.550756  1560 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:14.630232 26784 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:14.631124 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:14.563818  1918 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:14.592850 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:14.448870  1661 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:14.469808  1939 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.368s	user 0.001s	sys 0.003s
I20260709 12:27:14.605510  1681 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:14.569952  1830 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 12:27:14.611467 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3968) took 1042 ms. Trace:
I20260709 12:27:14.593040  1611 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:14.655268 26120 rpcz_store.cc:276] 0709 12:27:13.569086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.570157 (+  1071us) service_pool.cc:224] Handling call
0709 12:27:13.591287 (+ 21130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.591630 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.591698 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.614893 (+ 23195us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.978472 (+363579us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:14.192082 (+213610us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.232220 (+ 40138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.232366 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.611238 (+378872us) raft_consensus.cc:1685] finished
0709 12:27:14.611258 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.611404 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32857,"prepare.queue_time_us":317082,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":1770,"raft.queue_time_us":41096,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":22584,"thread_start_us":483195,"threads_started":2,"wal-append.queue_time_us":454083}
W20260709 12:27:14.619253  1823 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.002s
I20260709 12:27:14.568783  1842 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:14.657792  1823 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:14.463271  1903 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.658527  1823 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:14.658931  1823 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.659277  1823 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.660442 26200 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:14.560249 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 435518us
I20260709 12:27:14.661759 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.662132 25944 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 14 to 21, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:14.663295 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.477589  1840 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.620623 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.432029  1941 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.641412 26051 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:14.642534 26783 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.547060  1899 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:14.649227 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.649336 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:14.649906  1682 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.650288  1640 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:14.650985 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:14.651539  1749 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:14.653872 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3888) took 1158 ms. Trace:
I20260709 12:27:14.701721 26054 rpcz_store.cc:276] 0709 12:27:13.495944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.496279 (+   335us) service_pool.cc:224] Handling call
0709 12:27:13.496707 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.504128 (+  7421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.504176 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.505941 (+  1765us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.954946 (+449005us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:13.954987 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.955056 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.469125 (+514069us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:14.653530 (+184405us) raft_consensus.cc:1685] finished
0709 12:27:14.653557 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.653808 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6950,"raft.queue_time_us":35264,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8946,"spinlock_wait_cycles":13706112,"thread_start_us":448713,"threads_started":1,"wal-append.queue_time_us":558132}
I20260709 12:27:14.652068  1759 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:14.673489 26757 leader_election.cc:336] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.778s (SENT)
I20260709 12:27:14.704206 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.705217 26047 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:14.706207 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:14.674294 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.707054 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:14.663829 25944 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 20 to 22, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 22 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:14.664989  1944 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:14.421182  1949 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.421983  1958 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.620050  1932 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.708206 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:14.709345 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:14.682899  1903 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.709658 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:14.719669 26783 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:14.679117  1830 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:14.727964 26785 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:14.688027 26788 raft_consensus.cc:2410] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:27:14.682744  1560 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:14.677898 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.732478 25944 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 16 to 19, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 19 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:14.733733  1943 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.683109  1941 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:14.678864  1611 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:27:14.735502  1840 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:14.736423  1678 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:14.738896  1759 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.687685  1455 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.740579 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:14.711831  1900 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:14.741240 26782 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:14.742772  1932 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:14.751929  1960 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.738178  1682 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.748301  1856 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:14.728178  1795 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.755690  1811 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.753738 25944 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:14.760108 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 82312.1us
I20260709 12:27:14.771907 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:14.772576  1961 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.742939  1949 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.766377 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:14.753677  1916 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:14.767160  1840 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:27:14.771265  1811 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.749462  1962 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:14.770653  1795 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:14.777336 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.762210  1844 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:14.741195  1899 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.793723 26196 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:14.792308 25944 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 15 to 23, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 23 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:14.748106  1918 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.768848  1743 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:14.805625 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:14.769275  1932 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:14.807330 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:14.807269 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 269630us
I20260709 12:27:14.828576 26200 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:14.773198 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:14.836270 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 63126.3us
I20260709 12:27:14.837019  1967 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:14.837596  1967 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:14.838482  1967 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.840684  1560 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:14.806013  1823 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:14.846746 26020 leader_election.cc:400] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:14.847117 26020 leader_election.cc:403] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:27:14.847831  1823 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:14.848331  1560 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:14.833146 26169 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:14.773097 25947 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 287) took 106841 ms (client timeout 15000 ms). Trace:
I20260709 12:27:14.849720 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:14.861301 26195 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:14.788035  1611 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:14.766255  1455 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:14.780491  1773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.008s	sys 0.005s
W20260709 12:27:14.805089  1923 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.011s	sys 0.011s
I20260709 12:27:14.787472  1856 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:14.853677 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45545.9us
W20260709 12:27:14.886276 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.849215  1790 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:14.858206 25944 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:14.804636  1844 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:14.833021  1963 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:14.826764  1743 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:14.886083 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53066.9us
I20260709 12:27:14.804665  1962 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:14.858089 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 290839us
W20260709 12:27:14.889799  1773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.013s	sys 0.013s
W20260709 12:27:14.891230 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 274765us
I20260709 12:27:14.904683 25944 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:14.907270 26169 leader_election.cc:400] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25.
I20260709 12:27:14.773141  1961 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.928889 26169 leader_election.cc:403] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
W20260709 12:27:14.892307  1923 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.015s	sys 0.018s
I20260709 12:27:14.856377  1749 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:27:14.933928 26783 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:27:14.936482  1793 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:14.937418  1795 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:14.885447  1744 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:14.853652  1951 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 12:27:14.945416 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:14.887683  1790 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:14.964381 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:14.966116 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:14.953778 25944 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 13 to 22, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 22 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:14.890244  1844 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.967782 26198 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:14.969403 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:14.907824  1823 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:14.890290  1769 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.901120  1760 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:14.910373  1962 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.908985  1963 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:14.891191  1964 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:14.975396 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66997us
I20260709 12:27:14.940115  1682 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.982402  1915 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:14.984264  1856 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.986937  1811 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:14.971609  1773 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:27:14.989382  1955 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.542s	user 0.002s	sys 0.000s
I20260709 12:27:14.945448  1661 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:14.992517  1760 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.993654  1974 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:14.970266  1903 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:27:14.994735  1919 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:14.996531  1830 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Leader election lost for term 25. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:14.999059  1840 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:14.998792  1941 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.033285  1915 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:15.003997 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:15.005376 25944 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 13 to 20, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:15.008111 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.055891 26788 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:15.008993  1769 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:14.977833  1823 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:14.974295  1969 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
W20260709 12:27:15.036744 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70695.5us
I20260709 12:27:15.025388  1958 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:15.064976 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:15.026439  1975 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:15.051926  1941 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.000383  1967 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:15.047973  1560 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:14.961171 26195 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:15.057895 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:15.086521 26787 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:15.085652 25944 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 13 to 17, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 17 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:15.058722 26788 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:15.024672  1682 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:15.039716  1882 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.007s	sys 0.011s
I20260709 12:27:15.042996  1919 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:15.037406 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64528.9us
W20260709 12:27:15.106093 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 211438us
I20260709 12:27:15.107240 26169 leader_election.cc:400] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:15.107738 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3891) took 1112 ms. Trace:
I20260709 12:27:15.086371 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:15.062932  1823 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.030301  1790 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.107546 25944 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:15.108649 26050 rpcz_store.cc:276] 0709 12:27:13.995246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.995497 (+   251us) service_pool.cc:224] Handling call
0709 12:27:13.996004 (+   507us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:14.008669 (+ 12665us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:14.008704 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:14.009393 (+   689us) log.cc:844] Serialized 52 byte log entry
0709 12:27:14.130045 (+120652us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:14.130122 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.130261 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.988849 (+858588us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:15.100786 (+111937us) raft_consensus.cc:1685] finished
0709 12:27:15.100807 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.107676 (+  6869us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4883,"raft.queue_time_us":38708,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3797,"spinlock_wait_cycles":77692672,"thread_start_us":120106,"threads_started":1,"wal-append.queue_time_us":300513}
W20260709 12:27:15.110097 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.107968 26169 leader_election.cc:403] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.017964  1964 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.067345  1749 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:15.119823 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.120817 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.122059 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:15.122315 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:15.123003 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.098423 26787 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:15.127461 26197 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:15.105405  1882 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.007s	sys 0.011s
I20260709 12:27:15.129860 26203 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:15.134979 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:15.135931 26203 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:15.130352 26196 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:15.130662 26198 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.130983 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:15.145056 26198 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:15.147176 26198 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:27:15.128394 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:15.149415 26201 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:15.101557  1678 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:15.187940 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80819.2us
I20260709 12:27:15.105327  1958 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:15.194833 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:15.199934 26203 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:15.197248 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.205305 26050 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:15.107343  1983 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:15.210819 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.213686 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.213511 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.245373 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.246820 26050 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:15.101977  1560 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.252575 26054 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:15.253275 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.253854 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.253938 26203 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:15.132632  1900 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:15.153402  1987 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:15.111789  1941 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:15.254635 26050 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:15.252628 26051 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:15.099876  1967 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.251014 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:15.260663 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.198112  1968 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.357075 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.200851  1960 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:15.370105 26196 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:15.355782 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.369971 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.371953 26051 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:15.192816  1744 consensus_queue.cc:1175] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 21 status { last_received { term: 21 index: 14 } last_committed_idx: 12 last_received_current_leader { term: 21 index: 14 } }
I20260709 12:27:15.378197 26052 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:15.372545 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:15.214711  1790 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.376312 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.385535 26053 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 19.
I20260709 12:27:15.387023 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.387852 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:15.376479 26047 raft_consensus.cc:2368] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23.
I20260709 12:27:15.249660  1990 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.398921 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.393615 25944 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:15.395179 26049 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:15.077263  1963 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.401057 26784 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:15.424751 26198 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:15.418673 25944 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 19 to 20, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:15.400400 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 141431us
I20260709 12:27:15.155064  1749 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.452013 26050 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:15.463282 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 252330us
W20260709 12:27:15.379477  1944 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.651s	user 0.002s	sys 0.009s
I20260709 12:27:15.465458 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.341085  1961 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:15.464845 26019 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:15.466735 26198 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:15.473479 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:15.476576 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 206181us
I20260709 12:27:15.478188 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.478637 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:15.397454  1976 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.380s	user 0.001s	sys 0.000s
I20260709 12:27:15.490556 26050 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:27:15.469167 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.374434  1988 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:15.206401  1678 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.462095 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.484265 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.250053  1974 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:15.489955 25944 catalog_manager.cc:5697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 2 to 16. New cstate: current_term: 16 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:15.495424 26054 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:15.396687  1455 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.451401  1903 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.530534 26052 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:15.463629  1773 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.535800 26054 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: [27.13-27.13]
I20260709 12:27:15.477773  1944 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27.
I20260709 12:27:15.383872  1968 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.499184  1988 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:15.546705 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 173311us
I20260709 12:27:15.370707  1941 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.548398 26051 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:15.554566 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:15.476892  1994 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.557219 26047 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:27:15.249136  1919 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.247982  1915 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.569892 25944 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:15.573081 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.370783  1987 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.533413  1760 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.532021  1992 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:15.594826 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:15.589521 26049 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:15.579870 25944 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 13 to 18, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 18 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:15.580582 26054 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:15.587677 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3979) took 1289 ms. Trace:
I20260709 12:27:15.569449  1900 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:15.621443  1989 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 12:27:15.621837  1900 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:14.860661 25947 rpcz_store.cc:269] 0709 12:25:27.931433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.931864 (+   431us) service_pool.cc:224] Handling call
0709 12:27:14.772991 (+106841127us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:07.492724 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:07.493603 (+   879us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:07.493655 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:27:07.493672 (+    17us) tablet.cc:662] Decoding operations
0709 12:27:07.494147 (+   475us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:07.494245 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:07.494271 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:27:07.494598 (+   327us) tablet.cc:701] Row locks acquired
0709 12:27:07.494649 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:27:07.547121 (+ 52472us) write_op.cc:270] Start()
0709 12:27:07.569103 (+ 21982us) write_op.cc:276] Timestamp: P: 1783600027498075 usec, L: 0
0709 12:27:07.569154 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 12:27:07.570437 (+  1283us) log.cc:844] Serialized 541 byte log entry
0709 12:27:07.853072 (+282635us) op_driver.cc:464] REPLICATION: finished
0709 12:27:08.065986 (+212914us) write_op.cc:301] APPLY: starting
0709 12:27:08.066084 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:08.067791 (+  1707us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:08.067930 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:08.185025 (+117095us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:08.185278 (+   253us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:27:08.185339 (+    61us) write_op.cc:312] APPLY: finished
0709 12:27:08.231315 (+ 45976us) log.cc:844] Serialized 25 byte log entry
0709 12:27:08.266867 (+ 35552us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:08.276957 (+ 10090us) write_op.cc:454] Released schema lock
0709 12:27:08.293509 (+ 16552us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:08.293934 (+   425us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:08.293963 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":661724,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":208494,"apply.run_cpu_time_us":5036,"apply.run_wall_time_us":227918,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8313,"num_ops":1,"prepare.queue_time_us":184243,"prepare.run_cpu_time_us":14108,"prepare.run_wall_time_us":209633,"raft.queue_time_us":113484,"raft.run_cpu_time_us":41133,"raft.run_wall_time_us":107011,"replication_time_us":283366,"thread_start_us":311916,"threads_started":4,"wal-append.queue_time_us":159228,"wal-append.run_cpu_time_us":25687,"wal-append.run_wall_time_us":1610719}]]}
I20260709 12:27:15.624455  1682 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.624691 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.380861  1983 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.564699  1988 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:15.633957 26203 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 16.
I20260709 12:27:15.468438  1990 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.628296 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.627871 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.555719  1773 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.616266 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.642125 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:15.645802 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.618623 26045 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:15.586846  1944 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:27:15.620499  1678 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.620800  1941 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.694736 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.695564 26051 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:15.696718 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:15.697367 26051 raft_consensus.cc:2376] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20. Re-sending same reply.
I20260709 12:27:15.698458 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.699394 26051 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:15.597046 26129 rpcz_store.cc:276] 0709 12:27:14.298705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.655547 (+356842us) service_pool.cc:224] Handling call
0709 12:27:14.655874 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.685544 (+ 29670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.685582 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.686094 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.857676 (+171582us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:14.857869 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.857940 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.587384 (+729444us) raft_consensus.cc:1685] finished
0709 12:27:15.587415 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.587595 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29280,"raft.queue_time_us":31732,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":16332,"thread_start_us":170959,"threads_started":1,"wal-append.queue_time_us":284654}
I20260709 12:27:15.577996  1997 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.549237  1903 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:15.676987  1999 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:27:15.677413  1760 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:15.678098 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.568992  1994 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:15.680217  1986 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.565s	user 0.001s	sys 0.000s
I20260709 12:27:15.680546  1769 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.682397  1983 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.639223 26054 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 17.
I20260709 12:27:15.688033 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:15.631995  1919 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.701722 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.748594 26051 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:27:15.749552 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.750777 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:15.710806  1844 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.001s
I20260709 12:27:15.677446  1560 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:15.727144  1975 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.001s
W20260709 12:27:15.729177  1991 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:27:15.729858  1964 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.679636  1987 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:15.745811 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57892.1us
I20260709 12:27:15.738430  1941 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:15.744912  1678 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:15.788676  1678 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.791302  2007 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.791970  2007 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.767030  1844 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.603731  1992 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.637919  2000 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.121s
I20260709 12:27:15.769872 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.738858  1997 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.855105  1997 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.862323  1743 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:15.754175  1969 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:15.758607  1944 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:15.850298 26195 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:15.922724 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:15.812913  1941 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:15.852193  1974 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:15.852716  1678 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:15.869715  1972 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.841s	user 0.007s	sys 0.000s
I20260709 12:27:15.881108  1844 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.883404  1943 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:15.883687  1968 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:15.886265  1903 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.736371  1961 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:27:15.921953  1944 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:15.889710  1455 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.889871  1992 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.890565  1542 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.038175  1542 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:15.897666  1947 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.483s	user 0.012s	sys 0.023s
W20260709 12:27:15.745970 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 281169us
I20260709 12:27:15.737865 26047 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:15.745895 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:15.923731 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.010s	sys 0.010s
W20260709 12:27:16.007953 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.018s	sys 0.014s
W20260709 12:27:16.008720 26050 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.001s	sys 0.001s
I20260709 12:27:16.010133 26051 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:27:16.012581 26046 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.001s	sys 0.001s
W20260709 12:27:16.030078 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.030081 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.023720 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3984) took 1723 ms. Trace:
I20260709 12:27:16.204484 26127 rpcz_store.cc:276] 0709 12:27:14.299993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.558560 (+258567us) service_pool.cc:224] Handling call
0709 12:27:14.558977 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.559360 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.559390 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.559905 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.805370 (+245465us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:14.805407 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.805463 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.868827 (+1063364us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:16.022500 (+153673us) raft_consensus.cc:1685] finished
0709 12:27:16.022586 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.023379 (+   793us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":87,"spinlock_wait_cycles":6264192,"thread_start_us":245169,"threads_started":1,"wal-append.queue_time_us":417227}
W20260709 12:27:16.031626 26199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.006s	sys 0.015s
I20260709 12:27:16.032796 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.033907 26198 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
W20260709 12:27:15.932101  1882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.002s	sys 0.019s
I20260709 12:27:16.035802 26789 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:16.036746  1472 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:15.891332  1950 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.444s	user 0.004s	sys 0.003s
W20260709 12:27:15.746877 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 192450us
W20260709 12:27:15.913242  1979 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.851s	user 0.004s	sys 0.001s
W20260709 12:27:15.887424  1986 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:15.848984  1749 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.765282  1760 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:15.902985  1773 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:15.910499  1823 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.002s
I20260709 12:27:15.901886  1918 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:15.765949  1967 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:15.773022  1975 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25.
I20260709 12:27:15.894536  1990 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:15.905090  2004 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.968698  1988 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.956569  1944 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:15.927522 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1933) took 1363 ms. Trace:
I20260709 12:27:16.065220  1811 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:15.932052  1943 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:16.097831  1952 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.228767 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3983) took 1928 ms. Trace:
I20260709 12:27:16.097227  1455 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:15.946750  1970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.014s	sys 0.038s
I20260709 12:27:16.102953 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.105684  1949 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.207945 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.260536  1949 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.215024 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.251791  1988 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.262036 26125 raft_consensus.cc:2376] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23. Re-sending same reply.
I20260709 12:27:16.266988 26052 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:27:16.206768 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 402011us
I20260709 12:27:16.255657  1811 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.273000 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.248466 26782 rpcz_store.cc:276] 0709 12:27:14.563687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.563858 (+   171us) service_pool.cc:224] Handling call
0709 12:27:14.564156 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:14.763842 (+199686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.927177 (+1163335us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:15.927422 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406403840}
I20260709 12:27:16.112424  1971 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.275411  1971 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.109069  1542 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.117079 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:15.971932  1925 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:15.975831  1994 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:16.184803  1932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.009s	sys 0.018s
I20260709 12:27:15.946404  1941 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.261219 26127 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:15.928102  1743 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.945874  2007 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:15.929435  1969 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.249s
I20260709 12:27:16.213600 26121 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:16.217614 26046 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.002s	sys 0.001s
W20260709 12:27:16.218063 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.010s	sys 0.010s
W20260709 12:27:16.223724 26199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.006s	sys 0.015s
I20260709 12:27:16.228459 26054 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:27:16.217712 26050 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.002s	sys 0.001s
W20260709 12:27:16.217811 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.019s	sys 0.014s
I20260709 12:27:16.210860 26047 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:16.216940 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.217263 26789 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:16.217044 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 2993) took 2454 ms. Trace:
I20260709 12:27:16.212782 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:16.238823  2004 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.209770 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.229997 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 3982) took 1930 ms. Trace:
W20260709 12:27:16.217321  1882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.002s	sys 0.019s
I20260709 12:27:16.214316 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.213546 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.272552  1952 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:15.950277  1962 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.934s	user 0.010s	sys 0.001s
W20260709 12:27:16.280623  1970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.014s	sys 0.040s
I20260709 12:27:16.217830  1472 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.232097  1760 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.275322  1811 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.278175  1943 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.186781 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.284281 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:16.233597  1823 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.264006  1949 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.254099 26124 rpcz_store.cc:276] 0709 12:27:14.299731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.660675 (+360944us) service_pool.cc:224] Handling call
0709 12:27:14.661072 (+   397us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:14.707402 (+ 46330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.707432 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:14.730768 (+ 23336us) log.cc:844] Serialized 78 byte log entry
0709 12:27:14.864444 (+133676us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:14.864492 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.864566 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.912342 (+1047776us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:16.228020 (+315678us) raft_consensus.cc:1685] finished
0709 12:27:16.228059 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.228562 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68984,"raft.queue_time_us":66797,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3089,"spinlock_wait_cycles":17491200,"thread_start_us":100902,"threads_started":1,"wal-append.queue_time_us":275747}
W20260709 12:27:16.313433 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.318758 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.319621 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1118) took 1071 ms. Trace:
I20260709 12:27:16.320371 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.316571 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.261349  1944 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:16.298802  1932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.009s	sys 0.018s
I20260709 12:27:16.315860 26128 raft_consensus.cc:606] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:16.312505  1773 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.334420 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.337142 26123 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:16.334008 26128 raft_consensus.cc:614] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:16.339493 26198 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:27:16.342157 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:16.342330 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.14746e+06us
I20260709 12:27:16.343357 26198 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:16.343472  1900 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:16.344720 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.344614 26125 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:16.345306 26789 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.345587  1773 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.344625  1900 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.351950  1900 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.262066  1842 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.353510  1842 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:16.297390 26126 rpcz_store.cc:276] 0709 12:27:14.299477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.773538 (+474061us) service_pool.cc:224] Handling call
0709 12:27:14.773932 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.833457 (+ 59525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.833501 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.854615 (+ 21114us) log.cc:844] Serialized 26 byte log entry
0709 12:27:15.010617 (+156002us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:15.027725 (+ 17108us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.032003 (+  4278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.032105 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.229481 (+1197376us) raft_consensus.cc:1685] finished
0709 12:27:16.229539 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.229792 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37224,"prepare.queue_time_us":18341,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":67152,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":9200,"spinlock_wait_cycles":1637760,"thread_start_us":81661,"threads_started":1,"wal-append.queue_time_us":167971}
I20260709 12:27:16.353668 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1361) took 1102 ms. Trace:
I20260709 12:27:16.355614  1994 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:16.358568  1994 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.359107  1994 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:16.356029 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.314471  1455 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:16.357081 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.356477 26051 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:16.344025 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:16.291538 26194 rpcz_store.cc:276] 0709 12:27:13.762364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.762475 (+   111us) service_pool.cc:224] Handling call
0709 12:27:13.762859 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.763255 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.763288 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.763806 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.027850 (+264044us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:14.027900 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.027977 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.896907 (+1868930us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:16.216416 (+319509us) raft_consensus.cc:1685] finished
0709 12:27:16.216472 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.216838 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6214,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":7279,"spinlock_wait_cycles":279747840,"thread_start_us":263657,"threads_started":1,"wal-append.queue_time_us":650363}
W20260709 12:27:16.365778 26020 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.511s (SENT)
W20260709 12:27:16.367592  1919 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.008s
I20260709 12:27:16.357856  1997 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.368613 26196 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:27:16.368511  1997 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.369823  1823 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.370488 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.370625 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:16.372383 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.355041 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 168740us
I20260709 12:27:16.372712 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.353015 26197 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:16.372159 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:16.373404 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84161.2us
I20260709 12:27:16.365825 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.371877 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.323876 26046 rpcz_store.cc:276] 0709 12:27:15.248048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.248185 (+   137us) service_pool.cc:224] Handling call
0709 12:27:15.377123 (+128938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.319004 (+941881us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:16.319081 (+    77us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.319174 (+    93us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:16.319208 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.319340 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.319534 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24294,"raft.queue_time_us":253774,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":1320,"thread_start_us":895010,"threads_started":1}
W20260709 12:27:16.374009 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 102061us
I20260709 12:27:16.355507 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1115) took 1110 ms. Trace:
I20260709 12:27:16.379964 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.352838 26203 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:16.320530  1962 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.378087 26045 rpcz_store.cc:276] 0709 12:27:15.245015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.245135 (+   120us) service_pool.cc:224] Handling call
0709 12:27:16.355009 (+1109874us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540022b354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:16.355260 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735654144}
W20260709 12:27:16.396978 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.396711 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.399957  1904 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:16.400646  1904 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.401201  1904 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:16.402091  1952 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.402891  1943 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:16.405488 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.406437 26203 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:16.406345 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.409368 26202 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:16.411026  1962 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.411237 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.402942  1925 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:16.412266 26127 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:16.414050  1925 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.415774  1925 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.354844 26201 rpcz_store.cc:276] 0709 12:27:15.251348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.251503 (+   155us) service_pool.cc:224] Handling call
0709 12:27:15.251837 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.586281 (+334444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.586317 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:15.593035 (+  6718us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.035037 (+442002us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:16.035164 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.035278 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.353209 (+317931us) raft_consensus.cc:1685] finished
0709 12:27:16.353240 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.353465 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147666,"raft.queue_time_us":198386,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":17947,"thread_start_us":569019,"threads_started":2,"wal-append.queue_time_us":615025}
I20260709 12:27:16.420279 26196 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:16.423002  2019 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:16.423694 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50550.1us
I20260709 12:27:16.424513 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2592) took 1079 ms. Trace:
I20260709 12:27:16.424805 26128 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:16.433391 26198 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:16.425613 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.372571  2004 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.411062 26129 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:16.435803 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88843.3us
I20260709 12:27:16.441210 26120 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:27:16.376999  1971 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.448902  2007 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:16.449200 26020 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.521s (SENT)
I20260709 12:27:16.449769  2007 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:16.451125 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 167924us
I20260709 12:27:16.431571 26048 rpcz_store.cc:276] 0709 12:27:15.344637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.495547 (+150910us) service_pool.cc:224] Handling call
0709 12:27:16.026674 (+531127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.027387 (+   713us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:16.152840 (+125453us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.285802 (+132962us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.290458 (+  4656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.290519 (+    61us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:16.297654 (+  7135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.303928 (+  6274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.424128 (+120200us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54000f8550. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:16.424276 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25558,"prepare.queue_time_us":386126,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":5943,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":6266,"spinlock_wait_cycles":1098356992,"thread_start_us":224809,"threads_started":2,"wal-append.queue_time_us":276630}
I20260709 12:27:16.453095 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3000) took 1330 ms. Trace:
I20260709 12:27:16.451295 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:16.439061 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.473101 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.463755 26049 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
W20260709 12:27:16.461046  1759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.131s	user 0.008s	sys 0.031s
W20260709 12:27:16.474571 26755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.016s
I20260709 12:27:16.487011 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.487965 26049 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:27:16.487586  2019 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.490505  2019 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.492985 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.495893  1919 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:16.496454  1919 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.496858  1919 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:16.497936  2007 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:16.500177 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44530.2us
W20260709 12:27:16.503109 26755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.012s	sys 0.017s
W20260709 12:27:16.479809  1759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.009s	sys 0.032s
I20260709 12:27:16.501938 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.453568 26199 rpcz_store.cc:276] 0709 12:27:15.122331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.122660 (+   329us) service_pool.cc:224] Handling call
0709 12:27:15.122930 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.123160 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:15.492348 (+369188us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.416936 (+924588us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.452567 (+ 35631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.452656 (+    89us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:16.452773 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.452894 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.453039 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57838,"prepare.queue_time_us":429736,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":100,"raft.queue_time_us":113885,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":1099422,"threads_started":2}
I20260709 12:27:16.501601  1455 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.514183  1455 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:16.514698 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 171021us
I20260709 12:27:16.515774 26789 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:16.517068  2030 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:27:16.474009  1472 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.536753 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.511663 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.524888  1988 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:16.524794 26054 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:16.528880 26195 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:16.531195 26128 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:27:16.533342 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.534466 26048 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:16.536412  1790 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.139s	user 0.013s	sys 0.014s
I20260709 12:27:16.508319  1967 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:16.474778  1773 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.528069 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.565743  1967 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:16.567147  1967 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.508090  1983 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:16.569470  1983 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.571153  1759 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:16.572026 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.572942 26124 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:16.574090 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.577087 26125 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:16.578230 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.578928 26125 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:16.569905 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2591) took 1320 ms. Trace:
I20260709 12:27:16.557897 26047 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:16.584802 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.577122 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1355) took 1777 ms. Trace:
W20260709 12:27:16.594785 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.585919 26202 rpcz_store.cc:276] 0709 12:27:14.799655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.800887 (+  1232us) service_pool.cc:224] Handling call
0709 12:27:14.801256 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.409847 (+1608591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.576775 (+166928us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400335c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:16.577028 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3697375104}
I20260709 12:27:16.608170 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.575734  1449 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:16.608557 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.562726  1997 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.579646 26050 rpcz_store.cc:276] 0709 12:27:15.249725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.491500 (+241775us) service_pool.cc:224] Handling call
0709 12:27:15.491917 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.492178 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:16.370514 (+878336us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.476689 (+106175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.501713 (+ 25024us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:16.554668 (+ 52955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.569534 (+ 14866us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.569818 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22912,"prepare.queue_time_us":855693,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":74,"raft.queue_time_us":29010,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6590,"thread_start_us":861923,"threads_started":2}
I20260709 12:27:16.628777 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.600888 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.638573 26046 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:16.642361 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.634212 26200 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:16.648594 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77040.7us
I20260709 12:27:16.651424 26120 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:16.659075 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109997us
I20260709 12:27:16.661857 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.648182 26756 leader_election.cc:400] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23.
I20260709 12:27:16.665347 26756 leader_election.cc:403] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:16.668802 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.679927  1915 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
W20260709 12:27:16.682277 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 233189us
I20260709 12:27:16.683399  1905 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23.
W20260709 12:27:16.683676 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 130199us
I20260709 12:27:16.686313 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:16.686446  1905 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:16.687361  1455 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.686641 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:16.727963 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.694598  1958 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:16.736584 26047 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:16.732564  1823 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.760051  1823 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.768817  2019 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.769680  2019 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.772379  1963 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:16.772797  2022 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.773927  2022 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.747645  1472 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.777930  1472 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.781888  1918 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.782827  1918 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.784751 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.786355 26120 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:16.787546 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.788959 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.791085 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.792446  1992 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.793004  1992 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:16.795169  2045 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.226s	user 0.001s	sys 0.002s
W20260709 12:27:16.800079 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.769093  2020 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:27:16.801600 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.801904  1718 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:16.802292  1718 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:16.802845 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.803251 26053 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
W20260709 12:27:16.804476 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61919.6us
I20260709 12:27:16.804760 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.805696 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.806460  1744 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.807262 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.807231  1744 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.808686 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.809939 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:16.811040 26050 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:16.811184 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.811975 26202 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:16.812036 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.812645 26194 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:16.813163 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.814245 26202 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:16.815531 26202 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:16.815891 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:16.816545 26199 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:16.817349  1899 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.818002  1899 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.818933  1744 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.817245 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:16.820394  1958 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:16.820649 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:16.821527 26046 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 19: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:16.821755  1958 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.824470  1952 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:16.824640 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2602) took 1199 ms. Trace:
I20260709 12:27:16.825073  1952 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.825702  1952 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.825147 26053 rpcz_store.cc:276] 0709 12:27:15.625331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.628057 (+  2726us) service_pool.cc:224] Handling call
0709 12:27:16.824389 (+1196332us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001b49d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:16.824576 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2678291712}
I20260709 12:27:16.827339  1993 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:16.827802  2035 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.265s	user 0.003s	sys 0.000s
I20260709 12:27:16.828002  1993 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.828605  1974 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.829092  1923 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.829178  1974 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.829826  1923 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.830777  1993 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.831590  1974 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.795225  1918 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.813289  2032 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:16.754621 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2616) took 1108 ms. Trace:
I20260709 12:27:16.803345  1718 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.762914  1823 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:16.862464 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.865768 26789 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:27:16.832157  1923 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.868445  1472 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.869822  1718 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.820096  1919 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.872422 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:16.872856 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.874473  1919 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:16.850425 26052 rpcz_store.cc:276] 0709 12:27:15.642228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.286852 (+644624us) service_pool.cc:224] Handling call
0709 12:27:16.518256 (+231404us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:27:16.518481 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.518915 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:16.519365 (+   450us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.739331 (+219966us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.741913 (+  2582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.741974 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:16.747905 (+  5931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.748038 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.748331 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2310,"prepare.queue_time_us":450,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":13055,"raft.queue_time_us":38537,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":23982,"spinlock_wait_cycles":14822144,"thread_start_us":116372,"threads_started":1,"wal-append.queue_time_us":294188}
I20260709 12:27:16.876451 26129 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:27:16.877660  1917 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:27:16.878253 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.878894  1449 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:16.879559 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.832736  1905 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:16.880519 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.881018 26196 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:16.881538 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2668) took 1468 ms. Trace:
I20260709 12:27:16.881800 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.882050 26200 rpcz_store.cc:276] 0709 12:27:15.387239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.387547 (+   308us) service_pool.cc:224] Handling call
0709 12:27:16.834123 (+1446576us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400398d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:16.855993 (+ 21870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962411904}
I20260709 12:27:16.882565  1842 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:16.883028  1842 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.880540  1455 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:16.884147  1455 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.884585  2022 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.880529  1905 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.881018 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.886049  1905 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.886404 26046 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:16.880919 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.891981 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:16.892140 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:16.892519 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:16.892948 26786 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:16.864984 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.893374 26049 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:16.895048 26784 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:27:16.896791 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:16.897014 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:16.897534 26788 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:16.898779 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:16.899085 26790 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:16.883548  1842 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:16.900576 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2663) took 1657 ms (client timeout 1664 ms). Trace:
W20260709 12:27:16.901012 26197 rpcz_store.cc:269] 0709 12:27:15.243139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.250640 (+  7501us) service_pool.cc:224] Handling call
0709 12:27:16.900362 (+1649722us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b5400399254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:16.900519 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1556606720}
W20260709 12:27:16.903083  1944 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 12:27:16.903869  1992 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:16.828387  1994 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.884646  1455 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:16.916426 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1940) took 2201 ms. Trace:
I20260709 12:27:16.904903  1899 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:16.903896  1952 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:16.907768 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:16.908346  1943 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.005s
I20260709 12:27:16.847054  2032 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.882951 26125 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:16.881450 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:16.882579 26054 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:16.899729 26791 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:27:16.874155 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.750916  1882 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:16.912487 26120 raft_consensus.cc:606] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:16.931690 26120 raft_consensus.cc:614] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:16.875460  1904 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.909052  1823 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:16.914134  2045 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:27:16.934212 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.940421 26129 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:16.916865 26789 rpcz_store.cc:276] 0709 12:27:14.715130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.715334 (+   204us) service_pool.cc:224] Handling call
0709 12:27:14.715620 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.035450 (+1319830us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540056b6a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:27:16.515565 (+480115us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b58007daed0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b7439bfe7 00007f5b7439d7de 00007f5b743a7b49 00007f5b743a8416 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4
0709 12:27:16.866758 (+351193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.916321 (+ 49563us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582779008}
I20260709 12:27:16.935065 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.945086 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:16.959769 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 85708.1us
I20260709 12:27:16.872129  1472 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:27:16.931623  1882 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.969461 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:16.949206  2022 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:16.964049 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.952872  1941 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:16.967733 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 281172us
I20260709 12:27:16.931813  2032 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:16.975617 26120 raft_consensus.cc:2368] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18.
I20260709 12:27:16.964550  1918 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.973722 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.978478 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.978703 26127 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:16.979508 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:16.979940  1949 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:16.980978  2022 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:16.974911  1882 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:16.989902 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:16.978513  1952 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.001535  1918 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.003592 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.004931 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.005517 26122 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:17.005760  1904 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.006409 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.991919  1542 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:17.009514 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.012882 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 329816us
W20260709 12:27:17.019177 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45642.5us
I20260709 12:27:17.020710 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:16.992813  1949 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.955747  1992 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:27:17.042662 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:17.048785 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83827.6us
I20260709 12:27:17.041400  1844 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:17.046803  1974 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.974363  1823 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.055743 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:17.063602 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.042500  1949 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.031236  1983 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.074743 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:17.072057  1918 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.095084 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.096499 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.066437  1844 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:17.112142 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69932.6us
I20260709 12:27:17.081043  1790 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:17.110838 26123 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:17.073553  1974 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.094756  1943 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:17.109634 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.121388 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:17.112643  1988 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.001s
I20260709 12:27:17.088801  1993 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:17.111930 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:17.124996 26123 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:17.126416 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:17.120599 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:17.133524 26049 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:17.117787  1472 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.117516  1790 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:17.105257  1992 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:17.121902  1943 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:17.132527  1988 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:17.157020 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.158048 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:17.150471  1790 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:17.115061  2007 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 12:27:17.155030  1943 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.165380 26121 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:27:17.168671  1472 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:17.179003  2038 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:27:17.176595 26129 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
W20260709 12:27:17.182564 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 84322.2us
W20260709 12:27:17.183084 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.184784 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.187593 26202 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:17.188040 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.171440  1472 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.199888  1941 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.200529  1941 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.202867  1944 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:17.203337  1944 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:17.160668  1993 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.159785  1919 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 12:27:17.212193 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.212891 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.209491 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:17.171828  1449 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:27:17.181100  1992 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.225409 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.225915  1682 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.226425  1994 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.227875  1974 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.228336  1951 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:17.234403  1951 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:17.234843  1951 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.234777  1949 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.236560 26790 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:27:17.229275  1947 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:27:17.237701 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46599.3us
W20260709 12:27:17.237915 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2679) took 1535 ms (client timeout 1794 ms). Trace:
I20260709 12:27:17.238619 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.238967  1974 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.240896  1542 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.241428  1542 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:17.241916  1823 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.242589 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.245812 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.247088  1983 raft_consensus.cc:2764] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:27:17.247908  1983 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.252920  1844 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.253787 26202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
new_leader_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:44736
I20260709 12:27:17.254530 26202 raft_consensus.cc:606] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Received request to transfer leadership to TS 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.230213  1744 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:17.255257  1842 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.256040  1744 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:17.257364  1988 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:17.190919  1963 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 12:27:17.257295  1744 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:17.258189 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.258818  1744 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.259442  2022 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.232831 26788 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:17.255754  1743 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.253911  2019 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.212577  2007 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:27:17.262071  2007 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:17.262455  2007 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:17.263381  2007 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.233325  1943 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.257810  1988 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:17.265722  1988 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.267127  1988 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.267359  1943 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:17.238340 26195 rpcz_store.cc:269] 0709 12:27:15.702117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.031837 (+329720us) service_pool.cc:224] Handling call
0709 12:27:17.237721 (+1205884us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400397454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:17.237859 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719925120}
W20260709 12:27:17.269416  2051 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:27:17.273195  1993 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.274206  1542 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.275483  1992 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.276954 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.277904 26195 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:17.278838  1743 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.280628 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:17.282464 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.282483 26201 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:27:17.268419  2039 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:27:17.285036 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:17.229768  1971 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:17.292321 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.295692  2022 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.296486  2022 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:17.296926  2022 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.297434  2032 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.298851 26120 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:17.296967 26196 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:17.288985 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:17.295859 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.314281 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.315181 26198 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:17.322649 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:17.316779 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:17.326442  1542 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.327316 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.294732  1882 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.334816 26202 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:17.327991 26124 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:17.336895 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:17.301086 26201 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:17.348863 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.349853 26197 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:17.344094 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:17.290170  1971 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.351626 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.352981 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.358217  1958 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:17.359681  1918 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:17.363576 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.364105 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.335948  1823 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:17.344528  1974 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:17.376794 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:17.377990 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:17.268082  1905 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.212555  1944 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.360673  1971 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.378809 26050 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:17.358666  1949 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.383797 26127 raft_consensus.cc:2368] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27.
W20260709 12:27:17.385404 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 93652.5us
I20260709 12:27:17.358688  1958 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:17.362164  1918 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:17.391494 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:17.393877 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.396054 26782 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:17.396250 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73710.2us
I20260709 12:27:17.392541 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:17.397255 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.374500  1823 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.381441  1974 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.380833  2007 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:17.412783 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.413226 26054 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:17.385170  2024 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.385921  1899 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.395448  1958 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:17.375506  1455 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.421792 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:17.422531 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.419688 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:17.414639  1823 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:17.427003 26169 leader_election.cc:400] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18.
I20260709 12:27:17.401070  1944 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.417551  1974 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.421252  1993 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:17.434221 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:17.422607 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 235803us
I20260709 12:27:17.414872  1967 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:27:17.466774 26122 raft_consensus.cc:2368] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20.
I20260709 12:27:17.431003  1904 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:17.469659 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:17.472870 26127 raft_consensus.cc:606] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:17.473199 26127 raft_consensus.cc:614] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:17.474154 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52456.9us
I20260709 12:27:17.435662  2032 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.488049 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.475524 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.486264  1904 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.460889  1992 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:17.424541  1899 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:17.497473 26167 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.667s (SENT)
I20260709 12:27:17.464624  1917 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:17.490654 26121 pending_rounds.cc:85] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:27:17.495085 26125 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
W20260709 12:27:17.503567 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 265036us
I20260709 12:27:17.508320 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:17.510272 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83406.3us
I20260709 12:27:17.503024 26121 pending_rounds.cc:107] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:27:17.514408 26121 raft_consensus.cc:2889] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.432116  2024 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:17.507869  2057 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.106s	user 0.001s	sys 0.001s
I20260709 12:27:17.495649  1992 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:17.505141  2019 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.517596  1842 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:17.531061  1718 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.002s
I20260709 12:27:17.534083  1941 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.531239  1992 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.540689  1882 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.520432  1917 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.565284 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:17.532364  2032 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:27:17.540652 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:17.570335 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43628.6us
I20260709 12:27:17.577711 26128 raft_consensus.cc:606] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:17.587468 26128 raft_consensus.cc:614] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:17.515121  1918 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:17.596653 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60785.9us
I20260709 12:27:17.598045 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.566478  1905 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.584848  1943 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.602802 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:17.614423 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 107376us
I20260709 12:27:17.615428 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.607690 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.619417 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1362) took 2024 ms. Trace:
I20260709 12:27:17.620522  1904 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.624542 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:17.625206 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 243146us
I20260709 12:27:17.626076 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:17.626289  2024 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:17.626379 26167 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.687s (SENT)
I20260709 12:27:17.626750 26783 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:17.627019 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.627354  1951 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.627961  1988 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:17.629050  1962 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:17.629607 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:17.630553  2063 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:17.624607 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef606226bb434b95911632eec3ede435"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:17.631758 26128 raft_consensus.cc:606] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:17.632128 26128 raft_consensus.cc:614] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:17.629750  1951 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.635496 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.637609 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.636467 26201 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:17.597393  2004 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.619880 26203 rpcz_store.cc:276] 0709 12:27:15.595495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.407085 (+811590us) service_pool.cc:224] Handling call
0709 12:27:16.434632 (+ 27547us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:16.691180 (+256548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.691229 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:16.773344 (+ 82115us) log.cc:844] Serialized 78 byte log entry
0709 12:27:17.124345 (+351001us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:17.490173 (+365828us) log.cc:844] Serialized 14 byte log entry
0709 12:27:17.618181 (+128008us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b5400154b20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b74320733 00007f5b743a5dbf 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c
0709 12:27:17.618251 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.618333 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.618384 (+    51us) raft_consensus.cc:1685] finished
0709 12:27:17.618469 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.619181 (+   712us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5400163350. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:17.619344 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207199,"prepare.queue_time_us":143018,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":182351,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":16288,"spinlock_wait_cycles":179238528,"thread_start_us":680447,"threads_started":2,"wal-append.queue_time_us":439388}
I20260709 12:27:17.644117 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:17.649446 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:17.650789  2070 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:17.651402 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:17.651877  2070 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:17.652499  2070 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:17.653036  2070 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.653540  1952 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:17.654357  1952 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:17.657920  1882 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:17.659523  2068 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:17.659956  2068 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:17.660983  2068 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.662448  2068 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.664547 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:17.666268 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.667071  1917 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.667831  1952 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:17.605369  2072 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:17.674669 26201 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:17.675205 26197 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:17.585141  1718 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:27:17.674288  2072 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:17.678232  1760 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:17.678548  1919 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.679473 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.613618  1455 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.668043  1899 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.684882  1941 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.610608  1943 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:17.680416  1904 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.698448 26196 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:17.699146 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 170839us
W20260709 12:27:17.700496 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85179us
I20260709 12:27:17.640817  1988 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:17.716603 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:17.721088 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:17.721675 26201 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.620217  1790 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:17.724507 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47884.1us
I20260709 12:27:17.725296 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.729079 26123 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:17.729230 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.730072 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:17.695138  2068 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.731318 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:17.703187  1943 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.733451 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.732962 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:17.732555 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53152.6us
I20260709 12:27:17.739574 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:17.743589 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:17.745260  1919 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:17.745795  1919 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:17.746224  1919 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.746588 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:17.746768  1823 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.743687  1904 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.748123 26195 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:17.749565  1904 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:17.750146  1472 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 12:27:17.750607  1472 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.691780  2004 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.757202  2004 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.759989  2007 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 12:27:17.767041 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.768146 26783 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:17.774133 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:17.776749 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 150442us
I20260709 12:27:17.778528 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.779139 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.776204 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:17.780740 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.781356 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:17.782490 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51240.4us
W20260709 12:27:17.787480 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.789356 26195 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:17.795823 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.781960  2019 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:17.792470  1760 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:17.803097 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:17.808341 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.783878  2076 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:17.813055 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.589s (SENT) [suppressed 22 similar messages]
I20260709 12:27:17.793556  2032 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:17.816677 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51186.1us
I20260709 12:27:17.765489  2022 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.763953  1941 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:17.838054  1823 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.840562  1958 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.841638  1941 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:17.843679  1682 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.002s
I20260709 12:27:17.813432  1611 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.779835  2068 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:27:17.825603 26169 leader_election.cc:400] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27.
I20260709 12:27:17.802254  2019 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:17.850732 26169 leader_election.cc:403] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:27:17.798158  1842 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:27:17.824996  1967 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.844194  1682 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.868852 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:17.870949 26791 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.869452 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:17.854729  2019 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.890900 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.855760  1472 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:17.896759 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 83736.6us
I20260709 12:27:17.871627  1744 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.899201 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:17.811578  2072 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.930292  2080 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:17.931170  2080 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:17.931494  2080 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:17.903189  2086 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:27:17.903605 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.934271  2077 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:17.934604 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.874591  1941 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.936579  1941 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.939198  1941 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.939831  1941 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.894956  1958 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.857055  2076 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.917023  2019 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.917703  1823 raft_consensus.cc:993] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: : Instructing follower 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:27:17.918872  2070 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.925521 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.920096  1790 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:17.925405  1974 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.957763 26120 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:27:17.958289 26197 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:17.926508  1971 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.004s
I20260709 12:27:17.928145 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.917729  1963 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:17.929293  2065 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.352s	user 0.004s	sys 0.001s
I20260709 12:27:17.934796  2077 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.941429 26201 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:17.936082 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.909976  2022 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.943069 26791 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:17.965476 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:17.966420 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:17.945999 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46487.6us
I20260709 12:27:17.948618 26128 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:17.899690  1611 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.952095 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.954213  2019 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:17.955046  1842 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:17.955502  1993 raft_consensus.cc:1081] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Signalling peer 9e23389325a6411c9b0634f4f6af54c3 to start an election
I20260709 12:27:17.970790 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:17.956200  2070 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:17.956420  1899 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.967804 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:17.970813 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:17.973668 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:17.977370  2019 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:17.975039  1682 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:17.978096  2019 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.978703  1682 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:17.979142  1682 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.974959  1842 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.980675  1842 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:17.980095  1744 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27.
I20260709 12:27:17.985345 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:17.986186 26050 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:17.966547  2077 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:17.988829 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60801.9us
I20260709 12:27:17.989635 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:17.990833 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:17.992056  2066 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.992204 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.993093 26124 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:17.994493 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:17.994881  1917 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.995236 26121 raft_consensus.cc:2376] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21. Re-sending same reply.
I20260709 12:27:17.996063 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:17.996492  1455 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:17.996706 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:17.996882  2070 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:17.997104 26129 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:17.997546  2070 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:17.999099 26122 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:17.962836  1971 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:17.999100  2070 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.001264  1905 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:17.998225 26123 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:27:18.002354 26788 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:17.874683  1967 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:17.997542 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.007203 26126 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.995380  1917 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:18.021222 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:18.031834 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.032826 26047 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.035951 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:17.962862  1904 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:27:17.996903  1455 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:18.010696  1917 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:18.038498 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 219595us
I20260709 12:27:18.023185  2024 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:18.049623 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.050951 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:18.038568 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.055428 26169 leader_election.cc:400] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20.
I20260709 12:27:18.056036 26169 leader_election.cc:403] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:18.020924  1963 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:18.062201  1943 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.049804  1455 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.961491  2080 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.053591  2024 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:18.063392  2092 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:18.097425 26127 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:18.097465 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.140684 26123 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29.
I20260709 12:27:18.104208 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:18.109766 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.101641  1967 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.146108  1967 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:18.146632  1967 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:18.128793  2076 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:18.129829 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.150743 26049 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:18.135239 26050 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 17.
I20260709 12:27:18.136667 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:18.158470 26124 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:18.096652 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.160041 26120 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:18.125766  1611 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:18.160967  1611 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:18.161002 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.161693  1611 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.112921  1963 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.114174  2066 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:18.163795  1993 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:18.125769  2022 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:18.123749  1917 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.125494  2024 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:18.168179  2082 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.253s	user 0.003s	sys 0.000s
I20260709 12:27:18.126207  2092 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.101074  1943 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.169860  1943 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.168644 26052 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:18.170495  2092 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.170337  1943 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.101293  2019 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.172597 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.173198 26052 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:18.163899  1611 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.177949  2024 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.173705  1974 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:18.107331  1971 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:18.099089  2087 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:18.174443  2092 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:18.175451  1917 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.176537 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.180651 26120 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:18.164435  1993 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:18.181775  2092 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:18.126467  1455 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.182184  2092 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:18.182041  1993 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.182736  2092 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:18.184429 26121 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:18.184777 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94842.3us
I20260709 12:27:18.185761  2066 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:18.186118  2066 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:18.186484  2066 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.188508  1842 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.190383 26053 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:18.190667  1842 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.109453  1919 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:18.178467  1992 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.000s
I20260709 12:27:18.123811  2080 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.199316  2022 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 19. Current term is 20.
I20260709 12:27:18.193836  1919 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:18.147795  1967 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.200999  1992 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.202893  2092 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.203801  1992 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.213891 26046 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.238822 26198 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:18.248284 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.245911 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:18.249799 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.250337 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 451884us
I20260709 12:27:18.093632  2074 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.260468  1743 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.261273  1743 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.264151  1743 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.270748 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:18.282899 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.317075 26791 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:18.317970 26167 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.777s (SENT)
I20260709 12:27:18.290510 26053 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:18.278348 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5591569729999719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 12:27:18.210276  2066 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.289675  1899 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:18.364871 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 398499us
I20260709 12:27:18.339597 26047 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:18.325906  1971 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.329603  1760 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:18.327582  1943 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.371448 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:18.371984 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:18.316629  1904 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:27:18.340744  1455 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:18.400470 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 82551.4us
I20260709 12:27:18.400506 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.363440  1899 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.413687  1988 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.414346  1988 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.372156  1918 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.336764  1682 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.323395  2074 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.422304 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.422911 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.354056  2077 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.426280 26124 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:18.420895 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:18.431632 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60282.8us
I20260709 12:27:18.433617 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.439497 26124 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:18.503835 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:18.410517  1899 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.433972 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:18.505784 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:18.507128 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:18.509203 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.025s
W20260709 12:27:18.537289 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.006s	sys 0.025s
I20260709 12:27:18.537292 26201 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:18.433405 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:18.473049 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 499407us
I20260709 12:27:18.435230 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.539597 26125 raft_consensus.cc:2368] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18.
W20260709 12:27:18.518956 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.536924 26198 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:27:18.417696  2024 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.554701 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:18.432138  1790 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:18.556875 26202 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:18.551458  2091 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:27:18.428160  1472 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:18.464883 26120 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
 from {username='slave'} at 127.0.0.1:56104
I20260709 12:27:18.609100 26120 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:18.609516 26120 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.568751 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:18.610332 26198 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:18.507936  1971 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.581123  2092 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.535919  2022 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:18.555629  2004 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:27:18.597226  1682 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.598083  2066 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.597998 26046 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:18.601599  2074 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.619096  1971 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.557267  1988 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:18.583384  1941 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.000s
W20260709 12:27:18.643253 26126 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.636s	user 0.008s	sys 0.000s
I20260709 12:27:18.644600 26047 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:18.652549 26126 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:18.433050  1905 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:18.614482  2094 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:27:18.431272  1918 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.649693 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.662576 26053 raft_consensus.cc:2410] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 12:27:18.551071  2072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.006s	sys 0.023s
I20260709 12:27:18.434377  1760 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:18.679853  2072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.006s	sys 0.024s
I20260709 12:27:18.642277 26120 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.671227 26052 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:18.675256 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.676982  2077 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:18.608377  2024 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:27:18.661849  1941 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:18.688961 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:18.697453 26198 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:18.688073 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.698724 26047 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:18.692890 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.694753 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.700146 26125 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:18.636093  2022 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.703361 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.707243 26120 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.655958  1790 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:18.697191 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.711566 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.712378 26789 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:18.685663  1919 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.655882  1944 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.718240 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.692102  1682 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:18.725730  2108 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.727025 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.721552  1919 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.704990  1943 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:18.751821 26047 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:18.754076 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.755415 26051 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:18.755451 26125 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:18.757635 26757 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:18.763734 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.717368  2103 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.770208 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.773856 26052 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:18.758745  1941 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:18.752254  1993 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:18.785816 26783 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:18.764675  2114 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:18.786583 26198 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:18.788081 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:18.775748  2080 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
I20260709 12:27:18.757043  1992 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.782814  1941 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:18.794950  2105 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:27:18.751111  1944 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.796350 26019 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:18.783035  1971 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:18.813058 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.814421 26047 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:18.810017 26019 leader_election.cc:403] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:18.787143  1993 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.819445 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.761356  1823 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:27:18.756639  2108 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.805573  1941 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:18.824762 26789 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:18.771123  1682 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.760583  1943 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:18.830014 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44907.9us
I20260709 12:27:18.828902 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.829021 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.821182  1718 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.833925  1718 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.836549 26053 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:18.830832 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:18.765035  1905 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.821790  1993 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.844569 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.846146 26047 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:18.783809  2103 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.848389 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.822546  1971 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:18.849627 26789 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:18.811631  1992 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.853374 26053 raft_consensus.cc:1217] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Deduplicated request from leader. Original: 21.16->[23.17-24.18]   Dedup: 23.17->[24.18-24.18]
I20260709 12:27:18.855396 26053 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [23.17-24.18]
I20260709 12:27:18.856612  1971 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.857417  1682 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.862622  1918 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.863693 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.864401 26048 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:18.866245  1993 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.867045 26789 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:18.867395 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:18.868108 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.868988 26788 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:18.869235 26047 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:18.837416  2066 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:18.867069 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.873867 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.874668 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:18.892279 26782 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:18.902213 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.844790  1967 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:18.905038 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.905982 26121 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:18.907732 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.835633  2106 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:18.850942  2097 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.909377 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.909510 26127 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:27:18.920447 26127 pending_rounds.cc:85] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 12
I20260709 12:27:18.910297 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:18.920668 26126 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:18.921183 26129 raft_consensus.cc:2368] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:18.912132 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:18.922569 26789 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:18.913254 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:18.923378 26045 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:18.917676 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:18.908866 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:18.918629 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.924090 26127 pending_rounds.cc:107] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:27:18.924635 26128 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:18.924896 26127 raft_consensus.cc:2889] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:18.923326 26120 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:18.927228 26786 raft_consensus.cc:2368] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27.
I20260709 12:27:18.846879  1611 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:27:18.933939 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:18.934773 26047 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.907409  1967 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.848311  1963 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.073s
W20260709 12:27:18.926024  2116 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:27:18.931807 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:18.939693 26169 leader_election.cc:400] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29.
I20260709 12:27:18.942080 26169 leader_election.cc:403] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:27:18.943111 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.831166  1790 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:18.947121 26050 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:18.922787  1993 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.957085 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
W20260709 12:27:18.960035  2107 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 12:27:18.962870  2098 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.494s	user 0.000s	sys 0.003s
I20260709 12:27:18.938118  1967 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.971939 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:18.973824 26788 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:18.916093  2066 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:18.940517  2022 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:18.985846 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:19.007597 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.008522 26783 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:19.010594  2076 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:19.011010  2076 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:18.987739  1718 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:18.990094  1919 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:19.012007  2076 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.017099  2076 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:18.987282  1988 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.024677  1988 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:19.025472  2118 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:27:19.026894 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3929) took 1052 ms. Trace:
I20260709 12:27:19.027004  1988 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:18.929106  2097 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.027304 26054 rpcz_store.cc:276] 0709 12:27:17.974249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.003949 (+ 29700us) service_pool.cc:224] Handling call
0709 12:27:18.022305 (+ 18356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:18.399318 (+377013us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:18.399350 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:18.429198 (+ 29848us) log.cc:844] Serialized 52 byte log entry
0709 12:27:18.783422 (+354224us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:18.783458 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.783540 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.026633 (+243093us) raft_consensus.cc:1685] finished
0709 12:27:19.026655 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.026826 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156589,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":672059,"threads_started":2,"wal-append.queue_time_us":364847}
I20260709 12:27:18.990963  1993 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:18.987906  2072 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:19.049067 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.053283  2103 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:19.075006 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:19.078388 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 101594us
I20260709 12:27:19.079180 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.079663 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:19.082571 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.083091 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.083832 26047 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:27:19.084776 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.085373 26048 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.083773 26052 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:19.086314 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.038281  2022 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:27:19.093621 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.096684 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.097944 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.129887 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.133934 26047 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:19.100554 26783 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 21: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:19.091188  1899 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.002s
I20260709 12:27:19.131872 26054 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:19.136035 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.136126 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.135947 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.142035 26051 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:19.132401 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.132272 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:19.097517  2108 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.200781 26050 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:19.157588 26048 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 18.
I20260709 12:27:19.261844 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.148051 26045 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:19.251267 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.252698 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.073076  2072 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.253178 26784 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:27:19.253405 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.253737 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.254065 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.254803  2066 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:19.255998  2129 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:27:19.257609 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.258967 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.265276 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.276996 26789 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:19.252948 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:19.268090 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 135919us
I20260709 12:27:19.278443 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:19.278671 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.279549 26783 raft_consensus.cc:2368] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25.
I20260709 12:27:19.270932 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.271992 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.281030 26047 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.134661  1790 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:19.292166 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.293937 26054 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:19.303820 26791 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:19.304672 26787 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:27:19.307576 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.265188  2022 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.310597 26202 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:19.313030 26169 leader_election.cc:400] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 17 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18.
I20260709 12:27:19.317667 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.318486 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.318950 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.319641 26789 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:19.271956  1988 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:19.327144 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.319175 26782 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:19.322549 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.322324 26045 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:19.330449 26196 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:27:19.329809 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.328856 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.329633 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.138970  1967 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.142943  1899 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.334362 26783 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:19.334813 26047 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:19.334964 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.347512 26786 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
W20260709 12:27:19.335260 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84294.5us
I20260709 12:27:19.336615 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:19.337101  2128 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 12:27:19.327085  2022 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.340698 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.369994 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:19.322837  1790 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:19.342645 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.319931  2136 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:19.343851 26194 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:19.322654  2102 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:19.353092 26782 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:19.353778 26050 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:19.352329  1899 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:19.355723  2131 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:19.348526 26787 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:19.381986 26201 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:19.358785 26049 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:19.360338 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:19.341058 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.343165  1967 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.398551 26124 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:19.404536 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.407613 26122 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:19.413676  2022 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:19.393638  2102 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:19.421337  2102 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:19.424418 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 54435us
I20260709 12:27:19.425036 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.425899 26200 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:19.426093  2100 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:19.426782 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.426997  2100 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:19.428556  2130 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:19.428990  2130 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:19.429911  2130 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.431391  2130 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:19.426787 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.434043  1967 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:19.435178 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.436136  1967 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:19.436580  2097 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.436797  1992 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.437741  1790 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.438575 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.443117  1993 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29.
I20260709 12:27:19.446699 26788 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
W20260709 12:27:19.447413  2137 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:27:19.446734 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:19.448563  2099 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.850s	user 0.001s	sys 0.010s
I20260709 12:27:19.449146 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.449913 26194 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:19.450814  2132 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:19.451388  2132 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.452363  2092 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.452854  2092 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.453557  2132 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:19.455195  1918 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.455705  1918 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:19.457561  1941 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.001s
I20260709 12:27:19.458011 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.459255 26194 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:19.458133 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3036) took 1519 ms. Trace:
I20260709 12:27:19.436131  2072 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:19.459856 26194 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:19.460305 26203 rpcz_store.cc:276] 0709 12:27:17.939076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.939425 (+   349us) service_pool.cc:224] Handling call
0709 12:27:17.939828 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:17.940291 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.940326 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:17.940929 (+   603us) log.cc:844] Serialized 52 byte log entry
0709 12:27:18.319275 (+378346us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:18.628495 (+309220us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.647785 (+ 19290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.647892 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.447802 (+799910us) spinlock_profiling.cc:243] Waited 9.87 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:19.457833 (+ 10031us) raft_consensus.cc:1685] finished
0709 12:27:19.457931 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.458074 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18897,"prepare.queue_time_us":401188,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":21277,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":3309,"spinlock_wait_cycles":22702848,"thread_start_us":654635,"threads_started":2,"wal-append.queue_time_us":531677}
I20260709 12:27:19.462265 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.462836 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.463568 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.464186 26791 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 24 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:19.463184 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.464844  2092 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:19.465539 26787 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:19.466104 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1984) took 1223 ms. Trace:
I20260709 12:27:19.462857 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:19.466785 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 709137us
I20260709 12:27:19.467160 26203 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 19.
I20260709 12:27:19.467478  2077 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:19.467944 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.468150 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.470148 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.470927 26782 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:19.471812  1944 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:19.472384  2080 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.472880  2109 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:19.472939  2078 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:19.428038 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.468019  2077 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:19.474350 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.476861  2104 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.874s	user 0.004s	sys 0.000s
I20260709 12:27:19.479176 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:19.487275 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.500046 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:19.562861 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.509907 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.536063 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.509284 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.570012 26123 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
I20260709 12:27:19.561892 26200 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:19.489517 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.486428 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.571820 26122 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:19.510998 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.572845 26051 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:19.569056 26047 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:19.565905 26196 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:19.510568 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.464187  1951 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:19.526799 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:19.480165  2135 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:27:19.483181 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.491393 26124 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:19.506913 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66570.9us
I20260709 12:27:19.503057 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.562295  2074 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.579843 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.457975  1941 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.489946  1943 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:19.512324 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.498804 26203 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
I20260709 12:27:19.462447 26194 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.572219 26127 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:19.590456 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:19.591065 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 88578.8us
I20260709 12:27:19.591329 26125 raft_consensus.cc:2410] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 12:27:19.579428 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 79495.1us
I20260709 12:27:19.526608  2097 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:19.596918 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.597628 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:19.598376 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:19.598537 26018 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.589s (SENT)
I20260709 12:27:19.589234  1951 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:19.566736  2078 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.668926 26049 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:19.669729 26784 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:19.466493 26790 rpcz_store.cc:276] 0709 12:27:18.242353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.242476 (+   123us) service_pool.cc:224] Handling call
0709 12:27:18.290069 (+ 47593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:18.290321 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:18.740178 (+449857us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.754031 (+ 13853us) log.cc:844] Serialized 14 byte log entry
0709 12:27:19.465570 (+711539us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b1800319180. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:27:19.465740 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:19.465791 (+    51us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:19.465838 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.465905 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.466019 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":721107,"prepare.queue_time_us":118963,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":4789,"raft.queue_time_us":729526,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11522,"spinlock_wait_cycles":12168064,"thread_start_us":383611,"threads_started":1}
I20260709 12:27:19.670967 26784 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:19.668988 26203 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:19.625414 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.599166  2092 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:19.628261 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:19.673995 26124 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.633157  2077 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.674635 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.638262 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.639453  2132 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:19.641150 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:19.679298 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80782.4us
I20260709 12:27:19.679767 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:19.641934  2072 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:27:19.683334 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 92971us
I20260709 12:27:19.654218 26045 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:19.659240 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:19.660998 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62700.9us
I20260709 12:27:19.565479  2080 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.612639 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:19.637059 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.681362 26046 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:19.676740 26783 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:19.655246 26194 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.684278 26020 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.685036 26120 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:19.693267 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:19.565346  2109 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.695375  1951 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.683386  2068 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 12:27:19.671922  2066 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:19.698230 26018 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.640s (SENT)
I20260709 12:27:19.701222 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.696655 26020 leader_election.cc:400] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:19.704169 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:19.707034  2136 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:19.698248  1952 consensus_queue.cc:1175] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 26 status { last_received { term: 25 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:19.701372 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.673408  2097 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:19.721688 26018 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.669s (SENT)
I20260709 12:27:19.722303 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.722330  1944 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:19.723755  2127 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:19.724094  1992 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:19.721777  2096 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.123s	user 0.007s	sys 0.004s
I20260709 12:27:19.725034  1918 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:19.726195 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1975) took 2026 ms. Trace:
I20260709 12:27:19.699384  1943 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:19.726581  1919 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:19.727065  1919 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.727402  1919 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:19.697983  2019 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:19.726552 26785 rpcz_store.cc:276] 0709 12:27:17.700261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.700384 (+   123us) service_pool.cc:224] Handling call
0709 12:27:17.723228 (+ 22844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.782809 (+ 59581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.782845 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.819669 (+ 36824us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.290969 (+471300us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:18.291013 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.432588 (+141575us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:18.432636 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.721116 (+1288480us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:19.725972 (+  4856us) raft_consensus.cc:1685] finished
0709 12:27:19.725993 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.726136 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36581,"raft.queue_time_us":33946,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5265,"spinlock_wait_cycles":124095232,"thread_start_us":365011,"threads_started":1,"wal-append.queue_time_us":627574}
I20260709 12:27:19.719331 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.730310 26200 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:19.690186  2022 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:19.739990 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.747087 26783 raft_consensus.cc:2368] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25.
I20260709 12:27:19.646683  2074 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.750878 26785 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:19.752166 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.696558  2132 consensus_queue.cc:1243] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 25.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:19.753566 26052 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:19.710398  2066 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:19.756824 26783 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:19.756771 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.717401  1963 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:19.763091 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:19.764508 26045 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:19.732154  2108 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:19.767900 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.758360 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:19.769373 26783 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:19.745860  1941 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.795845 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:19.712486  2139 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:27:19.774863 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:19.779265 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.794714 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.745023  2022 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.799068  2108 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.802295  2078 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.803295  2097 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.763199  1963 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.808274 26052 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:19.801466 26783 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.764202  2066 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.821213 26053 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:19.825745 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:19.831717 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.832690 26785 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:27:19.833729 26785 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:19.835446 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.836145 26785 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:19.837085 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.837684  1899 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:19.839583 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:19.800750  2076 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:19.814098  2022 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.847532 26198 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:19.814360  2108 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.825153  1963 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.861270 26194 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:19.855021  2019 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.875443 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 176606us
W20260709 12:27:19.876276 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 196552us
W20260709 12:27:19.901893 26018 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.775s (SENT)
I20260709 12:27:19.908861 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:19.930251 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.904597  1974 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 12:27:19.905515  2019 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:19.959777  2019 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:19.960893 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59066us
I20260709 12:27:19.961817 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:19.962584 26197 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:19.954383 26783 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
W20260709 12:27:19.945050 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.887871  2136 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:19.859918  2076 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:19.903853  1760 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:19.975796  2019 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.976536  2080 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.977411  2074 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:19.985842  1899 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:19.989148 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:19.995229  1971 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:19.999517 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:20.000214 26197 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.000756 26785 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:19.996941  2019 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:19.999699 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 224931us
I20260709 12:27:19.987214  1472 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:20.017382 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.019179 26124 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:20.024312 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.021665 26197 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:27:20.026032 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 186549us
I20260709 12:27:20.026700 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.028256 26199 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:20.029609 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.030464 26200 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:19.997784  1971 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.014192  2068 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:20.036953 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:19.993438  1760 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.031937  1472 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:20.051265 26167 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.020s
W20260709 12:27:20.052023 26167 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.007s	sys 0.020s
W20260709 12:27:20.052482 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 549328us
I20260709 12:27:20.040433  1971 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.042403  2068 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:20.067664 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.069178 26200 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:20.044521  2092 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:20.099472 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.055076  2153 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:20.111426 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:20.112088 26200 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:20.113919 26054 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:20.101547  2159 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.101583 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.073427  2068 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:20.120814 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 159146us
I20260709 12:27:20.124928 26197 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:20.095494  1472 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.076495  2166 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:20.146988 26198 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:27:20.154413 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52937.7us
I20260709 12:27:20.165962 26198 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:20.166821 26200 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:20.167429  2160 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:20.153311  2164 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.168802  2164 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:20.170189 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 102600us
I20260709 12:27:20.171391 26201 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:20.171283 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:20.057847  2136 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:20.157354  2150 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
I20260709 12:27:20.175201 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.177178 26203 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:20.180003 26203 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:20.168016  2160 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:20.173923 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.142642  2168 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.180804 26203 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.194017 26203 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.177743  2139 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.192734  2166 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:20.160737  1472 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:20.202973  2155 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:27:20.203766  2161 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:20.205473 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:20.195371  2147 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:27:20.207422 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:20.210667 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1395) took 1111 ms. Trace:
W20260709 12:27:20.196791  2146 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:27:20.210520 26199 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:27:20.218866 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.212299 26788 rpcz_store.cc:276] 0709 12:27:19.096902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.098036 (+  1134us) service_pool.cc:224] Handling call
0709 12:27:20.204812 (+1106776us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400680454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:20.207988 (+  3176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710673152}
I20260709 12:27:20.121739  2153 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.218098 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:20.221244 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.222747 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:20.221468 26203 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 25 index: 20. (term mismatch)
I20260709 12:27:20.227249  1944 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:20.228657  1944 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.191816  2092 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.230786 26049 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:27:20.236146 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71427.6us
I20260709 12:27:20.207940  2166 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:20.272996  2166 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.281406 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2634) took 1183 ms. Trace:
I20260709 12:27:20.281840 26046 rpcz_store.cc:276] 0709 12:27:19.098429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.098648 (+   219us) service_pool.cc:224] Handling call
0709 12:27:19.138055 (+ 39407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.688594 (+550539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.281167 (+592573us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:20.281343 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248881408}
I20260709 12:27:20.276149 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:20.228785  2151 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:27:20.232344  1963 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:20.233167  2163 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:27:20.285594  1993 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:20.283991  1472 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:20.286553  1993 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:20.286744  1963 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:20.286988  1993 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:20.222554  2130 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:20.287124 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:20.287621  2130 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:20.288697  1918 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:20.289098  1918 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:20.288601  2130 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.290024  1918 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.291059  1918 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.207453  2171 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:20.238704  2162 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:27:20.295645 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.224072  1790 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.265769  1944 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:20.229332  2019 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:20.235205  2164 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:20.264988  1760 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:20.306638  2077 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.308797  1951 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.299626  2168 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:20.236871  2127 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 12:27:20.311843  2153 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:20.306017  1919 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.009s
I20260709 12:27:20.232831  2022 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.293139  2102 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:27:20.302587  2076 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:20.309927  2160 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:20.324436 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56933.4us
I20260709 12:27:20.341856 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:20.346026  2136 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:20.302085  1790 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.348160 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.349149 26046 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:20.353667 26194 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:20.343254 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:20.330469 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66017.8us
I20260709 12:27:20.356330 26124 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:27:20.361766 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.225226 26203 pending_rounds.cc:85] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 19
I20260709 12:27:20.323467  2168 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:20.363454 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.371093 26123 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:20.363659 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.371351 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.371719 26126 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:20.371948 26124 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:20.371636 26049 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:27:20.364459 26127 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:20.354491  2136 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:20.369324 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 73830.2us
I20260709 12:27:20.375399 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.348421  1963 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.377002 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:20.352406  2019 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:20.322019  2068 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.380771 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:20.381057 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.387957 26128 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.388883 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.389657 26127 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:20.390100 26046 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:27:20.391382 26018 leader_election.cc:336] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.668s (SENT)
I20260709 12:27:20.391784 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.392846 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.394289 26123 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:20.395213 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.395924 26126 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:20.396862  2097 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:20.397452  2097 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.400490  2019 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:20.401844  2019 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.399796 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:20.385463 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.328536  1919 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.380919 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:20.408414 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:20.409140 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.409349  2171 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:27:20.400730  2173 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:27:20.410754  2168 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.412448  2167 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:20.415413 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:20.420074 26018 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.952s (SENT)
I20260709 12:27:20.414722  2167 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:27:20.421911 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:20.381976 26203 pending_rounds.cc:107] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 24.20
I20260709 12:27:20.351891  1760 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:27:20.347836  2139 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:27:20.427695 26203 raft_consensus.cc:2889] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:20.387480  2164 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:20.427237 26018 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.687s (SENT)
I20260709 12:27:20.426682 26128 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:20.445622 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:20.447062 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.423655  2077 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:20.472406 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.475164 26123 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:20.387280  2181 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:20.481150  2139 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:20.481684  2139 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:20.482669  2139 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.484854  2136 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:20.423230  2130 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:20.495767  2109 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:20.445776  2171 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:20.464437  2092 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.496217 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.387640  1472 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
W20260709 12:27:20.445533  2158 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:27:20.437325  2078 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:20.510385 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:20.509443 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 100777us
I20260709 12:27:20.448450  2177 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:20.524961 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.525118 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.523518 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:20.525944 26123 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:20.518272 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93690.9us
I20260709 12:27:20.425974  2097 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:20.523592 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.526790 26128 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:20.527840 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.529645 26127 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:20.531275 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.532025 26203 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:20.478119  2077 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.488986  2181 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.497539  2139 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:20.443396  1899 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 12:27:20.538867 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.539323 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.539661 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.416703  1963 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:20.545652 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.546012  1823 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.555923 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.545682 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.557206 26203 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:20.561892 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:20.530548  2154 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:20.564575 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:20.567888 26782 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:27:20.570242 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.541529  2077 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:20.540624  2078 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:20.497519  2092 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.525714  2164 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:20.575464 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.576398 26126 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:27:20.523912  2153 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:27:20.576910 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1995) took 1104 ms. Trace:
I20260709 12:27:20.542176  2181 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:20.579780 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2000) took 1092 ms. Trace:
I20260709 12:27:20.581881 26782 rpcz_store.cc:276] 0709 12:27:19.472012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.472128 (+   116us) service_pool.cc:224] Handling call
0709 12:27:20.567310 (+1095182us) spinlock_profiling.cc:243] Waited 8.99 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:27:20.567540 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.568343 (+   803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.576845 (+  8502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20687616}
I20260709 12:27:20.583809 26198 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:20.585300 26198 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:20.589205 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.582180 26784 rpcz_store.cc:276] 0709 12:27:19.487730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.718763 (+231033us) service_pool.cc:224] Handling call
0709 12:27:19.726589 (+  7826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.726997 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.727028 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.728353 (+  1325us) log.cc:844] Serialized 26 byte log entry
0709 12:27:20.006174 (+277821us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:20.006210 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.006284 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.579540 (+573256us) raft_consensus.cc:1685] finished
0709 12:27:20.579562 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.579711 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":276969,"threads_started":1,"wal-append.queue_time_us":417096}
I20260709 12:27:20.569443  2154 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.593357 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.596148 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:20.565181  1823 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.497120  2171 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.606289 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:20.608207 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:20.609455 26788 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:20.612564 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:20.613237 26784 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:20.614207 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:20.614979 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:20.614979 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.611963 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:20.618505 26167 leader_election.cc:400] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25.
I20260709 12:27:20.562999  2166 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.621187 26791 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 12:27:20.594673  2154 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:20.622479 26788 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:20.624042 26791 pending_rounds.cc:85] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 17
I20260709 12:27:20.626453 26791 pending_rounds.cc:107] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 12:27:20.598039  2022 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:20.626785 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.599566  2187 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:20.624744 26788 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:20.631719 26784 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:20.584090  2136 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.035s
I20260709 12:27:20.633368 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.633889 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.634982 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.637540 26784 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:20.611776  1899 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.638461 26789 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:20.627061  1790 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:20.722774 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 187968us
W20260709 12:27:20.718525 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 85240.3us
I20260709 12:27:20.626833 26791 raft_consensus.cc:2889] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:20.648803  2168 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:20.737958 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 104181us
I20260709 12:27:20.737429 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.739519 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.720832 26788 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.740919 26788 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.732421 26784 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.718672  2068 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:20.749593 26756 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.750682 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.755406 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:20.743630  2189 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:27:20.723569  2187 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:20.726066  2191 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.760612 26784 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:20.755976  2076 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:20.770714 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:20.763377 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.759611  1790 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.780977  1823 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:20.790163  2193 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:27:20.791844 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:20.793242  1951 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:20.794240  2199 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:20.794764  2199 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.795174  2199 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.795316 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.796586  2022 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:20.804768  2183 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:27:20.818497 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:20.819536 26128 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:20.823655 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1529) took 1195 ms. Trace:
I20260709 12:27:20.824121 26121 rpcz_store.cc:276] 0709 12:27:19.628107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.645340 (+ 17233us) service_pool.cc:224] Handling call
0709 12:27:19.657057 (+ 11717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.657317 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.657356 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:19.657416 (+    60us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:19.657452 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.657505 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.823422 (+1165917us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5400185bd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:20.823578 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25947008}
W20260709 12:27:20.824839  2186 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:27:20.826185 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:20.830251  2197 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:20.837162 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 409916us
W20260709 12:27:20.839926  1944 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 12:27:20.840459  2130 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.840710  2092 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:20.840979 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 327403us
I20260709 12:27:20.902387 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:20.841174  1963 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:20.730448  2164 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.843261 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.843854  2019 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:20.844609  2108 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.939s	user 0.010s	sys 0.003s
I20260709 12:27:20.736557  2078 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.845989  1760 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.845890  1823 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:20.743036  1899 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:20.849081  2166 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:20.772601  2068 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.078s
I20260709 12:27:20.850865 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:20.904492  1971 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.851s	user 0.007s	sys 0.006s
I20260709 12:27:20.771920  2201 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:20.858899 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2687) took 1600 ms. Trace:
W20260709 12:27:20.924154  1918 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.007s
I20260709 12:27:20.765690  2168 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:20.863222 26018 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.107s (SENT)
W20260709 12:27:20.868547 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74809.1us
I20260709 12:27:20.776479  2076 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:20.917265 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:20.915829 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72636.8us
W20260709 12:27:20.925820 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 176279us
I20260709 12:27:21.008334 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.922650  2157 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:21.009902 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.008450 26195 rpcz_store.cc:276] 0709 12:27:19.258181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.258317 (+   136us) service_pool.cc:224] Handling call
0709 12:27:20.858644 (+1600327us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:20.858815 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179697408}
W20260709 12:27:21.013705 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96594.1us
I20260709 12:27:21.014217 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:20.918870  1941 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:20.860229  2191 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.085s
W20260709 12:27:21.019570 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1488284830002158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
I20260709 12:27:20.918097  2159 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.092s
W20260709 12:27:20.933090  2132 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.092s	user 0.015s	sys 0.009s
I20260709 12:27:21.023958 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:20.914325  2195 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:21.028137 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:21.029395 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.036638 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:20.915218  1944 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.020262  2076 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.923568  2108 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.025645  2019 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:21.035104  2132 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.189s	user 0.015s	sys 0.010s
I20260709 12:27:20.931506  1760 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.945075  1823 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.008333  1971 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.036265  2201 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:21.052032 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.074347 26195 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:21.055162 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.059484 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.053103  2130 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:21.051537  1918 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:27:21.058576  2022 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:27:21.063150  1971 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:21.063526  1823 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.063736  2176 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:21.066605  1760 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.086876 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:21.087580 26120 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:21.086895 26122 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:21.089148 26122 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:21.067893 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2638) took 1926 ms. Trace:
I20260709 12:27:21.070801 26196 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:21.076632 26788 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 12:27:21.091547 26788 pending_rounds.cc:85] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 16
I20260709 12:27:21.091998 26788 pending_rounds.cc:107] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:27:21.092370 26788 raft_consensus.cc:2889] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:21.082361 26129 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:21.051120  2195 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:21.056849  2092 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:21.091284 26048 rpcz_store.cc:276] 0709 12:27:19.141910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.296999 (+155089us) service_pool.cc:224] Handling call
0709 12:27:19.326755 (+ 29756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.333125 (+  6370us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:19.333158 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.355825 (+ 22667us) log.cc:844] Serialized 26 byte log entry
0709 12:27:19.622968 (+267143us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b4800080358. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:27:20.820869 (+1197901us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b18000e3080. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:27:21.011591 (+190722us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:21.011649 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.011779 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.067661 (+ 55882us) raft_consensus.cc:1685] finished
0709 12:27:21.067683 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.067828 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1991,"raft.run_cpu_time_us":3305,"raft.run_wall_time_us":275736,"spinlock_wait_cycles":45704192,"thread_start_us":190514,"threads_started":1,"wal-append.queue_time_us":1708007}
W20260709 12:27:21.085386  2066 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.231s	user 0.015s	sys 0.000s
I20260709 12:27:21.105451  2066 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.051669  2157 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 12:27:21.107848  2066 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.085588  2019 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:27:21.108812  1944 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:21.111649  2181 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.000s
I20260709 12:27:21.083004  2198 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:21.123648 26196 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:21.125427 26196 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:27:21.126358  2181 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.129688  2108 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.033928  2187 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:20.723181 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:21.123911  2209 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.134425 26201 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:21.134970  1899 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.137028  2181 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.079434  2168 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:21.084090  2097 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:21.089941 26122 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.141283 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3048) took 1236 ms. Trace:
I20260709 12:27:21.148763 26194 rpcz_store.cc:276] 0709 12:27:19.905336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.905521 (+   185us) service_pool.cc:224] Handling call
0709 12:27:19.905861 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:20.354250 (+448389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.140934 (+786684us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:21.141204 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029145344}
W20260709 12:27:21.139220  2154 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.000s
I20260709 12:27:21.084285  2176 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:21.165652 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 302435us
I20260709 12:27:21.167539 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:21.174363 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 154020us
I20260709 12:27:21.154089 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.154397 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.119412  2019 consensus_queue.cc:1243] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:21.154347 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:21.155764 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 292170us
I20260709 12:27:21.160625 26122 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:21.147733 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 119733us
W20260709 12:27:21.179816 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.181612 26129 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:21.169450  2109 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
W20260709 12:27:21.184140 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:21.185890 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.155359  1944 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:21.189778 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:21.190649 26126 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:21.199515 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.199854 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.200258  2181 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:21.202383  2210 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.141s
I20260709 12:27:21.215507 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.174705  2164 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
W20260709 12:27:21.210623 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.210306 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.188500  2197 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:27:21.227532 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.228384  1952 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:21.228952  1952 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:21.229893 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.228330 26194 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:21.211385  2215 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:21.211673  1899 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:21.211860  2211 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:21.239559 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.212548  2167 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:21.233212 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.220435  2168 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:21.234773 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.241606 26791 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:21.235126 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.242661 26122 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:27:21.242803 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.163579  2154 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:21.212405  1988 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
W20260709 12:27:21.233770  2139 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.003s
I20260709 12:27:21.216853  2108 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.247371 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.250145 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:21.216198  2077 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.635s	user 0.005s	sys 0.003s
I20260709 12:27:21.253104 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.192422  1944 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.252646 26782 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:21.277757  1974 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:21.277896 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.278503  1974 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:21.278743  1944 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.209656  2195 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.284097 26122 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:27:21.238274  2177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.009s	sys 0.022s
I20260709 12:27:21.257553 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.285584  2080 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:21.286063 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.244217  2167 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:21.286903  2167 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:21.246512  2019 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:21.289760  2145 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.448s	user 0.009s	sys 0.000s
I20260709 12:27:21.291376 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2691) took 1953 ms. Trace:
I20260709 12:27:21.293218 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.293895 26784 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:21.294821 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.295163  2176 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.262809  1790 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.296103  2127 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:21.296646  2127 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:21.297698  1790 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.297634  2127 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.255702 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:21.298319  1790 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.298907  2127 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:21.287860  2167 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.245450  2215 consensus_queue.cc:1243] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:21.264390  2190 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:27:21.246452  2197 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.236022  2210 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.300879  2197 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.301584  2210 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.303059  2197 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.232323  2213 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:27:21.303882  1790 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:21.230544  2072 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:27:21.305694  2077 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:21.276850  2216 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:27:21.284689  1472 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 28 current term: 28 is higher.
I20260709 12:27:21.285720 26125 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:21.274896  1993 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.264133 26198 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:21.244498  2181 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.252915 26791 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:21.318274 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.319669 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.291761 26202 rpcz_store.cc:276] 0709 12:27:19.337732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.360916 (+ 23184us) service_pool.cc:224] Handling call
0709 12:27:19.361299 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.361566 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.361593 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.362135 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:27:19.735784 (+373649us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:19.735835 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.735925 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.256202 (+1520277us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:21.290870 (+ 34668us) raft_consensus.cc:1685] finished
0709 12:27:21.290893 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.291312 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27026688,"thread_start_us":373265,"threads_started":1,"wal-append.queue_time_us":478690}
I20260709 12:27:21.321836 26791 raft_consensus.cc:2410] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:27:21.295922 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.322727 26784 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:21.324005 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.311336 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1996) took 1839 ms. Trace:
I20260709 12:27:21.286154  2080 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:21.265194  2154 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:21.307487  2177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.010s	sys 0.023s
I20260709 12:27:21.327059 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:21.329600 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:21.336205 26121 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:21.315413  2199 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.004s
I20260709 12:27:21.338616 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.339984 26791 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 24 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:21.341113 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.342103 26784 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:27:21.334955 26782 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
W20260709 12:27:21.318571 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75624.9us
I20260709 12:27:21.324761 26786 rpcz_store.cc:276] 0709 12:27:19.472367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.480846 (+  8479us) service_pool.cc:224] Handling call
0709 12:27:19.481162 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.672281 (+191119us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:19.690800 (+ 18519us) log.cc:844] Serialized 14 byte log entry
0709 12:27:20.283931 (+593131us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b1800336100. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:27:20.384322 (+100391us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:20.384372 (+    50us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:20.384410 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.311089 (+926679us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b540056aed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:21.311145 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.311272 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2153,"prepare.queue_time_us":17907,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1088,"raft.queue_time_us":4796,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":483881088,"thread_start_us":97975,"threads_started":1,"wal-append.queue_time_us":873479}
I20260709 12:27:21.344005 26126 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:27:21.344610 26782 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:21.344318 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:21.344996 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:21.345618 26123 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.279912  1974 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.348656 26125 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:21.311367  2201 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:21.338827  2177 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:21.321977  2210 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.351774 26791 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:21.360052 26786 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:21.356760 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.335151  1472 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:21.319495  2074 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:27:21.368099 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.356273  2176 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:21.370121 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:21.358246  1882 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:21.353822  1993 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.367344 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.319479  2076 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.370992 26126 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:21.373903 26791 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:21.352451  2181 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.372510 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.363646  1974 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:21.379235 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.366182  2139 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:21.380995 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:21.377775 26123 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:21.383903 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.366854  1472 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.388422  2164 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:21.388959  2164 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:21.389298  1472 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.389844  2164 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.372160  2159 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:27:21.391701 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.392637 26783 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:21.392531 26788 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:21.385054  2197 raft_consensus.cc:2764] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:21.394044  2177 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.395839  2019 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:21.396553  2080 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.451807  1919 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.452844  2080 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.455261 26201 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:21.397156  2066 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:21.457494 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 1991) took 1987 ms. Trace:
I20260709 12:27:21.458493  1682 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:21.457993 26787 rpcz_store.cc:276] 0709 12:27:19.470386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.471476 (+  1090us) service_pool.cc:224] Handling call
0709 12:27:20.509243 (+1037767us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:27:20.832803 (+323560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.833042 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:20.833077 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:20.833135 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:20.833177 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.833258 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.457264 (+624006us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400715a50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:21.457415 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754303488}
I20260709 12:27:21.461782  2019 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.462370  2019 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.347574  2199 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.399680 26121 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:21.464686 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:21.465571 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:21.465493  1919 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:21.357046  2218 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:27:21.465505  2199 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:21.466843  2218 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:21.467195  2218 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:21.467978  2218 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.469372  2218 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:21.471094  1682 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.404273  2187 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:21.471626  1682 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.472205  1951 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:21.472675  1951 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:21.473511  2072 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:21.474045  2072 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.474411  2072 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:21.474677 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:21.403805  2108 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.405679  2077 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:21.405660 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.480216 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:21.480423 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74832us
I20260709 12:27:21.482081 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:21.484356  2187 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:21.484401  2197 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:21.485091  2187 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.485627  2187 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:21.486402 26125 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:21.487643 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:21.490226  2167 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:21.407356  2076 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.371971  1882 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:21.411274 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.496345 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.496748 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.497191 26051 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:21.411985  2210 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:21.378566  2195 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:27:21.417541 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.423645 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.425778  2213 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.429342  1952 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.437654 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:21.516331 26199 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:21.441113 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:21.517452 26194 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:21.442704 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.443573 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.443498 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.444581 26195 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:27:21.446576 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:21.522478 26200 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.446729 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2005) took 1251 ms. Trace:
I20260709 12:27:21.523520 26789 rpcz_store.cc:276] 0709 12:27:20.194867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.195017 (+   150us) service_pool.cc:224] Handling call
0709 12:27:20.195347 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.727886 (+532539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.446480 (+718594us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:21.446663 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018513664}
W20260709 12:27:21.524448 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 100910us
I20260709 12:27:21.447504 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.449676 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:21.473479  1951 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.475261 26126 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:21.380043  2181 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.477042 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.547806 26791 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.497570 26047 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:21.498313 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.499473 26053 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 22. Ops: []
I20260709 12:27:21.501130 26051 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:21.550164 26051 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:27:21.505519 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:21.551082 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45680.6us
I20260709 12:27:21.525260 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:21.526288 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 108868us
W20260709 12:27:21.528466 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78882.2us
I20260709 12:27:21.555153 26194 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:21.542338 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.543614 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.544523 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.556111 26052 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:21.556695 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.560379 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:21.494014  2199 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:21.543658 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1534) took 1036 ms. Trace:
W20260709 12:27:21.544775 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 97347.3us
I20260709 12:27:21.561477 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:21.597687 26198 raft_consensus.cc:2376] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24. Re-sending same reply.
I20260709 12:27:21.561477 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.595270 26048 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:21.597347 26127 rpcz_store.cc:276] 0709 12:27:20.507592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.530206 (+ 22614us) service_pool.cc:224] Handling call
0709 12:27:21.543461 (+1013255us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:21.543584 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606735104}
I20260709 12:27:21.596715 26054 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:21.598871 26201 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
W20260709 12:27:21.599567 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43956.5us
I20260709 12:27:21.601562 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.602159 26790 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:21.607882 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 49801.6us
W20260709 12:27:21.622831 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.655567 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:21.651530 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.656145 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.656378 26054 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:21.657917 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.658617 26054 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:21.660827 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.662549 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.523128  1941 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:21.654587 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.654225 26200 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:21.609293 26052 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:21.514714  2108 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:21.523239  1919 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.664775 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.685252 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1380) took 1144 ms. Trace:
I20260709 12:27:21.665068 26054 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:21.653883 26047 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:21.542568  1790 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.692025 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:21.655462 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.657917  2019 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.543396  2127 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.676023 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.676874 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:21.677184 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:21.717209 26783 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.662149  1718 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.558305  2190 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:21.558943  1952 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.689985 26787 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:21.685628 26203 rpcz_store.cc:276] 0709 12:27:20.540779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.558244 (+ 17465us) service_pool.cc:224] Handling call
0709 12:27:20.558711 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:20.584775 (+ 26064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:20.584813 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:20.616417 (+ 31604us) log.cc:844] Serialized 26 byte log entry
0709 12:27:21.104819 (+488402us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b180036b1a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:27:21.182343 (+ 77524us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:21.605633 (+423290us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.684809 (+ 79176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.684899 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:21.684947 (+    48us) raft_consensus.cc:1685] finished
0709 12:27:21.685022 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.685160 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45914,"prepare.queue_time_us":50552,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":51239,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6393,"spinlock_wait_cycles":7117696,"thread_start_us":453587,"threads_started":2,"wal-append.queue_time_us":627118}
I20260709 12:27:21.737689 26791 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:21.693414 26050 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:21.548995  2076 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:21.514546  2077 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.150s
I20260709 12:27:21.557624  2102 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:21.699200 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43800.8us
I20260709 12:27:21.772174 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.596318  2201 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:21.558033  2139 consensus_queue.cc:1175] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a981c9a7972c4e99a0903924c64fbb5c" responder_term: 24 status { last_received { term: 24 index: 17 } last_committed_idx: 16 last_received_current_leader { term: 24 index: 17 } }
I20260709 12:27:21.703094  2199 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:21.691704  1944 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:21.704030  1918 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.684823  2108 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.513973  2210 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.722388 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.722792 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.723660 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.724599 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.814046 26053 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:21.814731 26054 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:21.726959 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.759806  2127 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:21.545080  2213 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.696033  2074 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:21.755499 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.758627 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:21.699265  2097 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.666970  1760 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:27:21.723793  1905 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:21.777305 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:21.687698  1941 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.732656  2166 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:21.749019  2156 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:21.694612  2211 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 12:27:21.785861 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:21.847260 26047 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:21.799705 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
W20260709 12:27:21.775501  2220 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:21.749786  2076 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.698764  1919 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.758567  2022 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.683324 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
W20260709 12:27:21.813180 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 124704us
I20260709 12:27:21.843482  2154 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:21.827852 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.828159 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.862295 26054 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.724793  2019 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:27:21.834780 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:21.865129 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:21.836336 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59402.8us
I20260709 12:27:21.866305 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:21.838626 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.867250 26051 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.840449 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:21.868299 26048 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:21.828339  2132 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.813279  2201 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:21.847536 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61804.8us
I20260709 12:27:21.847973  1905 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:21.891685 26196 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:21.854001  2130 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:21.876209 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.843701  2097 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.857816  2076 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:21.887323  1472 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:21.844519  2166 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.919374 26051 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:21.866844  2154 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:21.846112  2074 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.954056 26199 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:21.921406 26046 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:21.943497 26048 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:21.956452 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:21.947760 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.910192 26054 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:21.921093  1993 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:21.901037  2132 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:21.957334 25947 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 17 to 20, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:21.959430  2074 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.959785  2166 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.932868  1472 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:21.965318  2228 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:27:21.966192  2232 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:21.871562  2022 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:21.987274 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:22.030470 26052 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:21.933650  2181 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:21.944913  2154 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.865329  2199 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.045882 26790 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:21.987733  2066 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.006938 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.013816 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:21.983428  2211 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:21.966957  2201 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.028911  2108 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.007350 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:22.075868  1963 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:22.025326  2167 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
W20260709 12:27:22.025056  2072 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.550s	user 0.002s	sys 0.009s
W20260709 12:27:22.079162 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 212949us
I20260709 12:27:22.080690 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:22.081126 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:22.107892 26048 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.081751 26053 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:22.024971 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:22.108783 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83945.4us
I20260709 12:27:22.083191  2181 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:22.115469 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:22.110894 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:22.086961  2130 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:22.123853 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:22.090670  2210 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.019307  1905 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:22.139820  2238 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:27:22.113147  1790 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.156162 26048 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
W20260709 12:27:22.146915 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.118916  2244 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.112282  1919 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.156433  1988 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:22.171576 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 164713us
I20260709 12:27:22.106690  2066 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.120198  2108 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:22.112835  1963 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.114382  2072 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:22.178529 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.190653 26126 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:22.194023 26126 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:22.173584 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.196549 26126 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 12
I20260709 12:27:22.197084 26126 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:27:22.197551 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.119206  2211 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.203626 26126 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.141642  2181 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.224561 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:22.226142 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.227103 26122 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:22.224794 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:22.228860 26121 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [20.12-21.13]
I20260709 12:27:22.230651 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.182672  2248 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:22.182394  2108 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.240622 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:22.200399  1993 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:22.186844  2097 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.166917  2210 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:27:22.217976  1919 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:22.265296 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.170042  2247 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
W20260709 12:27:22.256894  2239 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 12:27:22.238440  2166 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:22.172459  2244 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.208269  2167 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
W20260709 12:27:22.265148  1974 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.879s	user 0.011s	sys 0.000s
I20260709 12:27:22.259622  2108 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.183110  1988 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:22.266286  2235 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 12:27:22.260434  1993 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:22.295389  1993 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:22.273618  2132 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.292448 26198 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
W20260709 12:27:22.296442  2080 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.843s	user 0.010s	sys 0.010s
I20260709 12:27:22.293012 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.297909  1918 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:22.300513  2243 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:27:22.300587 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.301551 26121 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:22.301649  2258 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:22.303231  2181 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.304641 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.307085  1760 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:22.307260  1993 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.308565  1760 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:27:22.308678  2240 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:27:22.309006  1760 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:22.309834  1760 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:22.310492  2187 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.824s	user 0.012s	sys 0.000s
I20260709 12:27:22.310564  1993 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.311084  1760 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.311169  2187 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.311697  1963 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.282202  1919 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:22.314731  1919 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.315866  1472 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.317832 26127 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:22.318063  2072 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.318543 26784 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:22.318686  2022 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:22.319130 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.320469 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:22.321224  2237 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 12:27:22.320706  2241 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 12:27:22.324108 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:22.324579 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.267256  2130 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.326467 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:22.327315  2249 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:22.327754  2249 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:22.328315 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.324522 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.299894 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.273893  2259 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:22.330358 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.334852 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:22.328143  2249 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:22.333335 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.336946 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:22.343669 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.344035 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.327943  1951 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:22.345158 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:22.345629  2258 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:22.345744 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.346056  2251 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.357651  2076 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.372270  2076 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.328173  2211 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.286974  2166 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:22.375605  1952 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.003s
W20260709 12:27:22.388259 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 277441us
I20260709 12:27:22.390240 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:22.344599 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:22.347396 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 482487us
I20260709 12:27:22.351938 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:22.355373  2234 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.389s	user 0.003s	sys 0.000s
I20260709 12:27:22.358641 26052 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:22.360559 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:22.361310 26053 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:22.393774 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.394930 26122 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:22.392290 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.395915 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.397353 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:22.397605 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53122us
I20260709 12:27:22.398943 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.410951 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.414533 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.410557 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.416225 26052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:27:22.416854 26052 raft_consensus.cc:606] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.417169 26052 raft_consensus.cc:614] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:22.417315 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.417601 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.412917 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.420650 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:22.426158 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.421527 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:22.428112 26197 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:22.393450  2072 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:22.404707  2022 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:22.434290 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:22.399461  2097 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.442441 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:22.436278 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.426049 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.406683  2167 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.443626 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.417790  2247 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.449932 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1539) took 1269 ms. Trace:
I20260709 12:27:22.426558  2251 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.456995 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:22.399629  2210 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:27:22.454368 26120 rpcz_store.cc:276] 0709 12:27:21.180931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.181081 (+   150us) service_pool.cc:224] Handling call
0709 12:27:21.191659 (+ 10578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.247936 (+ 56277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.247974 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.474511 (+226537us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:27:21.474897 (+   386us) log.cc:844] Serialized 26 byte log entry
0709 12:27:21.741421 (+266524us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:21.741465 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.741555 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:22.354822 (+613267us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:22.449718 (+ 94896us) raft_consensus.cc:1685] finished
0709 12:27:22.449741 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.449878 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55857,"raft.queue_time_us":57846,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":20384,"spinlock_wait_cycles":12080256,"thread_start_us":266145,"threads_started":1,"wal-append.queue_time_us":433413}
I20260709 12:27:22.448197 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.419965  1905 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.430058  2019 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:22.430507  1918 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:22.426296  2198 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.423406  2250 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:22.425870  1993 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:22.460719 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:27:22.462383 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:22.464154 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:22.435796  2074 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:27:22.451790  2218 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.978s	user 0.009s	sys 0.005s
I20260709 12:27:22.466182 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:22.452687  2246 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:22.409943  2154 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.438145  2181 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:22.446771  2022 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:22.468413  1918 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:22.500607 26789 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:22.466755  2076 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:22.479352 26045 pending_rounds.cc:85] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 10
I20260709 12:27:22.498597 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:22.503412 26045 pending_rounds.cc:107] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:27:22.503835 26045 raft_consensus.cc:2889] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.504459 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:22.505339 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:22.472960  2187 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:22.506599 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.017s	sys 0.021s
I20260709 12:27:22.509485 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:22.544545 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.019s	sys 0.022s
W20260709 12:27:22.545084 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 558873us
I20260709 12:27:22.545998 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:22.546864 26052 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 20.
I20260709 12:27:22.490629  1993 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.480603  2019 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:22.550546 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:22.505203  1919 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.461395  2072 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 12:27:22.555737 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46386.4us
I20260709 12:27:22.555898 26052 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.496280  2198 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.473713  2249 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.510690  1918 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.495932  2132 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:22.499258  2022 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.505239  2244 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.495536  2250 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:22.580334  2257 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:27:22.559319  2248 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.559062  1993 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:22.576041 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 132219us
I20260709 12:27:22.596732 26783 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:22.605784 26048 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:27:22.609208  2268 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:27:22.565946  1988 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.609649  2202 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:22.609994  2019 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.610941  2250 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.575760  2246 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.612385  2224 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.615087  2213 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.445215  2258 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.572985  2076 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:22.653324  2076 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:22.653803  2076 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:22.576995  1919 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.505796  2181 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.658756  2181 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.655285  2167 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.661042 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:22.661674 26194 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.663792 26053 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:22.663978  2072 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.631181 26052 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:27:22.635510 26048 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:22.589540  2132 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.589006  2166 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.575560  1472 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:27:22.584611  1963 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.057s
W20260709 12:27:22.638511  2269 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:27:22.691450  2074 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:22.664475  2072 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.694348  2072 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.697189 26052 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:22.619500 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.698871 26201 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:22.675180 26053 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:22.585644  2198 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.676127 26194 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:22.640112  2202 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:22.700886 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2657) took 1203 ms. Trace:
I20260709 12:27:22.702296 26050 rpcz_store.cc:276] 0709 12:27:21.497138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.514540 (+ 17402us) service_pool.cc:224] Handling call
0709 12:27:21.544175 (+ 29635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.738263 (+194088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.700574 (+962311us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002e2554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:22.700791 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300926336}
I20260709 12:27:22.705740 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.707209 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.713267 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:22.678496  2132 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:22.744884  1963 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:22.745757  1472 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.642427  2246 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.749323  1905 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:22.750350  1919 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.650358  2224 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:22.761509  2272 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:22.763870  2154 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.679855  2167 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:22.765120  2270 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:27:22.701869  2166 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:27:22.768955 26194 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:22.778820 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:22.775560  1472 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.794739  1472 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:22.795032  2271 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:27:22.796906 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:22.770370  1963 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:22.798924  2224 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.799074  1963 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.800654  1963 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:22.802229  2246 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:22.806617 26123 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:22.740648  2267 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:27:22.809473  2267 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:22.809885  1905 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.810072  2267 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:22.810599  1905 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.813513 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1417) took 1210 ms. Trace:
I20260709 12:27:22.814152 26785 rpcz_store.cc:276] 0709 12:27:21.602901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.789172 (+186271us) service_pool.cc:224] Handling call
0709 12:27:21.789558 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.789942 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.789974 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.790556 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:27:22.373289 (+582733us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b1800306220. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:27:22.554562 (+181273us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:22.554605 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.554669 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:22.813076 (+258407us) raft_consensus.cc:1685] finished
0709 12:27:22.813107 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.813431 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":373,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"spinlock_wait_cycles":56875264,"thread_start_us":181084,"threads_started":1,"wal-append.queue_time_us":849838}
I20260709 12:27:22.820376  1941 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.829227  2218 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:22.830090  2280 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:22.831128 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2655) took 1334 ms. Trace:
I20260709 12:27:22.830129  2218 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.832609  2192 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.833153  2192 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.835165  2192 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:22.831504 26049 rpcz_store.cc:276] 0709 12:27:21.496589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.514371 (+ 17782us) service_pool.cc:224] Handling call
0709 12:27:21.548787 (+ 34416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.845210 (+296423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.845238 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:21.845276 (+    38us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:21.845302 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.830895 (+985593us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400172854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:22.830937 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.831070 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681236224}
I20260709 12:27:22.840072  2108 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:22.841696 26046 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:22.841866 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3965) took 1121 ms. Trace:
I20260709 12:27:22.842298 26047 rpcz_store.cc:276] 0709 12:27:21.720651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.723372 (+  2721us) service_pool.cc:224] Handling call
0709 12:27:22.841710 (+1118338us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:22.841811 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76195072}
I20260709 12:27:22.840718 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3069) took 1763 ms. Trace:
I20260709 12:27:22.843425 26195 rpcz_store.cc:276] 0709 12:27:21.077404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.078383 (+   979us) service_pool.cc:224] Handling call
0709 12:27:21.444288 (+365905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.528150 (+ 83862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.840524 (+1312374us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400217fd0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:22.840663 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":840329600}
I20260709 12:27:22.844794 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:22.846657  2260 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 12:27:22.847210  2108 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.847924  2251 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:22.844117 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 3971) took 1116 ms. Trace:
I20260709 12:27:22.848755 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:22.848781  2251 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.849086 26046 rpcz_store.cc:276] 0709 12:27:21.727879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.121782 (+393903us) service_pool.cc:224] Handling call
0709 12:27:22.122188 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.842222 (+720034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.843793 (+  1571us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54000f8a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:22.844041 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654103808}
I20260709 12:27:22.849711 26791 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:22.847780  2167 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.850988 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:22.851132  2251 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.819752  1993 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:22.828533  2190 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.079s	user 0.008s	sys 0.012s
I20260709 12:27:22.811437  2267 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.868096 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.852577  2218 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.870652  2164 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:27:22.873787  1905 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:22.869261  2022 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.875067  2282 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.864370  2192 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:22.883164 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:22.866861  2190 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:22.900017 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:22.903472 26054 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:22.860909  2202 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:22.904223 26201 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:22.897267  2192 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:23.038350 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.043318 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 143368us
I20260709 12:27:23.046657  2022 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:23.047188  2231 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.079s	user 0.003s	sys 0.003s
W20260709 12:27:23.048833  2288 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:27:23.049348  1993 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:23.053450  2285 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:27:23.055578 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.040657  1952 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.043370  2192 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:23.043309  2190 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.072782 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.076644 26202 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
W20260709 12:27:23.074204 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.082155 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.086076 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.044227  2076 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.094918 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:23.112660 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.113801 26202 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:23.091054  1718 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.127506  2292 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:23.131601 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1156) took 1138 ms. Trace:
I20260709 12:27:23.143776 26054 rpcz_store.cc:276] 0709 12:27:21.992817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.013548 (+ 20731us) service_pool.cc:224] Handling call
0709 12:27:23.131384 (+1117836us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:23.131528 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866759424}
I20260709 12:27:23.138592 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:23.142668 26790 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:23.145884 26202 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:23.150944 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:23.080919  2190 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.096942  1952 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:27:23.160980 26194 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:23.161126 26202 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:27:23.161777 26202 pending_rounds.cc:85] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 8
I20260709 12:27:23.162103 26202 pending_rounds.cc:107] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:27:23.162436 26202 raft_consensus.cc:2889] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.162798 26194 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:23.123658  2198 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:23.094825  1918 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.167744 26194 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:27:23.149278  2292 consensus_queue.cc:1243] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 26.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:23.145835 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:23.174418 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 271712us
I20260709 12:27:23.182308 26202 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 22. Ops: []
W20260709 12:27:23.183084 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.188925 26048 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:23.192695 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:23.190245 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.196076 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.196934 26051 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:23.197402 26202 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:23.198752 26202 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:27:23.200716 26783 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:23.204594 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1760) took 1961 ms. Trace:
I20260709 12:27:23.205070 26784 rpcz_store.cc:276] 0709 12:27:21.243155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.343524 (+100369us) service_pool.cc:224] Handling call
0709 12:27:21.343766 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.318923 (+975157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.204336 (+885413us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54004ba854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:23.204529 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2240969088}
I20260709 12:27:23.206967 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.207738 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.167563  2282 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:23.158216  2132 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.213828 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.214932 26049 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 20.
W20260709 12:27:23.219872  2290 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:27:23.193830  1941 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:23.220604  1941 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.221014  1941 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:23.210628  2282 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:23.224951  2282 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:23.233687 26202 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:23.238579 26195 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:23.212854  2292 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:23.231693  2192 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.271610 26202 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:23.263756  2076 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:23.263684  2199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.013s	sys 0.009s
I20260709 12:27:23.246836  2292 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:23.277731  2199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.013s	sys 0.009s
I20260709 12:27:23.276558 26788 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:23.295403  2300 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:23.295406  2076 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:23.296391  2300 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.296517  2076 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.299569  2300 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.300650  2076 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.259366  1882 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.322923  2293 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.007s
W20260709 12:27:23.354063 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.330360  1918 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.329593  2190 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.374452  2132 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:23.402388 26168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.953s (SENT) [suppressed 30 similar messages]
W20260709 12:27:23.403244 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 308457us
W20260709 12:27:23.404522 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.405166 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:23.408596 26121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:23.410168 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.410756 26126 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:23.411691 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.412189 26126 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.419589  1882 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:23.432586 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.433431 26120 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:23.441027  1941 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.442660 26054 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 28. Ops: []
I20260709 12:27:23.414369  2164 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:23.408291  2293 consensus_queue.cc:1243] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:23.453127 26126 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
W20260709 12:27:23.453480 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51136.3us
I20260709 12:27:23.456421 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.457460 26124 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:23.468726 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:23.469718 26784 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:23.453307  2302 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:27:23.473294 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:23.484643 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.486526 26120 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:23.478857 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:23.453935  2192 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.510469 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:23.518103 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.512102 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.519964  2254 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:23.480798  2218 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:23.530566 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 99445.7us
I20260709 12:27:23.508745  2213 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:23.520097  2304 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:23.516697  1941 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.523769  2295 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:23.527681  2202 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:23.534736 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:23.512475  1944 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:23.536684  2213 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:23.539819 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.559244 26124 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:23.465826  2293 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:27:23.545461 26784 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:23.471853  2282 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.539086 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.560638 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:23.567185 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.551800  1952 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:23.552995  2295 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.555682  2190 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:23.536427  2218 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.557850  1944 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.564178  2213 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:23.577975 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:23.580628 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.581390 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.581975 26124 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:23.582707  2190 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:23.583106  2282 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.583664  2190 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.585201  2295 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:23.586298  2301 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:23.574384  2310 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.587859  2310 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.588254  2218 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.590340  2310 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.649986 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.580574  1944 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:23.654278 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:23.648985 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 71131.8us
I20260709 12:27:23.596809  2292 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:23.617956  2266 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.242s	user 0.004s	sys 0.000s
I20260709 12:27:23.651525  2282 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:23.657538  2282 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:23.657970  2282 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.650308 26125 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:23.603012  2213 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.660756  2080 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:23.661342  2080 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.660808 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.662495 26195 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:23.642036 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1415) took 2225 ms. Trace:
I20260709 12:27:23.663825 26786 rpcz_store.cc:276] 0709 12:27:21.416745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.424348 (+  7603us) service_pool.cc:224] Handling call
0709 12:27:21.424692 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.425062 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.425092 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.425567 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:27:21.719121 (+293554us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:22.904442 (+1185321us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:23.641313 (+736871us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b1800342f40. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b77d9d3c7 00007f5b743af9a7 00007f5b743bf0bf
0709 12:27:23.641477 (+   164us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.641693 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.641772 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.641812 (+    40us) raft_consensus.cc:1685] finished
0709 12:27:23.641822 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.641947 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":127071,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1152,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":120121472,"thread_start_us":418904,"threads_started":2,"wal-append.queue_time_us":446045}
I20260709 12:27:23.665437  1944 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.568800  2167 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:23.669184  2167 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.669588  2167 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.662549  2066 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.624159  2187 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:23.671239  2187 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:23.575663  1952 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:23.672791  2213 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.672276  2187 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.673175  1952 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.656512 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.673799  2187 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:23.641505  2274 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:27:23.604605 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2730) took 2044 ms (client timeout 2214 ms). Trace:
I20260709 12:27:23.674260 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3087) took 1243 ms. Trace:
I20260709 12:27:23.710364 26201 rpcz_store.cc:276] 0709 12:27:22.430541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.712919 (+282378us) service_pool.cc:224] Handling call
0709 12:27:23.674022 (+961103us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:23.674180 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368822016}
W20260709 12:27:23.711790 26198 rpcz_store.cc:269] 0709 12:27:21.559936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.662183 (+102247us) service_pool.cc:224] Handling call
0709 12:27:23.604391 (+1942208us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:23.604527 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446668032}
I20260709 12:27:23.643376  2249 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.680924  1944 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:23.614444 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.715299 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.716830 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.681704 26125 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:23.718200 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.718587  1963 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.719092  1963 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:23.719388 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.720428 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2734) took 2121 ms. Trace:
I20260709 12:27:23.720870 26203 rpcz_store.cc:276] 0709 12:27:21.598553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.758992 (+160439us) service_pool.cc:224] Handling call
0709 12:27:21.759337 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.786530 (+ 27193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.786589 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.787172 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:27:22.084190 (+297018us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:22.111095 (+ 26905us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.111360 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.111461 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.716498 (+1605037us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b1800412960. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:23.720109 (+  3611us) raft_consensus.cc:1685] finished
0709 12:27:23.720228 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.720375 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23944,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":11423,"raft.queue_time_us":61999,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":16276,"spinlock_wait_cycles":18777856,"thread_start_us":230674,"threads_started":1,"wal-append.queue_time_us":343976}
I20260709 12:27:23.619259 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2735) took 2003 ms. Trace:
I20260709 12:27:23.612726 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.631003  1682 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.684844 26199 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:23.730623 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:23.744053 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:23.686025 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.686345  2198 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:23.678655  2218 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.687258  2132 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.684882 26124 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:23.724637 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.796198 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.797026 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.797338 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.797902 26124 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:23.798655 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.799619 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.689905  2022 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:23.691610 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.700973 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2016) took 1109 ms. Trace:
I20260709 12:27:23.820963 26790 rpcz_store.cc:276] 0709 12:27:22.591185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.592955 (+  1770us) service_pool.cc:224] Handling call
0709 12:27:22.593301 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.146216 (+552915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.700713 (+554497us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54003101d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:23.700904 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262733184}
I20260709 12:27:23.701926 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2015) took 1111 ms. Trace:
I20260709 12:27:23.822530 26788 rpcz_store.cc:276] 0709 12:27:22.590798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.592341 (+  1543us) service_pool.cc:224] Handling call
0709 12:27:22.592715 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.281079 (+688364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.701695 (+420616us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:23.701866 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572135296}
I20260709 12:27:23.702840 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.704807  1952 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.705440  2166 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:23.705768 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.820720 26195 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:23.827050 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.706669  2202 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:23.834944  2167 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.836813  2213 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.837337  2213 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:23.706971  2076 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:23.649518 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:23.681320 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.723277 26200 rpcz_store.cc:276] 0709 12:27:21.615786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.855791 (+240005us) service_pool.cc:224] Handling call
0709 12:27:21.856353 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.036046 (+179693us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:22.036077 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.036564 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:27:22.136093 (+ 99529us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:22.136149 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.618776 (+1482627us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400398854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:23.618839 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.618897 (+    58us) raft_consensus.cc:1685] finished
0709 12:27:23.618914 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.619189 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310,"raft.queue_time_us":111781,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":11611,"spinlock_wait_cycles":411430272,"thread_start_us":99169,"threads_started":1,"wal-append.queue_time_us":325366,"wal-append.run_cpu_time_us":3150,"wal-append.run_wall_time_us":1299104}
I20260709 12:27:23.785459 26201 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 28
W20260709 12:27:23.848194 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 166947us
I20260709 12:27:23.731535 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:23.786118 26197 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:23.849162 26198 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:23.744707 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.849671 26054 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:23.850392 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.850854 26054 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:27:23.851197 26054 pending_rounds.cc:85] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 12
I20260709 12:27:23.726241 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:23.782275 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.851727 26203 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:23.752777 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:23.852542 26196 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:23.726874  2080 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:23.855944 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 206500us
I20260709 12:27:23.856899 26054 pending_rounds.cc:107] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:27:23.857353 26054 raft_consensus.cc:2889] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.858942 26054 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:27:23.748724 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:23.859529 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:23.758953 26122 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:23.788841 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:23.860582 26046 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.791493 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.861495 26054 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:23.861708 26047 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:23.739424 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:23.863137 26202 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:23.792981 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:23.864147 26052 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:23.802601 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.805913 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:23.815860 26053 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
W20260709 12:27:23.823276 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 312910us
I20260709 12:27:23.828320 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:23.736249  2267 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:23.828878 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.833763 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.834455  2166 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:23.757805 26125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3118) took 1563 ms (client timeout 1564 ms). Trace:
I20260709 12:27:23.730309 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:23.853216 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.734083  2314 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:23.894375 26125 rpcz_store.cc:269] 0709 12:27:22.194731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.195079 (+   348us) service_pool.cc:224] Handling call
0709 12:27:23.757481 (+1562402us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001c4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:23.757668 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3269048192}
I20260709 12:27:23.737811  2164 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:23.735047 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:23.863209 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.795614  2198 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:23.886641 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.895407 26124 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:23.895453 26049 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:23.855487  2254 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.864557  1682 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:23.839385  2213 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:23.827909  2132 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:23.867532 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 65017.3us
W20260709 12:27:23.899516 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70744.1us
I20260709 12:27:23.984834 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:24.031302 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:23.899518 26052 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:27:23.986099 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 278341us
I20260709 12:27:23.838515  2181 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:23.920738 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:23.869565 26051 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:23.946246 26201 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:24.033053 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:23.956928 26127 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:23.969336 26046 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
W20260709 12:27:24.041780 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.922555 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:24.053231 26125 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:24.054266 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:24.054827 26125 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.893041  2300 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:24.080317 26048 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 18.
W20260709 12:27:24.075935 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 155316us
W20260709 12:27:23.985399  2127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.035s	user 0.002s	sys 0.015s
I20260709 12:27:24.076297 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:24.084914 26122 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:24.078773 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46255.2us
I20260709 12:27:24.076471 26127 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:24.082123  1944 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:24.081079 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 158595us
I20260709 12:27:23.918418  2190 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:24.088550 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:24.020081  2076 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.122750 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:24.112093 26051 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:23.892866  2311 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.217s
I20260709 12:27:23.820739  2218 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.129357 26125 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:24.158891 26125 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:23.839586 25947 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 8 to 21, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 21 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:23.922981  2139 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:24.007254  2192 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:23.841113  2022 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.147267 26124 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:27:24.081624 26120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.022s
I20260709 12:27:24.034003  2249 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:24.084506  2127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.139s	user 0.003s	sys 0.016s
I20260709 12:27:23.898332  2308 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.190s
I20260709 12:27:24.080677  1472 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:23.927865  2159 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.195s
W20260709 12:27:24.161954 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74430.9us
I20260709 12:27:24.201201 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:24.006498  2282 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:24.168913 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46235.4us
I20260709 12:27:24.169173 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:24.163862 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 451442us
W20260709 12:27:24.173300 26120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.000s	sys 0.023s
I20260709 12:27:24.154654  2108 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.168154 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:24.214321 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:23.922812  2314 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:24.193413 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:24.148204  2300 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:24.221405 26785 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:24.215013 26788 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:24.157102  2076 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.177729  1944 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.169636  2210 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:24.168373  2166 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:24.217325 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:24.204531 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:24.230386 26125 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:24.224532 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 171001us
I20260709 12:27:24.208801  2318 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:24.209550  2213 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:24.200064  1472 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:24.217274  2305 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:24.236169 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:24.240432 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 325154us
I20260709 12:27:24.184609  2139 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.259369 26196 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
W20260709 12:27:24.261066 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 175427us
I20260709 12:27:24.253377 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:24.253080 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 470031us
I20260709 12:27:24.226015  2254 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:24.226579  2210 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:24.264151 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:24.237999  2181 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:24.224156  2322 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:24.204717  2066 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:27:24.250823  1472 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:24.275705  2210 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:24.231917  1944 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:24.364944  2315 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:27:24.365782  2317 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:27:24.209553  2164 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.396425  2300 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.399274  1963 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:24.401400  2076 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:24.402863  2282 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:24.404251  2218 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:24.406090  2139 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.406575  2321 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:24.281987 26093 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:24.224366  2108 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.268784  2254 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:24.278075  2224 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:24.431759  2314 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:24.432478  2324 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:24.433127  2198 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.436020  1882 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:24.458567 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 176665us
I20260709 12:27:24.464936 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3133) took 1056 ms. Trace:
W20260709 12:27:24.458680 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 252541us
I20260709 12:27:24.440172  2326 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.476665  2326 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:24.476379 26123 rpcz_store.cc:276] 0709 12:27:23.408654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.408764 (+   110us) service_pool.cc:224] Handling call
0709 12:27:23.444216 (+ 35452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.554408 (+110192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.554440 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.555144 (+   704us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.005870 (+450726us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:24.005925 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.006071 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.464636 (+458565us) raft_consensus.cc:1685] finished
0709 12:27:24.464672 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.464862 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85921,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":10481,"thread_start_us":490336,"threads_started":2,"wal-append.queue_time_us":845009}
I20260709 12:27:24.446050  2333 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.440408  2132 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.458122  2224 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.463732  2139 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:24.484797 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:24.473671  1882 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:24.609398 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1549) took 1134 ms. Trace:
W20260709 12:27:24.613567 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 371149us
I20260709 12:27:24.482441  2322 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:24.482151  2080 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:24.574672  2076 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:24.619514 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 388623us
I20260709 12:27:24.613415 26126 rpcz_store.cc:276] 0709 12:27:23.474763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.474959 (+   196us) service_pool.cc:224] Handling call
0709 12:27:23.475436 (+   477us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:23.479635 (+  4199us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:23.479665 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:23.509491 (+ 29826us) log.cc:844] Serialized 52 byte log entry
0709 12:27:23.963795 (+454304us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:23.963857 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.965446 (+  1589us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.609128 (+643682us) raft_consensus.cc:1685] finished
0709 12:27:24.609152 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.609321 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3729,"raft.queue_time_us":72448,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":153,"thread_start_us":416365,"threads_started":1,"wal-append.queue_time_us":638602}
I20260709 12:27:24.622579 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.483563  2332 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.623646 26198 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:27:24.612376 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 127681us
I20260709 12:27:24.624197 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:24.470978  2324 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:24.574584  2224 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:24.634004 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.574136  2218 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:24.603821  2311 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.641891 26203 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:24.457737  2321 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.571060  2333 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.608320  2314 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:24.627815  2326 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:24.631080  2022 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.647768 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.638378 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.686558 26200 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
W20260709 12:27:24.635861 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 380037us
W20260709 12:27:24.687798 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 423732us
I20260709 12:27:24.693147 26201 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:24.641750  2080 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:24.643563  2198 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.643466  2076 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:24.704311 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.725584 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:24.726866 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.705518 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:24.728022 26199 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:24.671558  1918 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:24.705956 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.713188  2066 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:24.716740  2308 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:24.636463 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:24.684257 26203 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:24.719904 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:24.689179 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:24.732988 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:24.693346  2282 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.840643  2282 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.845757  2076 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.851295  2321 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.854836  2333 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:24.855697  2333 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:24.766750  2308 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:24.872980  2308 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:24.752209 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63144.6us
I20260709 12:27:24.758378  2167 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:24.759473  2267 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.842s	user 0.005s	sys 0.005s
I20260709 12:27:24.693974  2314 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:24.634229  2322 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.675778  2218 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.762944 26199 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:24.763211  2324 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:24.729802  2198 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.726783  2336 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
I20260709 12:27:24.767951 26194 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:27:24.676556  2311 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.646083  2332 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.796595  2305 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:24.642735  2254 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:24.806443  2108 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.807613  1944 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:24.700608  2202 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:24.700292  1882 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.810318  2080 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:24.745247  2187 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.025s	user 0.012s	sys 0.000s
I20260709 12:27:24.811745 25941 maintenance_manager.cc:419] P 5eede3841e1242afbeab30840148b558: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033362
I20260709 12:27:24.681373  2325 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.178s
I20260709 12:27:24.979625  2254 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:24.895228 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 162334us
I20260709 12:27:24.903211  1718 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:24.920965 26127 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 12:27:25.040614 26127 pending_rounds.cc:85] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 16
I20260709 12:27:25.040974 26127 pending_rounds.cc:107] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:27:25.041271 26127 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:24.915220 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 3992) took 1184 ms. Trace:
W20260709 12:27:24.917496 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.035s	sys 0.054s
W20260709 12:27:25.000988 26755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.006s	sys 0.004s
I20260709 12:27:25.009639 26128 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:24.874358  2308 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:24.900311  1472 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:25.021812 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 301988us
W20260709 12:27:24.962414  2199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.035s	sys 0.096s
W20260709 12:27:24.993525  2318 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.002s
I20260709 12:27:24.935310  2305 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:25.001292  2097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.006s	sys 0.005s
I20260709 12:27:24.978940  2139 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:25.005159  2177 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:25.043064  2341 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:25.028462  2339 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:25.042070  1682 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.097548  2097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.006s	sys 0.006s
I20260709 12:27:25.047315  2314 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:25.028067  1944 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.929905  2333 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.112s
W20260709 12:27:25.000736  2190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.001s	sys 0.010s
I20260709 12:27:24.979372  2202 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:25.067087 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.035s	sys 0.054s
I20260709 12:27:25.066906 26048 rpcz_store.cc:276] 0709 12:27:23.730569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.734767 (+  4198us) service_pool.cc:224] Handling call
0709 12:27:24.914941 (+1180174us) spinlock_profiling.cc:243] Waited 50.5 ms on lock 0x7b54002e2554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:24.915131 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116051712}
W20260709 12:27:25.145751 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 878259us
W20260709 12:27:25.124437  1952 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.271s	user 0.015s	sys 0.000s
I20260709 12:27:25.158635  2311 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:25.159857  2342 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:25.114017  1682 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.147547  1944 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.163244  1944 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.164202  2218 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:25.106897  2341 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.168331  2341 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.104045  2177 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:25.170100 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 291) took 26136 ms (client timeout 15000 ms). Trace:
I20260709 12:27:25.170809  2177 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:25.171279 25947 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 25, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 25 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.172724  2202 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:25.125770  2190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.001s	sys 0.011s
I20260709 12:27:25.173218 25947 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 23, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 23 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.176489 26124 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 12:27:25.177021 26124 pending_rounds.cc:85] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 16
I20260709 12:27:25.177287 26124 pending_rounds.cc:107] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:27:25.177569 26124 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:25.067237 26755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.007s	sys 0.004s
I20260709 12:27:25.052145  1718 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.186347  1963 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.191201  2198 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:25.081784  2199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.035s	sys 0.096s
I20260709 12:27:25.045991  2332 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.192058  2343 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:25.192795 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1557) took 1690 ms. Trace:
I20260709 12:27:25.192698  2343 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.193213 26128 rpcz_store.cc:276] 0709 12:27:23.502328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.827378 (+325050us) service_pool.cc:224] Handling call
0709 12:27:23.827698 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.078879 (+1251181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.192551 (+113672us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003d1654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:25.192734 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2716984576}
I20260709 12:27:25.195029  2343 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.114367  2139 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.197577  2139 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.199808  1682 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.201910 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1565) took 1117 ms. Trace:
I20260709 12:27:25.202461  2305 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.196024  1882 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:25.205808  2340 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:25.206283  2340 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:25.207173  2340 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.173637  2339 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:27:25.106194 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3143) took 1173 ms. Trace:
I20260709 12:27:25.186883 25947 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 25. New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:25.254923 26127 rpcz_store.cc:276] 0709 12:27:23.932782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.086640 (+153858us) service_pool.cc:224] Handling call
0709 12:27:24.087035 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.041842 (+954807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.103944 (+ 62102us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:25.106117 (+  2173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916514048}
I20260709 12:27:25.097213  2318 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:24.890854 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:25.117301  2282 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.190899  1963 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.113817  2213 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.073s
W20260709 12:27:25.251235 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.202325 26124 rpcz_store.cc:276] 0709 12:27:24.084714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.175513 (+ 90799us) service_pool.cc:224] Handling call
0709 12:27:24.175908 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.178086 (+1002178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.201647 (+ 23561us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54001f90d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:25.201846 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299939200}
W20260709 12:27:25.108773 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.264729 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.023s	sys 0.041s
W20260709 12:27:25.257553 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103199us
I20260709 12:27:25.125156  2308 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:25.259553  2300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.021s	sys 0.024s
I20260709 12:27:25.267153 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:25.189543  1718 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:25.256083 25947 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:25.268399 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.023s	sys 0.042s
I20260709 12:27:25.257228  1944 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.258237  2340 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:25.266611  2132 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.020s	sys 0.025s
I20260709 12:27:25.282377 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:25.269307  2300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.021s	sys 0.024s
I20260709 12:27:25.285637 26046 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:25.273403 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:25.286859 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:25.293143 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:25.294159 26053 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:25.265997  2322 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:25.289621 26051 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:25.280939  2249 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.013s	sys 0.038s
W20260709 12:27:25.295066 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 404283us
W20260709 12:27:25.296408 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.003s	sys 0.074s
I20260709 12:27:25.297688 26046 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:25.299381 26050 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:25.288585  2132 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.020s	sys 0.025s
I20260709 12:27:25.280830  2341 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.079603 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:25.290836 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.021s	sys 0.054s
W20260709 12:27:25.336393 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.021s	sys 0.054s
W20260709 12:27:25.337042 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 644769us
I20260709 12:27:25.287949  1918 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:25.341763 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:25.341981 26126 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:25.341907 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:25.343021 26124 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:25.346195 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.347467 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:25.305069 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:25.313902 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:25.348663 26051 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:25.353073 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.353996 26046 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:25.355212 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:25.312602 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.301157 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:25.314692 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.003s	sys 0.074s
W20260709 12:27:25.357741 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 725593us
I20260709 12:27:25.313680 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:25.282246  2313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.055s	sys 0.090s
I20260709 12:27:25.361986 26045 raft_consensus.cc:2410] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:25.302286 26047 pending_rounds.cc:85] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 12
I20260709 12:27:25.287423  2318 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.274396  2332 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:25.332490  2202 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:25.393786 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:25.314397  2249 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.013s	sys 0.038s
I20260709 12:27:25.274454  2198 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:27:25.304358  2323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.007s	sys 0.041s
W20260709 12:27:25.361197 26200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.040s	sys 0.029s
W20260709 12:27:25.361361 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47553.1us
I20260709 12:27:25.362692 26047 pending_rounds.cc:107] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 26.13
W20260709 12:27:25.417047 26200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.040s	sys 0.029s
I20260709 12:27:25.417387 26047 raft_consensus.cc:2889] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.357091 26785 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:25.368491 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.419430 26046 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:25.336409 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 256876us
W20260709 12:27:25.383530 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.387001 26051 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:25.421903 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.422777 26051 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:25.423454 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:25.339948  2305 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.313048  2321 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:25.306368 25947 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 19 to 20, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:25.395964 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.003s	sys 0.014s
I20260709 12:27:25.647696 25947 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 17 to 24, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 24 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:25.649317 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.004s	sys 0.014s
I20260709 12:27:25.335603  2300 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.400909 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.407023 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
W20260709 12:27:25.362380  2210 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.004s
I20260709 12:27:25.410423 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:25.410784 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:25.362224  2313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.056s	sys 0.091s
I20260709 12:27:25.675078 26790 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:25.676105 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:25.287551  1944 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.387734 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:25.677982 26788 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:25.433583 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.433923 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:25.437747 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:25.680126 26199 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:27:25.438499 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.685642 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.687222 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.439308 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.688421 26198 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:25.440557 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:25.464437 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 67079.4us
W20260709 12:27:25.466094 26167 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.019s	sys 0.018s
W20260709 12:27:25.690246 26167 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.020s	sys 0.018s
I20260709 12:27:25.336978  2327 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:27:25.568260 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:25.631970  2076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.012s
W20260709 12:27:25.521222 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.398262  2322 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:25.400964  2202 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:25.416949  2323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.007s	sys 0.042s
I20260709 12:27:25.530948 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:25.170517 25944 rpcz_store.cc:269] 0709 12:26:59.034266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.569043 (+2534777us) service_pool.cc:224] Handling call
0709 12:27:25.169854 (+23600811us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:16.238867 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:16.240256 (+  1389us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:16.240317 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:27:16.240371 (+    54us) tablet.cc:662] Decoding operations
0709 12:27:16.263724 (+ 23353us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:16.263838 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:16.263870 (+    32us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:27:16.266712 (+  2842us) tablet.cc:701] Row locks acquired
0709 12:27:16.266785 (+    73us) write_op.cc:260] PREPARE: finished
0709 12:27:16.267053 (+   268us) write_op.cc:270] Start()
0709 12:27:16.267284 (+   231us) write_op.cc:276] Timestamp: P: 1783600036266952 usec, L: 0
0709 12:27:16.267350 (+    66us) op_driver.cc:348] REPLICATION: starting
0709 12:27:16.326730 (+ 59380us) log.cc:844] Serialized 7593 byte log entry
0709 12:27:17.550728 (+1223998us) op_driver.cc:464] REPLICATION: finished
0709 12:27:17.918274 (+367546us) write_op.cc:301] APPLY: starting
0709 12:27:17.918377 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:18.587787 (+669410us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:18.587923 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:18.605524 (+ 17601us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b18004536a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:19.054408 (+448884us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:19.054852 (+   444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:27:19.054919 (+    67us) write_op.cc:312] APPLY: finished
0709 12:27:19.055438 (+   519us) log.cc:844] Serialized 172 byte log entry
0709 12:27:19.248848 (+193410us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:19.249048 (+   200us) write_op.cc:454] Released schema lock
0709 12:27:19.249127 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:19.249589 (+   462us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:19.249631 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1518962,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":361187,"apply.run_cpu_time_us":61017,"apply.run_wall_time_us":1331448,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":221200,"prepare.run_cpu_time_us":33308,"prepare.run_wall_time_us":268485,"raft.queue_time_us":322359,"raft.run_cpu_time_us":18289,"raft.run_wall_time_us":257724,"replication_time_us":1261537,"spinlock_wait_cycles":10955520,"thread_start_us":658793,"threads_started":5,"wal-append.queue_time_us":989293,"wal-append.run_cpu_time_us":37387,"wal-append.run_wall_time_us":2494937}]]}
I20260709 12:27:25.535270 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:25.414966  2022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.005s	sys 0.011s
I20260709 12:27:25.548007 26046 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:27:25.533995 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 496815us
I20260709 12:27:25.588361 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:25.413087  2164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.001s	sys 0.015s
I20260709 12:27:25.412050  2218 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:25.593954 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173692us
I20260709 12:27:25.340876  2341 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.602572 26791 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:25.397935  2213 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.174s
I20260709 12:27:25.400741  2333 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.187s
I20260709 12:27:25.387282  1918 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.400588  2318 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:25.754307  2080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.032s	sys 0.044s
I20260709 12:27:25.408351  2334 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.192s
I20260709 12:27:25.568208  2347 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.619578 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 285872us
I20260709 12:27:25.398193  2346 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.208s
I20260709 12:27:25.411517  2190 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.190s
I20260709 12:27:25.400749  2329 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.204s
I20260709 12:27:25.629575  1963 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:25.630441  2350 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:27:25.461899  2167 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.011s	sys 0.006s
W20260709 12:27:25.814345  2167 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.011s	sys 0.006s
I20260709 12:27:25.386778  2339 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:25.772341  2325 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:25.640851 26093 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:25.656925 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.000s	sys 0.004s
I20260709 12:27:25.668563 26054 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:25.671070 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 114313us
W20260709 12:27:25.690536 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 814859us
I20260709 12:27:25.703449 26199 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:25.706483 26202 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:25.569610 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:25.707360 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:25.712296 26197 raft_consensus.cc:2393] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:25.650494 25947 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 22, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 22 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.716295 26194 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:25.720731 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:25.730453  2076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.010s	sys 0.012s
I20260709 12:27:25.748454 26785 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:25.755686 26787 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:25.766695 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.713291  2210 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:25.753650  2022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.013s	sys 0.015s
I20260709 12:27:25.746622  2202 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:25.536391  2097 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:25.747345  1472 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:25.802891  2080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.032s	sys 0.044s
W20260709 12:27:25.761267  2254 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.000s
I20260709 12:27:25.778452 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:25.747664  2345 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:25.706573  2300 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.805969 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:25.761642  2218 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.816922 26052 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:27:25.639701 26196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.004s	sys 0.007s
I20260709 12:27:25.777262  2127 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.103s
I20260709 12:27:25.879384 26788 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:25.820093  2339 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:25.889704 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.883525 25947 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 25. New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:25.900200  2337 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:25.901491 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:25.914753 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:25.932181 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 153802us
I20260709 12:27:25.953096 25947 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 24, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 24 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:25.954746 25947 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:25.934635 26196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.004s	sys 0.008s
W20260709 12:27:25.880798 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.001s	sys 0.014s
I20260709 12:27:25.925083 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.877782  2323 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:27:25.961580 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.759320  2164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.001s	sys 0.016s
I20260709 12:27:25.819921  2348 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.816684  2355 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:25.956595 25947 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 23, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 23 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:25.924592 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:25.939509  2218 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:25.952373 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:25.986318 25947 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.819599  2108 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.142s
I20260709 12:27:25.932646  2345 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:25.955518  1963 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.954034  2339 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:25.954147  2350 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:25.940419  2177 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:25.820487  2347 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.990373  2305 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:25.998864  2224 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.319s	user 0.010s	sys 0.004s
I20260709 12:27:25.963239 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:25.953747  1472 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.994853 25947 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:26.015831 25947 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:27:25.951843 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 145981us
I20260709 12:27:25.953303  2337 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:25.903802  2097 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:25.974803  2323 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:25.979146  1882 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:26.004489 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80124.7us
I20260709 12:27:25.980979  2199 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:25.950243  2080 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:26.006731  2313 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:26.048399 26198 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:26.056746 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 255183us
I20260709 12:27:26.060820 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:26.017689 25947 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 24. New cstate: current_term: 24 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.826377 26050 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:26.093243 25947 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 19 to 20, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.095371 25947 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.013607  2305 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:26.104753 26203 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:26.098901 26045 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:26.102645 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:25.987499  2355 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:25.986196  2348 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.016177  2350 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:25.806620  2343 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.110086 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.079041 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:26.004736  1963 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.112587 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:26.114429 26788 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:26.114641 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.115307 26785 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:27:26.097689  2363 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.368s	user 0.000s	sys 0.002s
I20260709 12:27:26.059779  2377 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.125397 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:26.078425  2313 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:25.789650  2321 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.316s
I20260709 12:27:25.790055  2198 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.316s
W20260709 12:27:26.119982  2361 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 12:27:26.015079  1472 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.011487  2177 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:26.127528 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48574.8us
W20260709 12:27:26.110621  2357 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.364s	user 0.000s	sys 0.001s
I20260709 12:27:26.131690 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:26.064993  1882 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.128985  2332 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:26.122869 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 482083us
I20260709 12:27:26.097813 25947 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 15 to 23, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 23 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:26.141765 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.161087 26788 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:26.160775 25947 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 21 to 23, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 23 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.142094 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:26.161780 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.144677  2358 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.354s	user 0.000s	sys 0.003s
I20260709 12:27:26.145414 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.162997 26790 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:26.145684  2338 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:26.018136  2345 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.137367  1944 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.144723 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.109303  2305 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:26.028576  2224 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.127228  2254 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.093592  2080 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.124598  2387 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:26.131194  2127 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.166581 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:26.145938  2327 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:26.163373 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:26.137792  1682 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.103263  2325 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.146529 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.133441  2384 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:26.277668 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2759) took 1578 ms. Trace:
I20260709 12:27:26.280139 26200 rpcz_store.cc:276] 0709 12:27:24.699062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.699224 (+   162us) service_pool.cc:224] Handling call
0709 12:27:24.699521 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.701983 (+  2462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.702026 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.702616 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.417561 (+714945us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:25.417601 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.417672 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.144104 (+726432us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:26.237699 (+ 93595us) raft_consensus.cc:1685] finished
0709 12:27:26.237736 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.277583 (+ 39847us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2026,"raft.queue_time_us":60282,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"spinlock_wait_cycles":10918144,"thread_start_us":690619,"threads_started":1,"wal-append.queue_time_us":1029012}
I20260709 12:27:26.119832  2337 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.183002 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:26.188719 26198 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:26.191744 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2758) took 1496 ms. Trace:
I20260709 12:27:26.154452  2341 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.198597 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:26.204387 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55790.6us
I20260709 12:27:26.126897  2343 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.164155  2159 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:26.165206  2338 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:26.311474  2375 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 12:27:26.311939  2338 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.312260  2159 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.313437  2338 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:26.314775  2159 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.315379  2159 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.315871  2159 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:26.316866  2374 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:27:26.318537  2159 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.319195  2159 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:26.319589  2159 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.319957  2159 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:26.319979  2202 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.322582  2202 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:26.323810  2308 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.051s	user 0.010s	sys 0.001s
I20260709 12:27:26.170682  2318 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.175164  1918 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.154512  2324 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:27:26.348922  2387 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:26.311496  2389 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:26.350329  2389 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.169351  2364 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:26.352470  2318 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.352559  2364 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.353193  2318 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.354043  2325 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.295766  2353 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:26.355086  2364 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.355319  2318 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.185448  2385 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.200901  2345 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:26.131587  2323 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.220000  2254 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.199759  1944 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.098s
I20260709 12:27:26.327310  2337 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:26.212700  2313 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.133340  2377 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.270712  2327 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:26.389801  2327 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:26.391499  2377 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.200196  2359 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.097s
W20260709 12:27:26.396975  2249 consensus_queue.cc:1175] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "24e61c63097e4d3dbb149050e6fcc997" responder_term: 24 status { last_received { term: 24 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 24 index: 16 } }
W20260709 12:27:26.398343  2322 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.011s
I20260709 12:27:26.398801  2322 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:26.398820  2327 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.228487  2350 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.401387  2327 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.328722  2159 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.329669  2224 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:26.330324  2376 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 12:27:26.330840  2366 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 12:27:26.237287  2022 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:26.219612  2300 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.343539  2355 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.343829  2177 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:26.344357  2370 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:27:26.344966  2210 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.345301  2097 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:26.344022  2340 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.054s	user 0.011s	sys 0.000s
I20260709 12:27:26.346359  2348 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.403383  2323 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:26.334846  2367 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:27:26.171286  2332 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.236398  2127 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:26.341444  2368 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
I20260709 12:27:26.354818  2353 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:26.356117  2385 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.385334  2343 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.431136  2353 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.434662  2385 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.437201  2254 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.387434  2305 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.444172  2305 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.389001  2313 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.446722  2353 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.448969  2353 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.358919  2308 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:26.274727  2392 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.109s
I20260709 12:27:26.411361  2389 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.417759  2218 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.419106  2377 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:26.420974  2332 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:26.427100  2360 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
I20260709 12:27:26.355577  2347 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.451594  2377 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:26.456946  2384 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:26.457087  2377 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.458847  2377 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:26.232357 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:26.503065 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1163) took 1221 ms. Trace:
W20260709 12:27:26.503333 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 270937us
W20260709 12:27:26.274894 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 111623us
I20260709 12:27:26.515009 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:26.237767 26127 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:26.517359  2389 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:26.518020 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2747) took 1902 ms. Trace:
I20260709 12:27:26.518174  2389 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.518998  2389 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:26.519656  2385 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:26.520702  2341 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:26.304330 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:26.228214 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 96640.5us
I20260709 12:27:26.293414 26201 rpcz_store.cc:276] 0709 12:27:24.695142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.697250 (+  2108us) service_pool.cc:224] Handling call
0709 12:27:24.697700 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.702829 (+  5129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.702865 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.737702 (+ 34837us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.424702 (+687000us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:25.424740 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.424812 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.191480 (+766668us) raft_consensus.cc:1685] finished
0709 12:27:26.191502 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.191662 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3594,"raft.queue_time_us":61744,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":495225,"threads_started":1,"wal-append.queue_time_us":713755}
I20260709 12:27:26.175619 26203 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:27:26.310772 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 125188us
I20260709 12:27:26.549590  2308 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:26.328387 26786 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:26.345108  1472 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.282670  1682 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.346354  1882 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.564919 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2674) took 1263 ms. Trace:
I20260709 12:27:26.565508 26047 rpcz_store.cc:276] 0709 12:27:25.301401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.301560 (+   159us) service_pool.cc:224] Handling call
0709 12:27:25.302020 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:25.559181 (+257161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.559329 (+   148us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:25.561161 (+  1832us) log.cc:844] Serialized 52 byte log entry
0709 12:27:25.658303 (+ 97142us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:25.955270 (+296967us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.979753 (+ 24483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.979840 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.554223 (+574383us) raft_consensus.cc:1685] finished
0709 12:27:26.564319 (+ 10096us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.564673 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34065,"prepare.queue_time_us":79252,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4430,"raft.queue_time_us":125420,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":50712,"thread_start_us":326813,"threads_started":2,"wal-append.queue_time_us":387425}
W20260709 12:27:26.569357 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 265034us
I20260709 12:27:26.347280  1963 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.388821  2080 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.423621  2127 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.419875  2159 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.445003  2199 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:26.576278  2199 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:26.565150  2308 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:26.509274 26092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.024s
I20260709 12:27:26.509581 26121 raft_consensus.cc:606] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:26.510025 26784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.008s	sys 0.011s
W20260709 12:27:26.580480 26092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.009s	sys 0.024s
W20260709 12:27:26.580711 26784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.008s	sys 0.011s
W20260709 12:27:26.511135  2336 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.009s	sys 0.039s
W20260709 12:27:26.529245 25940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.003s	sys 0.010s
W20260709 12:27:26.581488  2336 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.039s
W20260709 12:27:26.540824  1718 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.260s	user 0.004s	sys 0.009s
I20260709 12:27:26.584167 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.584715  1718 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.458863  2384 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.587639  1718 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:26.541424  2167 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.034s	sys 0.055s
I20260709 12:27:26.535778  2218 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:26.533303 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.518724 26196 rpcz_store.cc:276] 0709 12:27:24.614902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.628752 (+ 13850us) service_pool.cc:224] Handling call
0709 12:27:24.629223 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.067721 (+438498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.067751 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:25.094048 (+ 26297us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.966242 (+872194us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:25.966295 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.966381 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.517156 (+550775us) raft_consensus.cc:1685] finished
0709 12:27:26.517215 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.517604 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68812,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":26151,"thread_start_us":1269560,"threads_started":2,"wal-append.queue_time_us":906745}
I20260709 12:27:26.555642 26790 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27.
I20260709 12:27:26.552841 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2680) took 1214 ms. Trace:
I20260709 12:27:26.503789 26048 rpcz_store.cc:276] 0709 12:27:25.281568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.281752 (+   184us) service_pool.cc:224] Handling call
0709 12:27:25.282374 (+   622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.284567 (+  2193us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:25.284617 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:25.304125 (+ 19508us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.656370 (+352245us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:25.656433 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.656526 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.502016 (+845490us) raft_consensus.cc:1685] finished
0709 12:27:26.502128 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.502659 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1771,"raft.queue_time_us":5031,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":23393,"thread_start_us":267906,"threads_started":1,"wal-append.queue_time_us":760345}
I20260709 12:27:26.413666  2339 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.589658 26051 rpcz_store.cc:276] 0709 12:27:25.332565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.423881 (+ 91316us) service_pool.cc:224] Handling call
0709 12:27:25.424345 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.551659 (+127314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.551691 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:25.552182 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.763698 (+211516us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:25.763925 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.764380 (+   455us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.546292 (+781912us) raft_consensus.cc:1685] finished
0709 12:27:26.546347 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.546759 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126501,"raft.queue_time_us":133225,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":25154,"thread_start_us":150268,"threads_started":1,"wal-append.queue_time_us":454397}
I20260709 12:27:26.593237  2339 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.524303  2385 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.584002 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.594285  2385 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:26.595501  2022 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:26.597201  1718 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.597671 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.598268  1718 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.599182  2022 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:26.588475  2167 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.035s	sys 0.055s
I20260709 12:27:26.601994 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.603029 26126 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
W20260709 12:27:26.573196 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.581594 25940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.004s	sys 0.010s
I20260709 12:27:26.589007  2332 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:26.614426 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.588446  2324 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:26.624624 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.625238  2332 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.626418 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.627162 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1780) took 1209 ms. Trace:
I20260709 12:27:26.628779 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.629206 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:26.623759 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 70234.4us
I20260709 12:27:26.636785 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.639408 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.640744 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:26.633247 26784 rpcz_store.cc:276] 0709 12:27:25.417695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.420440 (+  2745us) service_pool.cc:224] Handling call
0709 12:27:25.420970 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.421412 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:25.932011 (+510599us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.626364 (+694353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:26.626503 (+   139us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:26.626651 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.626785 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.627058 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47575,"prepare.queue_time_us":349727,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":46996,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":182,"thread_start_us":966065,"threads_started":2,"wal-append.queue_time_us":527053}
I20260709 12:27:26.644310  2336 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.645140  2336 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.632750  2324 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.661298 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2046) took 1265 ms. Trace:
I20260709 12:27:26.661077  2199 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.661926 26786 rpcz_store.cc:276] 0709 12:27:25.395474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.395625 (+   151us) service_pool.cc:224] Handling call
0709 12:27:25.395920 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.551439 (+1155519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.660700 (+109261us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b540056aed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:26.661093 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2143875968}
I20260709 12:27:26.656308  1682 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.661383  2324 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:26.638013 25940 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.684s	user 0.005s	sys 0.010s
I20260709 12:27:26.668093  2199 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:26.670171 26168 connection.cc:256] RPC call timeout handler was delayed by -2.05342476099986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:27:26.661387 26125 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:26.658890 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.675010  2392 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.676005  2327 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.675899  2392 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.676779  2327 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.677460  2407 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:26.678267  2407 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.678666  2392 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.648064  2336 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.689791  2413 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:26.690495 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.691285  2413 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:26.691484 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.691735  2413 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:26.693521 26198 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:26.694859  1718 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.695114 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:26.696503 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.696630 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.697216 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.697889 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.698961 26128 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:26.700253 26052 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:26.701051 26195 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:26.701915 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58316 (request call id 43) took 3293 ms. Trace:
I20260709 12:27:26.696652 26197 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:26.697943 26194 raft_consensus.cc:2376] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21. Re-sending same reply.
I20260709 12:27:26.705803 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.702426 26121 rpcz_store.cc:276] 0709 12:27:23.408298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.408442 (+   144us) service_pool.cc:224] Handling call
0709 12:27:26.701615 (+3293173us) spinlock_profiling.cc:243] Waited 3.1 s on lock 0x7b54001d24d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439f853 00007f5b7936d60a 00007f5b728f37d4 00007f5b728f3762 00007f5b728f3691 00007f5b728f360c 00007f5b728f28c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:26.701760 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7129615744}
I20260709 12:27:26.709187 26126 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:26.709461  2332 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:26.710418 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.712828 26202 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:26.688134 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.716493 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.716938 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.720053 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:26.720839 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.721618 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:26.722183 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:26.722997 26053 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:26.724742  2132 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:26.726759 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:26.728372 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.728562 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.729274 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2677) took 1425 ms. Trace:
I20260709 12:27:26.729724 26045 rpcz_store.cc:276] 0709 12:27:25.303510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.400633 (+ 97123us) service_pool.cc:224] Handling call
0709 12:27:26.728902 (+1328269us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400228b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:26.729066 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990956928}
I20260709 12:27:26.731132 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.731962 26045 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:27:26.727713  2132 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:26.732908  2132 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:26.683900  2339 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:26.738530  2339 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.739261 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1567) took 1398 ms. Trace:
I20260709 12:27:26.741097  2339 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.722256 26051 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:26.741874 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.740089 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:26.745190  2407 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:26.739730 26127 rpcz_store.cc:276] 0709 12:27:25.340429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.347154 (+  6725us) service_pool.cc:224] Handling call
0709 12:27:26.738918 (+1391764us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:26.739071 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045005184}
I20260709 12:27:26.729269 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.746825 26788 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:26.746658  2385 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.748397 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.748736  2327 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.727622 26791 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:26.749727 26052 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 12:27:26.749791 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:26.751331  2413 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.753536  2413 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:26.756732 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.758338 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.759842  2389 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:26.762112 26050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2678) took 1450 ms (client timeout 1825 ms). Trace:
I20260709 12:27:26.762580  2389 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.745651 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:26.739117 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:26.762673 26050 rpcz_store.cc:269] 0709 12:27:25.311433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.345852 (+ 34419us) service_pool.cc:224] Handling call
0709 12:27:26.761670 (+1415818us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:26.761888 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102276864}
I20260709 12:27:26.785066 26788 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:26.803555 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69621.9us
I20260709 12:27:26.812780 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:26.815697 26786 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:26.820447 26790 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25. Re-sending same reply.
I20260709 12:27:26.821746 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:26.821882 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:26.822525 26790 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
W20260709 12:27:26.825518 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.813660 26202 raft_consensus.cc:2376] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21. Re-sending same reply.
I20260709 12:27:26.834115 26045 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:27:26.802796  1718 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.836920 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:26.842367 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:26.845665 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:26.852648  2419 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 12:27:26.840662  1682 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.855175 26790 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:26.855741  1682 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.856704 26783 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:26.844659 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:26.861577 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.870765 26197 raft_consensus.cc:2368] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24.
I20260709 12:27:26.877470  1718 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:26.880805  2347 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:26.881745 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47915.6us
I20260709 12:27:26.881549  2347 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.884542  2347 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.888458  2385 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:26.892833 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:26.870042 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 549154us
I20260709 12:27:26.905987 26045 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:26.907078 26053 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:26.880076  1718 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.872718  1682 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:26.932184 26788 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:26.872418  2022 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:26.935315 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.855216  2324 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:26.933660 26786 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:26.934717 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:26.942059 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.943998 26121 raft_consensus.cc:2368] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:26.941385 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.954581 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.962330 26051 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:26.969127 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:26.973205 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:26.973114 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 151430us
I20260709 12:27:26.941080  2347 raft_consensus.cc:2764] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:27:26.973109 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.974393 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:26.980289 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.982044 26127 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:27:26.974735 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.986130 26125 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:26.990882 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:26.992762 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:26.994403 26197 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:26.994808 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:26.984921 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.003059 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.005512 26203 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:27.005985 26126 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:27.007113 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:27.009490 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.014321 26120 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:27.019302  2337 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.020457 26053 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:27.020334  2337 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:27.021539 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.022213 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:27.022881  2337 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:27.023154 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:27.026372 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.037880 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:27.040033  2022 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:27.042065 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136501us
W20260709 12:27:27.048465 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48970.4us
I20260709 12:27:27.053941 26123 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:27:27.053798 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.060137 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:27.067094 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71655us
I20260709 12:27:27.059494 26051 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:27.104609 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.114050 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.121474  2327 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:27.122041  2327 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:27.122437  2327 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.126356 26053 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
W20260709 12:27:27.134464 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54147.2us
W20260709 12:27:27.135290 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 145526us
I20260709 12:27:27.141451 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.145905 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.150285 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.152375 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.152978 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.153568 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.149828 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.166800 26053 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.168543 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.168901 26019 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:27.169831  2385 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:27.170696  2385 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:27.152097 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:27.178926 26093 leader_election.cc:400] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27.
I20260709 12:27:27.179379 26093 leader_election.cc:403] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:27:27.148527  2324 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:27.179754 26791 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:27.192714  2181 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:27.193109 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:27.194831 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:27.202509 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56070.6us
W20260709 12:27:27.209619 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72607.4us
W20260709 12:27:27.234535 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.240815 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 115954us
I20260709 12:27:27.246337 26045 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:27.257630  2401 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:27.258507  2324 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.259123  2324 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:27.261675  2324 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:27.264644 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.264801 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.265632 26054 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:27.265761  2159 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:27.257994 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.273944 26054 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:27.274724 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:27.273476 26790 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:27.280840 26049 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:27.268393 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:27.238211  2199 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:27.290750 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.291640 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.196413  2347 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:27:27.301076 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.310452 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.243465  2022 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:27.318292 26128 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:27.323015  2198 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:27.323766  2198 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:27.325604  2430 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:27:27.326671 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:27.327164  2426 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:27:27.325023  2198 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:27.329177 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 134405us
I20260709 12:27:27.294373  2308 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:27.327435 26046 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:27.299952  2199 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.344862 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:27.339382 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 91702.1us
W20260709 12:27:27.355530 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81806us
I20260709 12:27:27.316139  2022 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.357016 26054 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
W20260709 12:27:27.338423  2132 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.001s
I20260709 12:27:27.248382  2218 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.098s
I20260709 12:27:27.364830 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.345839  2308 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.374629 26169 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.350437  2199 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.343642  2385 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:27.374711 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.309233  2339 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
W20260709 12:27:27.410247 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45286.2us
W20260709 12:27:27.410336 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 163021us
W20260709 12:27:27.408948 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51059.9us
W20260709 12:27:27.409788 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51801.6us
I20260709 12:27:27.379184  2308 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:27.454269 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.455325  2407 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:27.455775  2385 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:27.456748  2412 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:27.459033 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:27.460139 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.461508 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:27.464805  2387 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:27.364815  2022 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:27.387342 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.375757  2132 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:27.468632  2324 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:27.469776 25940 maintenance_manager.cc:643] P 5eede3841e1242afbeab30840148b558: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 2.426s	user 0.049s	sys 0.011s Metrics: {"bytes_written":4934,"delete_count":0,"lbm_write_time_us":1007,"lbm_writes_lt_1ms":5,"reinsert_count":0,"thread_start_us":631446,"threads_started":1,"update_count":190,"wal-append.queue_time_us":548148}
I20260709 12:27:27.470851  2339 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:27.412262  2413 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:27.499737 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45664us
W20260709 12:27:27.472213  2431 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:27:27.504771 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:27.509420 26046 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:27.503808 26127 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:27.477053  2407 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:27.503007  2429 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:27.560192 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:27.529371  2324 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.530179 26128 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:27.550340 26120 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:27.557346 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:27.529048 26093 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:27.565637 26122 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:27.563218 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.545256  2413 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.556586  2407 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.572312 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.576459 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.578868  2338 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:27.579360  2338 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:27.579798  2338 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.582562  2401 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:27.583382  2401 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:27.584380 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 111497us
I20260709 12:27:27.584571  2401 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:27.587357 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.567060  2324 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:27.594223 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 69251us
I20260709 12:27:27.615149 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.573403  2413 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:27.616978 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69725.8us
W20260709 12:27:27.620671 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.621906 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.560951  2339 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:27.626780 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.625999 26054 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:27.625952 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:27.624821  1472 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:27.630230  1472 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:27.628554 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:27.638487 26123 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:27:27.640347 26127 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:27.631134  1472 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:27.631182  2347 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:27:27.653239 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:27.656924 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 120276us
W20260709 12:27:27.658383 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 208457us
I20260709 12:27:27.643448  2218 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:27.650245  2401 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:27.660806 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:27.669370 26019 leader_election.cc:400] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24.
I20260709 12:27:27.669844 26019 leader_election.cc:403] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:27.652947  1682 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:27.659689 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 285042us
I20260709 12:27:27.681270 26122 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:27.680757 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:27.660583  2218 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.699921 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:27.702118 25941 maintenance_manager.cc:419] P 5eede3841e1242afbeab30840148b558: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.213806
I20260709 12:27:27.702607  2313 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24.
I20260709 12:27:27.689710 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:27.688913  2218 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:27.717989 26129 raft_consensus.cc:606] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:27.718554 26129 raft_consensus.cc:614] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:27.718575  2389 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:27.706135  2132 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:27.728803 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:27.732627 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.720005  2347 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:27.733868 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:27.728768  2339 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:27.732790  2108 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:27.789177  2181 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:27.789753  2181 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:27.791007  2181 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:27.792481  2443 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:27:27.793624  2377 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.794231  2377 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:27.795576  2181 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:27.828267 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:27.789180  2445 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:27:27.852921 26196 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.786769  2347 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:27.861511  2377 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:27.862939 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.863183 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.864310 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:27.865948  2412 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:27.866261  2159 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.001s
W20260709 12:27:27.870893 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 193079us
W20260709 12:27:27.874972 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 173292us
W20260709 12:27:27.875782 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 215829us
I20260709 12:27:27.875434 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.875588 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:27.876933 26202 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
W20260709 12:27:27.881639 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 200968us
I20260709 12:27:27.856721  2108 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:27.882283 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 231428us
I20260709 12:27:27.932787 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:27.975714 26201 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:27.907784 26196 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:27.908043 26198 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:27.935415 26199 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:27.878774  2159 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27.
W20260709 12:27:27.908116 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.977255  2327 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.851s	user 0.013s	sys 0.003s
I20260709 12:27:27.898103  2389 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:27.948594  2339 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.873502  2412 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:28.013682  2199 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.010s
I20260709 12:27:27.980126  2108 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:27.979650  2308 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.036656  2450 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:28.037350  2159 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:28.037474  2439 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:28.029289  2199 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:28.040501  2439 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:28.040974  2453 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:28.007591  2327 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:28.052937 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 67764us
I20260709 12:27:28.053022 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4016) took 1335 ms. Trace:
I20260709 12:27:28.048884  2199 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:28.054982 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.054543 26048 rpcz_store.cc:276] 0709 12:27:26.717458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.743030 (+ 25572us) service_pool.cc:224] Handling call
0709 12:27:26.743556 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.792709 (+ 49153us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:27.314010 (+521301us) log.cc:844] Serialized 14 byte log entry
0709 12:27:27.778850 (+464840us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:27.874468 (+ 95618us) log.cc:844] Serialized 26 byte log entry
0709 12:27:27.874708 (+   240us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:27.874762 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.874864 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:28.052777 (+177913us) raft_consensus.cc:1685] finished
0709 12:27:28.052798 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.052946 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66475,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":42168,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":39102,"thread_start_us":837059,"threads_started":2,"wal-append.queue_time_us":508907}
I20260709 12:27:28.054559  2439 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:28.036773  2412 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:28.053707  2339 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:27.982784  2347 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:28.050603 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 481729us
I20260709 12:27:28.076645 26127 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:28.057802  2452 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:28.067830  2387 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.016s	sys 0.016s
I20260709 12:27:28.110260  2451 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:28.062781  2159 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:28.066821  1682 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:28.132879 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.127679 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49633.1us
I20260709 12:27:28.136564 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.096380  2452 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:28.104703  2387 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.016s	sys 0.016s
I20260709 12:27:28.052156  2338 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:28.133973  2324 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.002s
I20260709 12:27:28.061689  2450 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.149439  2387 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:28.057055  2389 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.149173  2452 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:28.174842 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:28.176721 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 279088us
W20260709 12:27:28.178237 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 389780us
I20260709 12:27:28.161725  2451 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:27:28.180845 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 217332us
W20260709 12:27:28.171303  2407 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.003s
W20260709 12:27:28.183360 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 298154us
W20260709 12:27:28.191339 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136439us
I20260709 12:27:28.184734  2313 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.196599 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.193552 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.199654 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.158368  2324 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.206406 26124 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:28.194034  2413 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.006s
I20260709 12:27:28.225113  2337 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.225903  2337 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:28.228072 26018 leader_election.cc:336] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.680s (SENT)
I20260709 12:27:28.228717 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.230425  2385 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:28.234165  2337 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:28.236827 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.239253 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.240190 26123 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:27:28.234870  2337 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.243176 26196 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:27:28.196491  2108 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.200428  2339 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:28.256443 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81708.1us
I20260709 12:27:28.258201 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.259982 26120 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:28.189828  2407 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:28.261283 26126 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:28.195924  2313 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.262423 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:28.297485 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.243822  2455 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:28.322926 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64819.6us
I20260709 12:27:28.309724 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:28.320313 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101832us
I20260709 12:27:28.331310 26120 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
I20260709 12:27:28.331604 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:28.332316 26123 raft_consensus.cc:606] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Received request to transfer leadership to TS 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:28.332700 26123 raft_consensus.cc:614] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:28.308974  2218 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.005s
I20260709 12:27:28.334996 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.283325  2327 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:28.340979 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 112327us
I20260709 12:27:28.240942  2413 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.346916 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:28.349642 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:28.351590 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.297984  2339 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:28.353062 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 100864us
W20260709 12:27:28.357350 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115151us
I20260709 12:27:28.358258 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.321758  2455 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.361667 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.350869 26048 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:28.351579 26050 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:28.235798  2159 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.235601  1682 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.279676  2337 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.391515 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:28.395076 26049 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:28.401858 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.374495  2455 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.266031  2387 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:28.359532  2439 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:27:28.352403  2218 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.380216  2339 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:28.409706  2308 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.441469 26020 leader_election.cc:400] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23.
I20260709 12:27:28.406385  2450 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.405555  2487 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:28.454649  2457 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.242s	user 0.001s	sys 0.001s
W20260709 12:27:28.444672 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 86540.5us
W20260709 12:27:28.463292  2484 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:27:28.464156  2473 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:28.465232  2485 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.131s
I20260709 12:27:28.465628  2337 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:28.469468 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:28.488497 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92989.7us
I20260709 12:27:28.454722 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:28.498921 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.499454 26787 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:28.504664 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 116534us
I20260709 12:27:28.486452  2337 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:28.525457  2462 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
I20260709 12:27:28.510932  2337 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.510977  2313 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.526333  2407 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:28.534535 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 295754us
W20260709 12:27:28.544786 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 343956us
I20260709 12:27:28.540170  2413 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.550004  2501 raft_consensus.cc:2764] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:28.543777 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.576664 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:28.577531 26198 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
W20260709 12:27:28.532145 26125 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.006s
I20260709 12:27:28.583706 26125 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
W20260709 12:27:28.586673 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 117762us
I20260709 12:27:28.587571 26169 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.577842 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.590041 26197 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:28.588822 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 203094us
W20260709 12:27:28.552862  2479 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:27:28.595708 26049 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:27:28.546833  2401 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.869s	user 0.009s	sys 0.005s
I20260709 12:27:28.561085  2456 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:28.537243  2337 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.567713  2451 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:28.557026  2218 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:28.561884  2492 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.621752 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:28.556727  2324 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.560125  2338 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.626971 26196 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:27:28.596202  2487 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:28.628957 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:28.632119 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88437.1us
W20260709 12:27:28.633708 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.620858  2407 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:28.650175 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.646067 26789 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:27:28.630976  2401 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:28.665668 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.654388 26198 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 12:27:28.630806  2487 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:28.668711 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.636262  2339 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:28.676661 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:28.677318 26790 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.677804 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.678596 26126 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:28.677178 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.683856 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.684927 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.685482 26197 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 19: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:28.689833 26121 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:28.673485  2487 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.630542  2492 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.654433  2452 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.681438  2451 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.683254  2338 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:28.705029 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54906.6us
I20260709 12:27:28.705470  2480 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:28.705753  2471 raft_consensus.cc:993] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: : Instructing follower 24e61c63097e4d3dbb149050e6fcc997 to start an election
W20260709 12:27:28.708659 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46863.3us
W20260709 12:27:28.709868 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.721458 26787 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:28.681113  2407 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:28.725203  2022 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.183s	user 0.012s	sys 0.004s
I20260709 12:27:28.728298 26198 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:28.729369 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 130444us
I20260709 12:27:28.733831 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.734004 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.733835 26128 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:27:28.735137 26128 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:27:28.736554 26790 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:28.739282 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:28.713265  2472 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 12:27:28.740782 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:28.741410  2492 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:28.735369 26128 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:27:28.741765 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.742090 26128 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.742250  2313 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:28.742645  2313 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.742978  2313 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.743196  2452 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.743382 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.741765 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.733331 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.746122 26122 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:28.743824  2407 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.747929  2412 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:28.755276  2481 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:28.748093  2022 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.722630  2338 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:28.756486  2338 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.757697  2022 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:28.762449 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:28.764485  2492 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.764592  2451 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:28.764988  2324 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:28.768489  2480 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:28.768982  2480 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:28.769404  2480 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.769134  2455 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:28.770226 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.771459  2455 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:28.771582 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.223s (SENT) [suppressed 19 similar messages]
I20260709 12:27:28.772378 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.772733 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.773345 26122 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:28.774235 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.774937 26122 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:28.775758 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.740176  2377 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:28.775767 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:28.776415 26199 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:28.777156 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.777451  2499 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:28.777794 26195 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:28.777969  2499 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.778426  2499 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.780582 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.784965 26203 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:28.755340 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:28.791153 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.791539 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.791810 26049 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.801766 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.729943  2475 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:28.735369  2473 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:28.709890  2401 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.043s
I20260709 12:27:28.807621 26049 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:28.745110  2467 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.809037 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:28.807905 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:28.816846  2498 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:28.817456  2498 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:28.818620  2498 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.824503 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.825728 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:28.826001 26202 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:28.841220 26049 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:28.804131  2181 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:28.852862 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:28.850230 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108557us
I20260709 12:27:28.858669 26048 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:28.910588 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:28.912187 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49878.2us
I20260709 12:27:28.915831 26052 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:28.921799 26202 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:28.923007 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.925401 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:28.926576 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.806129  2377 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:27:28.856243  2475 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:28.925364 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 122206us
I20260709 12:27:28.937837  2489 raft_consensus.cc:1081] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Signalling peer 24e61c63097e4d3dbb149050e6fcc997 to start an election
I20260709 12:27:28.922569  2181 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:28.939314 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.940107 26201 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:28.941063 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:28.942440 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:28.858407  2401 consensus_queue.cc:1243] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 23.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:27:28.950361 26126 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:28.951346 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:28.954128 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:28.955296 26788 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:28.946761 26049 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:28.966327 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:28.968703 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:28.972656 26126 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.978018 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:28.977048 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:28.981436 26791 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:28.960850 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:28.813033  2473 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:28.977051 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:29.068198 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.069236 26788 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:28.937930  2377 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:29.067461 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:28.990829 26784 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [23.16-25.17]
I20260709 12:27:29.068430 26783 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.066846 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:29.071885 26123 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:29.072424 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.075508 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.076313 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:29.088690 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128691us
I20260709 12:27:28.924121  2467 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.071352  2377 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:29.101989 26790 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:29.103674 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.078493  2470 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:28.987699  2412 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.109s
I20260709 12:27:29.108690 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.085505 26791 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
 from {username='slave'} at 127.0.0.1:38286
I20260709 12:27:29.113197 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:29.118243 26784 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:29.116413 26197 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:29.119205 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.128892 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.131685 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.132370 26126 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:29.140275  2470 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:29.142788  2470 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:29.147171 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:29.152129 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.166467 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:29.130752  2512 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 12:27:29.171650 26094 connection.cc:256] RPC call timeout handler was delayed by -2.132112664000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
I20260709 12:27:29.125033  2455 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:29.175449 26791 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:29.176010 26791 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:29.204694  2525 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:27:29.206933 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:29.114836  2489 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.058s
I20260709 12:27:29.216619 26122 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:29.181402  1682 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.177937  2455 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:29.231671 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.233865 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 137894us
I20260709 12:27:29.235123 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:29.232534 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 138531us
I20260709 12:27:29.205762  2339 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:29.248654 26094 connection.cc:256] RPC call timeout handler was delayed by -1.8196720099999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2923s for a response
W20260709 12:27:29.249326 26094 connection.cc:256] RPC call timeout handler was delayed by -1.632809310000539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2737s for a response
W20260709 12:27:29.249814 26094 connection.cc:256] RPC call timeout handler was delayed by -1.417963181000232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:27:29.250231 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3873631810001825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
W20260709 12:27:29.250648 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3792631810001694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:27:29.256062 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3090631810000559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:27:29.273871 26094 connection.cc:256] RPC call timeout handler was delayed by -1.0912631810001585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
I20260709 12:27:29.225826  2451 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:29.127563  2475 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.254390 26791 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.282876 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.212798  2324 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:27:29.237054  2455 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:29.210507  2452 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:27:29.291422  2530 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 12:27:29.292548  2458 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:29.292992  2497 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:27:29.293484  2515 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
W20260709 12:27:29.294193  2519 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:27:29.294126  2347 raft_consensus.cc:2764] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:27:29.288012 26784 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:29.294932  2451 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:29.295112  2520 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
W20260709 12:27:29.298878 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92037.6us
W20260709 12:27:29.303453 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.310474 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1580) took 2068 ms. Trace:
I20260709 12:27:29.329118 26125 rpcz_store.cc:276] 0709 12:27:27.241994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.242131 (+   137us) service_pool.cc:224] Handling call
0709 12:27:29.310300 (+2068169us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:29.310403 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682680704}
I20260709 12:27:29.244892  1682 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.299913  2473 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:29.338918 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:29.340018 26125 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:29.342893 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.343954 26784 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:29.327956  2313 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.585s	user 0.004s	sys 0.008s
W20260709 12:27:29.333977  2523 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:27:29.354700 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.306722  2470 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 12:27:29.357693 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 122694us
I20260709 12:27:29.378072 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.358644 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:29.379261 26789 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:29.380192 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.381490 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:29.327872  2458 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:29.358959 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:29.361582 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:29.359423  2528 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 12:27:29.359907  2518 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:27:29.358700 26053 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:27:29.360220  2503 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.348s	user 0.001s	sys 0.001s
I20260709 12:27:29.360479 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.361109  2467 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:29.361574 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:29.333961  2451 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:29.363915  2526 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:27:29.364540 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:29.364926  2533 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:29.320870  2497 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:29.376020 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.385094 26782 raft_consensus.cc:2393] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:29.390043 26123 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:27:29.399597 26202 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:29.392750  2541 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:29.393462  2540 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:29.400452 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.407407  2497 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:29.394127  2338 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.637s	user 0.005s	sys 0.006s
I20260709 12:27:29.397136  2324 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:29.423799 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70099.9us
I20260709 12:27:29.422237 26123 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:29.373108  2313 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:29.421196 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 455682us
I20260709 12:27:29.430234 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.419137  2470 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:29.438923 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.431440 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.421720  2338 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.431905  2513 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.459741 26783 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:29.462152 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:29.466115 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 87737.8us
W20260709 12:27:29.441591  2487 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.006s
W20260709 12:27:29.464454 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104094us
I20260709 12:27:29.465020 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.435849  2452 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:29.436313  2470 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:29.429947  2324 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.470908 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.478031 26051 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:29.463654  2218 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:29.421593  2540 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:29.476056  2480 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.000s
I20260709 12:27:29.474514  2470 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:29.474202  2485 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:29.496456 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:29.475028  2487 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:29.503888 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:29.505429 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.486857  2546 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.501457  2480 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.493394  2218 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:29.510573 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 45622.8us
I20260709 12:27:29.511282 26198 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:29.519402 26785 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:29.486152  2452 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:29.509684  2337 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.878s	user 0.002s	sys 0.011s
I20260709 12:27:29.501008  2485 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:29.503131  2541 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:29.537684 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46165.4us
I20260709 12:27:29.511252  2532 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:29.520690  2218 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:29.519443  2456 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:29.473071  2513 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.526422  2540 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:29.521931  2501 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:29.578626 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87113.9us
I20260709 12:27:29.520803  2553 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:29.549809  1682 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:29.584775 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.487675  2458 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:29.543638  2492 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.774s	user 0.007s	sys 0.005s
I20260709 12:27:29.536703  2467 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:29.539551  2485 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.585736  2313 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:29.521627  2516 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:27:29.591378 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3130) took 1022 ms. Trace:
W20260709 12:27:29.589170 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 82350.9us
I20260709 12:27:29.591277  2544 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:29.520174  2546 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:29.596629 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 129751us
I20260709 12:27:29.808107 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:29.586289  2339 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:29.620784  2467 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.640652  2548 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:29.591776  2532 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:29.614240  1682 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.115s
I20260709 12:27:29.539733  2337 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.586915  2501 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:29.628397 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43689.8us
W20260709 12:27:29.800730 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.593089  2540 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:27:29.582113  2456 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:29.611956  2492 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:29.804107 26196 rpcz_store.cc:276] 0709 12:27:28.567521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.576374 (+  8853us) service_pool.cc:224] Handling call
0709 12:27:29.589912 (+1013538us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:29.590117 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102160128}
I20260709 12:27:29.802703  2460 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:29.822027  2548 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:29.822407  2554 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:29.841387 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:29.822515  2552 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:29.822925  1682 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:29.837613  2339 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.828399  2501 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:29.843585 26790 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
W20260709 12:27:29.821695  2543 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.356s	user 0.011s	sys 0.000s
W20260709 12:27:29.821296  2499 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.037s	user 0.021s	sys 0.000s
I20260709 12:27:29.833459  2480 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:29.863013 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55054.5us
I20260709 12:27:29.834578  2456 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:29.866247 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60260.6us
I20260709 12:27:29.858741  2558 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:29.872298 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.873947 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.843446  2132 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:29.870639  2313 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:29.843297  2548 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.879642 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:29.880488 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:29.886785  2337 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:29.877919  2558 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:29.892515  2549 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:27:29.893294  2558 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 13, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.894757  2558 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:29.896341  2482 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.005s	sys 0.027s
I20260709 12:27:29.879940 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:29.896752  2482 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.248s	user 0.005s	sys 0.027s
I20260709 12:27:29.897043 26782 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:29.828646  2544 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.897713 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.887688  2132 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.869592  2339 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.900946 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.902415 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.903028 26196 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:29.903393 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4125) took 1134 ms. Trace:
I20260709 12:27:29.903788 26125 rpcz_store.cc:276] 0709 12:27:28.769311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.349261 (+579950us) service_pool.cc:224] Handling call
0709 12:27:29.385651 (+ 36390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.411092 (+ 25441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.411127 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.411666 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.534352 (+122686us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:29.534423 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.534490 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.903170 (+368680us) raft_consensus.cc:1685] finished
0709 12:27:29.903192 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.903333 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5180,"raft.queue_time_us":7081,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":67481,"threads_started":1,"wal-append.queue_time_us":151230}
I20260709 12:27:29.909483 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:29.913767 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:29.916216 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:29.917359 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:29.917753 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:29.918015 26196 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:29.921643 26053 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 21: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:29.924206 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:29.930020 26756 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.520s (SENT)
I20260709 12:27:29.930954 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:29.932173  2545 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.343s	user 0.000s	sys 0.010s
W20260709 12:27:29.934770  2455 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.007s
I20260709 12:27:29.935194  2455 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.936939  2544 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:29.939343  2470 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:29.875142  2499 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:29.941155  2560 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:29.913349  2553 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.913153  2313 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:29.941329 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:29.922642  2562 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:29.944916  2513 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:29.851318  2460 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:29.958957 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85063.4us
I20260709 12:27:29.962787  2563 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:29.914708  2452 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:29.909608  2557 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.920832  2532 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:29.991452 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:29.985736  2324 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:29.991644  2313 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:29.992189  2508 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.366s	user 0.000s	sys 0.004s
I20260709 12:27:29.994065 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:29.994998  2485 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.998028 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3141) took 1225 ms. Trace:
I20260709 12:27:29.998414  2218 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:29.985193  2544 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:30.001127 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.009779 26789 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:30.010411 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1420) took 2146 ms. Trace:
W20260709 12:27:30.014381  2529 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.801s	user 0.004s	sys 0.004s
I20260709 12:27:30.015893 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1438) took 1097 ms. Trace:
I20260709 12:27:30.017309 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.018040 26782 raft_consensus.cc:2368] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:30.020711 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3131) took 1452 ms. Trace:
W20260709 12:27:30.021258 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.021615 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4048) took 1167 ms. Trace:
I20260709 12:27:29.949236  2338 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.027179 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:30.062093 26127 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:30.034840 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.067826 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.039551 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.055405 26787 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:30.053933 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:30.058322 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:30.073086 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.038501 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2783) took 1194 ms. Trace:
I20260709 12:27:30.059734 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.056499 26203 rpcz_store.cc:276] 0709 12:27:27.863845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.863978 (+   133us) service_pool.cc:224] Handling call
0709 12:27:30.010114 (+2146136us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b5400379454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:30.010286 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2101341568}
I20260709 12:27:30.058598 26785 rpcz_store.cc:276] 0709 12:27:28.918121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.918234 (+   113us) service_pool.cc:224] Handling call
0709 12:27:30.015622 (+1097388us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54004501d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:30.015793 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893488000}
W20260709 12:27:30.072379 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 162963us
I20260709 12:27:30.027551 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:30.055720 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 54698.9us
I20260709 12:27:30.075585 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:30.060494 26051 rpcz_store.cc:276] 0709 12:27:28.854392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.854555 (+   163us) service_pool.cc:224] Handling call
0709 12:27:28.863274 (+  8719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.478580 (+615306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.021377 (+542797us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:30.021553 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413174144}
I20260709 12:27:30.076542 26169 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.076542 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.036996 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:30.078852 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:30.060300 26199 rpcz_store.cc:276] 0709 12:27:28.567858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.570343 (+  2485us) service_pool.cc:224] Handling call
0709 12:27:28.576830 (+  6487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:28.776956 (+200126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.020444 (+1243488us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:30.020643 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458560768}
I20260709 12:27:30.074033 26197 rpcz_store.cc:276] 0709 12:27:28.844292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.866189 (+ 21897us) service_pool.cc:224] Handling call
0709 12:27:30.038262 (+1172073us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400162e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:30.038400 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435043200}
I20260709 12:27:30.081971 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.049885 26194 rpcz_store.cc:276] 0709 12:27:28.772257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.802608 (+ 30351us) service_pool.cc:224] Handling call
0709 12:27:28.815571 (+ 12963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.088286 (+272715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.088321 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:29.088365 (+    44us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:29.088400 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.997707 (+909307us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400399c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:29.997769 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.997938 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626620288}
W20260709 12:27:30.078904 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 114660us
W20260709 12:27:30.085636 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.648s (SENT)
I20260709 12:27:30.086262 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.088457 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:30.024194  2181 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.039723  2022 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:30.089115  2338 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.098598  2022 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:30.087311  2324 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:30.093717  2555 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.094204  2487 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:30.095161 26203 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:27:30.099514  2022 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.100315  2480 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:30.095611  2337 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.100929  2480 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:30.096920 26199 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:30.052454  2544 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.085027  2563 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:30.041890  2456 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.097565 26201 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:27:30.100854 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2779) took 1329 ms. Trace:
I20260709 12:27:30.103056 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:30.103571 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64114.9us
I20260709 12:27:30.100570  2022 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:30.110304 26195 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:30.061249  2199 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:30.041517  2439 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:30.039011  2460 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:30.059911 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 192826us
I20260709 12:27:30.119009 26200 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:30.117779 26198 rpcz_store.cc:276] 0709 12:27:28.771544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.772086 (+   542us) service_pool.cc:224] Handling call
0709 12:27:30.100633 (+1328547us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b5400399754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:30.100792 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1688064768}
I20260709 12:27:29.990337  2553 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:30.101961  2480 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.132220 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.112072  2339 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.114126  2544 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.140030 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:30.140334 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.120319  2324 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.124473  2487 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:30.147868  2485 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.148473  2570 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:27:30.149044  2501 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.124300  2563 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:30.149755  2218 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.041541  2557 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.126957  2439 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:30.150010 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:30.150707  2573 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:30.145216  2548 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.172478 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:30.173808 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:30.174193  2548 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.153043  2499 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:30.145483  2199 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:30.175099 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:30.160550  2181 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:30.155428  2324 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:30.144385  2313 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:27:30.176544 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:30.197124 26125 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:30.176772 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:30.182696  2324 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:30.185309 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 53204.5us
I20260709 12:27:30.189831 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4124) took 1420 ms. Trace:
I20260709 12:27:30.175630  2199 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.171060  2498 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:30.171842  2566 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:30.171976 26122 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:30.161922 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:30.195206 26784 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:30.176208 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:30.196359 26127 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:30.184854 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:30.194921 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:30.225308 26123 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 12:27:30.202960 26126 rpcz_store.cc:276] 0709 12:27:28.769033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.251845 (+482812us) service_pool.cc:224] Handling call
0709 12:27:29.252435 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.256626 (+  4191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.256657 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.003402 (+746745us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:27:30.003835 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.070425 (+ 66590us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:30.070466 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.070545 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.189463 (+118918us) raft_consensus.cc:1685] finished
0709 12:27:30.189496 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.189757 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3695,"raft.queue_time_us":4964,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4279,"spinlock_wait_cycles":41562880,"thread_start_us":66181,"threads_started":1,"wal-append.queue_time_us":100499}
I20260709 12:27:30.210983 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.195807  2573 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:30.211544 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 125926us
I20260709 12:27:30.195019  2553 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.212396 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.194105  2557 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.212738 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:30.212044  2313 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:30.214160 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.263836 26053 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:30.165580  2540 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.260118 26756 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.738s (SENT)
I20260709 12:27:30.273355 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.145421  2555 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:27:30.272012 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:30.278456 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 117037us
I20260709 12:27:30.185895  2554 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:30.194319  2501 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.210881  2456 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.289529 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.214958  2498 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:30.348340 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.215684  2548 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.272609  2313 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:30.277822  2470 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.280351  2218 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:30.362417 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 150081us
W20260709 12:27:30.364516 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 82508.3us
I20260709 12:27:30.367748 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.368443 26050 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:30.374475 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.287729  2566 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:30.352862  2487 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:30.171135  2439 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.451829 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.361546  2498 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.499357 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.293833  2577 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:30.501212  2337 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:30.388882  2471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.014s	sys 0.024s
I20260709 12:27:30.501354 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:30.502005  2471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.014s	sys 0.024s
I20260709 12:27:30.457360 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1801) took 1284 ms. Trace:
I20260709 12:27:30.502250 26053 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:30.502467 26783 rpcz_store.cc:276] 0709 12:27:29.173094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.173848 (+   754us) service_pool.cc:224] Handling call
0709 12:27:29.459559 (+285711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.465930 (+  6371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.457131 (+991201us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400565750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:30.457282 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656065920}
I20260709 12:27:30.460031 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.504108 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.504938 26045 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:30.504964 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45029.5us
I20260709 12:27:30.506256 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.506436 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:30.507395  2567 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:30.507824  2567 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:30.462425 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 202335us
I20260709 12:27:30.272301  2573 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:30.466992  2509 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.797s	user 0.006s	sys 0.000s
I20260709 12:27:30.471599  2181 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:30.510495  2573 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:30.472707 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.478603  2199 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.373476  2563 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.512108 26050 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:30.512876 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.513646 26789 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:30.513711 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1425) took 1154 ms. Trace:
I20260709 12:27:30.375531  2548 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:30.375455 26054 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:30.515184 26046 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
W20260709 12:27:30.483707 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109404us
I20260709 12:27:30.485432 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.360877  2540 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:30.485668 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 144688us
I20260709 12:27:30.518311 26018 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.517143 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.508014 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:30.514182 26197 rpcz_store.cc:276] 0709 12:27:29.359470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.088739 (+729269us) service_pool.cc:224] Handling call
0709 12:27:30.110527 (+ 21788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.141389 (+ 30862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.141436 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.156688 (+ 15252us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.356445 (+199757us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:30.356490 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.356569 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.513497 (+156928us) raft_consensus.cc:1685] finished
0709 12:27:30.513517 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.513651 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3513,"raft.queue_time_us":34351,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4818,"thread_start_us":202930,"threads_started":2,"wal-append.queue_time_us":345848}
I20260709 12:27:30.521543 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.484579  2580 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:30.526398 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.529021 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:30.529322 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.529942 26194 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.530120 26783 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:30.527058 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:30.531267 26203 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:30.508210  2567 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:30.527096 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.527654 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.495141  2487 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:30.525854 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.535081 26049 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23. Re-sending same reply.
I20260709 12:27:30.538338 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.532289  2463 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:30.543545 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.546747 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.524896  2555 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:30.556695 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:30.558902 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.558975 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.559768 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.555751 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.556692 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.555570 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.561894 26783 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:27:30.563189 26783 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [23.16-25.17]
I20260709 12:27:30.539868  2581 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:30.565938 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58004.3us
I20260709 12:27:30.539742  2487 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:30.519825  2470 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 12:27:30.573099 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.575376 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:30.576962 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.575559 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:30.516948  2313 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:30.578554 26128 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
W20260709 12:27:30.635283 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 79602.7us
I20260709 12:27:30.637773 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.575037 25940 maintenance_manager.cc:643] P 5eede3841e1242afbeab30840148b558: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 2.871s	user 0.122s	sys 0.076s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":25453,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":172811,"lbm_read_time_us":18624,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":20,"lbm_write_time_us":47235,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":6,"peak_mem_usage":266975,"reinsert_count":0,"thread_start_us":204352,"threads_started":2,"update_count":190}
I20260709 12:27:30.640479 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.606866 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:30.643337 26052 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:30.624107 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.642513 26194 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:30.648720 26018 leader_election.cc:400] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:27:30.649068 26018 leader_election.cc:403] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:30.649590 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:30.545698  2565 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.654328 26018 leader_election.cc:400] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:30.651881 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:30.572767  2337 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.666640 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4102) took 3188 ms. Trace:
I20260709 12:27:30.583943  2501 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:30.646585  2591 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:30.672912 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:30.672199 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.646814  2456 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:30.682799 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.666600  2499 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:30.666847  2181 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:30.693989 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:30.695044 26788 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:30.673026  2337 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:30.696885  2594 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:27:30.698168  2480 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.698675  2480 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.667450 26129 rpcz_store.cc:276] 0709 12:27:27.463574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.788009 (+324435us) service_pool.cc:224] Handling call
0709 12:27:27.851986 (+ 63977us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:28.185307 (+333321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:28.185339 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:28.218755 (+ 33416us) log.cc:844] Serialized 52 byte log entry
0709 12:27:28.490896 (+272141us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:28.490941 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.491009 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.454046 (+1963037us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:30.639292 (+185246us) raft_consensus.cc:1685] finished
0709 12:27:30.639315 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.651550 (+ 12235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182034,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":94,"spinlock_wait_cycles":11153152,"thread_start_us":513599,"threads_started":2,"wal-append.queue_time_us":379246}
I20260709 12:27:30.668972  2475 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:27:30.701633  2580 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:30.702039  2580 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:30.702842  2580 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.669746  2501 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:30.704566  2501 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:30.681320  2389 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:30.708671  2389 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:30.709198  2389 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.717092  2548 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:30.717319  2546 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.688854  2557 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:30.671486  2544 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:30.722676  2544 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.723925 26053 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:30.725179  2586 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:27:30.681864  2456 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.726086  2456 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:30.733544 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.735107 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:30.673431  2581 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:30.745098 26050 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:30.733850  2551 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:30.749615 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.750401 26049 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:30.751013 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.755746 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.758731 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.670396  2555 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 12:27:30.683985  2530 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:30.691174  2499 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.762939 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.823966 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:30.826630 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.827409 26050 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:30.831308  1472 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:30.831480 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 69533.2us
I20260709 12:27:30.632514  2566 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:30.748765  2551 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:30.835350 26049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c47092de97434584b0810190b58e7623"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:27:30.874471 26049 raft_consensus.cc:606] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:30.874984 26049 raft_consensus.cc:614] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:30.829553  2530 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:30.848997  2559 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.046s	user 0.001s	sys 0.007s
I20260709 12:27:30.872488  2540 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:30.891026 26050 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
W20260709 12:27:30.946126 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 113581us
I20260709 12:27:30.829941  2499 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:30.953615 26050 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:30.965905 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.967001 26049 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.969748 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4054) took 1536 ms. Trace:
I20260709 12:27:30.971288 26047 rpcz_store.cc:276] 0709 12:27:29.433342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.433520 (+   178us) service_pool.cc:224] Handling call
0709 12:27:29.447443 (+ 13923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.479615 (+1032172us) spinlock_profiling.cc:243] Waited 954 us on lock 0x7b1800432760. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b7439dbfb 00007f5b743a864b
0709 12:27:30.479798 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.479824 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.480389 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.681094 (+200705us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:30.681143 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.681598 (+   455us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.969143 (+287545us) raft_consensus.cc:1685] finished
0709 12:27:30.969179 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.969675 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29191,"raft.queue_time_us":74222,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":17098,"spinlock_wait_cycles":2194304,"thread_start_us":283137,"threads_started":2,"wal-append.queue_time_us":276569}
I20260709 12:27:30.957785  2540 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:30.956841 26052 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:31.001595 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:30.982226 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:30.983382 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.053328  2463 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:31.055341  2597 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:30.991741  2599 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:31.057469 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.059670 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:31.051322  2413 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:31.062144 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:31.063453 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.065289 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.066362 26052 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24. Re-sending same reply.
I20260709 12:27:31.068192 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:27:31.078902 26786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.021s	sys 0.021s
W20260709 12:27:31.079531 26786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.022s	sys 0.022s
W20260709 12:27:31.065320 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.055903  2555 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:31.082890 26129 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:27:30.998554  2540 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:30.985549  2551 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.162635 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:31.209506 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:31.168903 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3144) took 2391 ms. Trace:
W20260709 12:27:31.165148 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4060) took 1286 ms (client timeout 1644 ms). Trace:
I20260709 12:27:31.212373 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:31.161638 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.140655 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:31.161595 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:31.212592 26054 rpcz_store.cc:269] 0709 12:27:29.878669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.879163 (+   494us) service_pool.cc:224] Handling call
0709 12:27:31.164963 (+1285800us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400172354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:31.165084 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058025728}
I20260709 12:27:31.148969 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.213353 26053 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:31.064635  2413 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.220010 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.166993 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:31.186980 26124 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:27:31.211738 26195 rpcz_store.cc:276] 0709 12:27:28.773277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.778461 (+  5184us) service_pool.cc:224] Handling call
0709 12:27:30.102803 (+1324342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.124114 (+ 21311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.164008 (+1039894us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400399750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:31.164178 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3044995840}
I20260709 12:27:31.223078 26045 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 21. Current term is 22.
I20260709 12:27:31.232947 26129 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:31.238999 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:31.125730  2498 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.004s
W20260709 12:27:31.258056  2487 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.651s	user 0.008s	sys 0.001s
I20260709 12:27:31.257752  2498 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.258478  2487 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:31.161041  2567 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
I20260709 12:27:31.082569  2181 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22.
I20260709 12:27:31.250260 26049 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:31.253224 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:31.252787  2377 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.265568  2377 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.255357 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:31.234628  2501 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.529s	user 0.006s	sys 0.006s
I20260709 12:27:31.232743  2413 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.257108  2601 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:31.261060 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.261513 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2798) took 1375 ms. Trace:
I20260709 12:27:31.092018  2463 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:31.263300 26054 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:31.260716  2597 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:31.240418  2557 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:31.283998 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.301280 26201 rpcz_store.cc:276] 0709 12:27:29.886271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.134499 (+248228us) service_pool.cc:224] Handling call
0709 12:27:30.146948 (+ 12449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.347792 (+200844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.351421 (+  3629us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:30.351474 (+    53us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:30.351540 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.261235 (+909695us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400397e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:31.261282 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.261439 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313616640}
I20260709 12:27:31.094631  2577 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.172s
I20260709 12:27:31.304804 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:31.303913  2551 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:31.308048 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:31.314832 26121 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.315639 26129 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:31.093830  2389 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.070246  2599 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:31.319156 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.321235 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2086) took 1314 ms. Trace:
I20260709 12:27:31.321756 26784 rpcz_store.cc:276] 0709 12:27:30.007082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.008664 (+  1582us) service_pool.cc:224] Handling call
0709 12:27:30.008963 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.259480 (+250517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.320885 (+1061405us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:31.321153 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427871616}
I20260709 12:27:31.323030 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:31.330000 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.330098 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:31.332413 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.304679  2463 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:31.237402  2555 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.338583 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.324895  2181 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.324393  2487 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:31.269632  2567 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.320581  2603 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:31.302645  2501 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.308730  2601 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.337797  2498 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.343447 26129 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:31.355706 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:31.338608  2463 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.360328 26120 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:31.368396  2487 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:31.369282  2389 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.373027  2181 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:31.380851  2467 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.385658  2547 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:31.386241  2555 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.386382  2547 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.389173  2487 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:31.392762  2337 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:31.393774  2337 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:31.394377  2337 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.398273 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:31.398828  2606 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:31.399423  2606 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:31.400539  2606 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.403167 26121 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:31.405083 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:31.405646 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 166756us
I20260709 12:27:31.404074 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:31.406706 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:31.407439 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 280797us
W20260709 12:27:31.332487 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 165551us
W20260709 12:27:31.417788 26094 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.579s (SENT)
I20260709 12:27:31.420051 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:31.421748 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:31.426406 26129 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:31.438793 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.439786 26200 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:31.428593 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:31.469642 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49195.1us
I20260709 12:27:31.412009  2467 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:31.354809  2603 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.420069  2487 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:31.440042  2555 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:31.477552  2108 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:31.482795  2467 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.483366  2387 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:27:31.483959  2498 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:31.492509  2412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.344s	user 0.025s	sys 0.040s
W20260709 12:27:31.508121 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 79648.9us
W20260709 12:27:31.505076  2456 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.778s	user 0.013s	sys 0.000s
W20260709 12:27:31.595160 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73314.6us
W20260709 12:27:31.518045  2499 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.000s
I20260709 12:27:31.500600  2487 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:31.520454  2412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.389s	user 0.025s	sys 0.043s
I20260709 12:27:31.514401  2467 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:31.608345 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 199096us
W20260709 12:27:31.629448 26094 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.604s (SENT)
I20260709 12:27:31.630245 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:31.630548 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1427) took 2265 ms (client timeout 1824 ms). Trace:
I20260709 12:27:31.520293  2547 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.500339  2609 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.114s
I20260709 12:27:31.637521  2615 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:31.606747  2473 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:31.650473 26047 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:27:31.623869  2570 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:31.655346 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 151314us
W20260709 12:27:31.631798 26202 rpcz_store.cc:269] 0709 12:27:29.360627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.420951 (+ 60324us) service_pool.cc:224] Handling call
0709 12:27:31.608037 (+2187086us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b5400301e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:31.625466 (+ 17429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150452480}
I20260709 12:27:31.672467 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:31.629735  2337 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.599699  2456 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.643028  2615 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.623999  2499 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:31.688055 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 201518us
I20260709 12:27:31.655942  2567 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.661899  2599 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.661327  2389 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:31.703929  2487 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.713418  2557 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.688704  2552 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.644964  2501 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.688485  2625 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:31.729377 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 133290us
I20260709 12:27:31.703540  2337 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:31.671928  2473 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.707592  2615 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.682448  2570 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.740669 26049 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: [16.12-20.14]
I20260709 12:27:31.748706 26045 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:31.724774  2599 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:31.726771  2601 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.757030 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:31.705523  2413 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.786794 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.787601 26128 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:31.761260  2588 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.295s	user 0.006s	sys 0.023s
W20260709 12:27:31.762140 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1800) took 2630 ms (client timeout 1765 ms). Trace:
I20260709 12:27:31.765969  2570 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:31.767241 26789 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:31.768867 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.769337 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1186) took 1082 ms. Trace:
I20260709 12:27:31.792095 26048 rpcz_store.cc:276] 0709 12:27:30.687392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.687710 (+   318us) service_pool.cc:224] Handling call
0709 12:27:31.769115 (+1081405us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400029254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:31.769262 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257904384}
I20260709 12:27:31.740007  2337 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.794250  2413 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.770156  2540 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.737488  2625 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.773479  2480 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.777699 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:31.802323 26047 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:31.779475 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1192) took 1016 ms. Trace:
I20260709 12:27:31.781293  2499 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.813004 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3184) took 1647 ms. Trace:
I20260709 12:27:31.741271  2551 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:27:31.784346  2502 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:31.784570 26050 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: [25.15-27.16]
I20260709 12:27:31.822327 26050 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:31.757436 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:31.788187 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:31.789386 26790 rpcz_store.cc:269] 0709 12:27:29.131308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.131428 (+   120us) service_pool.cc:224] Handling call
0709 12:27:31.761859 (+2630431us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400650254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:31.762019 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634871552}
I20260709 12:27:31.746944  2619 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:31.808143 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.796968  2338 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:31.813496 26125 rpcz_store.cc:276] 0709 12:27:30.165544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.197745 (+ 32201us) service_pool.cc:224] Handling call
0709 12:27:30.198101 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:30.213319 (+ 15218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.213352 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:30.214804 (+  1452us) log.cc:844] Serialized 52 byte log entry
0709 12:27:30.365804 (+151000us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:30.482277 (+116473us) log.cc:844] Serialized 14 byte log entry
0709 12:27:30.482565 (+   288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.482666 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.760512 (+1277846us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:31.812637 (+ 52125us) raft_consensus.cc:1685] finished
0709 12:27:31.812753 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.812918 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9838,"prepare.queue_time_us":113225,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":21288,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":14153,"spinlock_wait_cycles":33104256,"thread_start_us":181727,"threads_started":2,"wal-append.queue_time_us":194229}
I20260709 12:27:31.823645 26054 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:31.824775  2619 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.838821  2619 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.839582 26052 raft_consensus.cc:2368] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25.
I20260709 12:27:31.825289  2338 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.842523  2338 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.824054 26120 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:31.844630 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.729646  2552 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.815374 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.802977 26053 rpcz_store.cc:276] 0709 12:27:30.821769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.232514 (+410745us) service_pool.cc:224] Handling call
0709 12:27:31.240371 (+  7857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.350940 (+110569us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:31.350993 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.354077 (+  3084us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.656509 (+302432us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:31.656569 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.778912 (+122343us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:31.778980 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.779050 (+    70us) raft_consensus.cc:1685] finished
0709 12:27:31.779069 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.779399 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253506944,"thread_start_us":149600,"threads_started":1,"wal-append.queue_time_us":278252}
I20260709 12:27:31.819778 26047 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:31.845681 26120 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:27:31.869268 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.869185 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:31.782191  2132 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.826234 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2799) took 1939 ms. Trace:
I20260709 12:27:31.830049 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:31.831003 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:31.873847 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43907us
I20260709 12:27:31.870328 26200 rpcz_store.cc:276] 0709 12:27:29.886572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.134658 (+248086us) service_pool.cc:224] Handling call
0709 12:27:30.145270 (+ 10612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.445954 (+1300684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.826002 (+380048us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400398354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:31.826173 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2976609152}
I20260709 12:27:31.883718 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:31.884552 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:31.877364 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 253693us
I20260709 12:27:31.885469 26125 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 12:27:31.886201 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.820822  2502 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:31.893229 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:31.905148 26200 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:31.856972  2337 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.829073  2499 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:31.871470  2455 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:31.910899 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.877182  2132 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.877426  2622 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.911836 26197 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:31.913075 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:31.914019 26201 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:31.915331  2625 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:31.919472  2463 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.001s
I20260709 12:27:31.921208 26199 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:27:31.958601  2622 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.890277  2456 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.961869  2622 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.921954 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:31.962949 26203 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:31.925839  2551 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.963860  2551 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.956418  2499 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.823573  2615 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.968426  2615 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:31.969465  2551 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.971053  2463 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:31.973727  2463 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:31.973286  2502 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:31.928557 26198 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:31.974793  2502 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:31.982035 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87117.9us
W20260709 12:27:31.986142 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.951644 26125 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:31.997181 26125 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:32.038126 26094 leader_election.cc:336] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.825s (SENT)
I20260709 12:27:32.038820 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:31.998345 26125 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:32.049243 26125 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:32.050863 26053 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:32.053411 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:32.054015 26053 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:32.057950 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:32.049162 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:32.061297 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3170) took 3867 ms. Trace:
I20260709 12:27:32.061816 26124 rpcz_store.cc:276] 0709 12:27:28.193897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.242203 (+ 48306us) service_pool.cc:224] Handling call
0709 12:27:28.267981 (+ 25778us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.240537 (+2972556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.051388 (+810851us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:32.061211 (+  9823us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6712088320}
I20260709 12:27:32.065438 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.068986 26124 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:32.077860 26125 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:32.079252 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.080454 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.082365 26054 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:32.081951 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 409548us
I20260709 12:27:32.050446  2463 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:32.088161  2467 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:32.119454  2467 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:32.120502 26195 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:32.121428  2132 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:32.080870  2455 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.122337  2455 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:32.124696  2455 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:32.097118 26124 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:27:32.127418 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.089093  2631 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:32.102135 26789 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:27:32.111554 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 128570us
I20260709 12:27:32.128822  2631 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:32.129631 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.983s (SENT)
I20260709 12:27:32.130059 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:32.152200 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:32.135188  2463 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:32.154261 26053 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
W20260709 12:27:32.156818 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 247355us
I20260709 12:27:32.131958  2631 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:32.157699 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.164466 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:32.169795 26168 connection.cc:256] RPC call timeout handler was delayed by -1.2049617009997746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
I20260709 12:27:32.161931  2413 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:32.179479 26168 connection.cc:256] RPC call timeout handler was delayed by -1.1581790740001452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 12:27:32.172749 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:32.175083  2626 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.175203 26783 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:32.176429 26784 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:32.176810 26790 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:32.181232 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.165969  2615 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:32.184031 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.170053  2463 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:32.191278 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.205076 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.201467 26791 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:32.190201 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.208204 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:32.229096 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:32.230234 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:32.225726 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:32.235414 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.235946 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38286 (request call id 2082) took 3248 ms. Trace:
I20260709 12:27:32.179976  2622 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:32.186619  2567 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.178s
I20260709 12:27:32.236439 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.557039 26123 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:32.277230 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.372738 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.556142 26791 rpcz_store.cc:276] 0709 12:27:28.957994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.993385 (+ 35391us) service_pool.cc:224] Handling call
0709 12:27:32.182952 (+3189567us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b54005c2954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7439ca3d 00007f5b7936ccd0 00007f5b728f1254 00007f5b728f11e2 00007f5b728f1111 00007f5b728f108c 00007f5b728f0342 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:32.206102 (+ 23150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143141120}
W20260709 12:27:32.555584 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 471551us
I20260709 12:27:32.516455 26785 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:32.386930 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.515484 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:32.417516 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.577801 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.578477 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.626978 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 401329us
I20260709 12:27:32.391271 26124 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 19.
W20260709 12:27:32.513670 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 285565us
I20260709 12:27:32.236755 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:32.644315 26786 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:32.628185 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:32.378487  2634 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:27:32.377213  2324 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:32.676102 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 489520us
I20260709 12:27:32.229784  2540 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:32.674506 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:32.711776 26123 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
W20260709 12:27:32.681232 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.681924 26121 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:32.682211 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:32.713419 26791 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:32.394361  2455 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.393761  2551 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.376471  2487 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.231227  2389 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 29.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.455s
I20260709 12:27:32.239141  2629 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.447s
I20260709 12:27:32.371558  2630 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.691771 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.197901  2577 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.481s
I20260709 12:27:32.648272  2338 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:32.730118 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 163307us
I20260709 12:27:32.735893 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:32.922183 26196 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:32.686702 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.690919  2324 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:32.770653 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:32.923344 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 152823us
I20260709 12:27:32.909960 26123 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:32.911177 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:32.919536 26125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.023s	sys 0.049s
W20260709 12:27:32.925146 26125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.023s	sys 0.049s
I20260709 12:27:32.921730 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:32.648919  2604 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.130s
I20260709 12:27:32.653939  2377 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:27:32.674644  2401 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.238s
I20260709 12:27:32.924330 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:32.231566  2615 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:32.711081  2540 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.982177 26194 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:32.212009  2626 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:32.579914  2622 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:32.941124  2467 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.947433 26123 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:32.927186  2338 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:32.948308  2407 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:32.949158  2641 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.391s	user 0.001s	sys 0.000s
I20260709 12:27:32.964818  2513 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:32.967371 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:32.745695  2551 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:32.922056  2389 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.947512  2450 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:33.016873 26123 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:32.986852  2540 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:32.950527  2577 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:33.035588 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.080s	sys 0.155s
W20260709 12:27:33.054605 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.080s	sys 0.155s
I20260709 12:27:32.746043  2455 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:32.768762  2487 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:33.043677 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2717) took 1304 ms. Trace:
I20260709 12:27:33.060000 26051 rpcz_store.cc:276] 0709 12:27:31.738975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.739122 (+   147us) service_pool.cc:224] Handling call
0709 12:27:31.777065 (+ 37943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.777476 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:31.777507 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.777552 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:31.777596 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.777658 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.043410 (+1265752us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54002d0350. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:33.043585 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2428,"raft.queue_time_us":12124,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2050,"spinlock_wait_cycles":73765248}
I20260709 12:27:33.061654 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4144) took 1712 ms. Trace:
I20260709 12:27:32.971719  2554 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:32.975486 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51303.2us
I20260709 12:27:33.047349 26123 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:33.066007 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.019s	sys 0.053s
W20260709 12:27:33.068459 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.020s	sys 0.053s
W20260709 12:27:33.070545 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 882803us
I20260709 12:27:32.988121  2498 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:33.027993  2338 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:33.078733 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:33.066586 26123 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:33.062227 26127 rpcz_store.cc:276] 0709 12:27:31.348835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.070410 (+721575us) service_pool.cc:224] Handling call
0709 12:27:32.070686 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.070877 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:32.168750 (+ 97873us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.061025 (+892275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.061104 (+    79us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:33.061308 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.061405 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.061573 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5865,"prepare.queue_time_us":166323,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"thread_start_us":953140,"threads_started":2}
I20260709 12:27:33.036217  2513 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.106546 26200 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:32.926200  2630 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:33.111188 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 548203us
W20260709 12:27:33.109432 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.075223  2554 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:33.112171 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.114331 26196 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:33.116017 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:33.117020 26127 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:33.117381 26127 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:33.117220 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:33.117679  2554 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:33.118288 26127 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.121295 26127 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:33.042824  2407 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.124574  2407 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:33.129400  2531 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:33.129895  2531 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:33.110997  2645 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.012s
I20260709 12:27:33.058774  2577 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:33.080041  2377 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:33.029142  2467 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:33.145377 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2805) took 1255 ms (client timeout 1612 ms). Trace:
W20260709 12:27:33.145871 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.145922 26195 rpcz_store.cc:269] 0709 12:27:31.889612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.889743 (+   131us) service_pool.cc:224] Handling call
0709 12:27:33.145086 (+1255343us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400335754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:33.145290 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374454656}
I20260709 12:27:33.146342 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:33.148085 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3190) took 2022 ms (client timeout 2268 ms). Trace:
W20260709 12:27:33.149163 26128 rpcz_store.cc:269] 0709 12:27:31.125551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.151224 (+ 25673us) service_pool.cc:224] Handling call
0709 12:27:33.147826 (+1996602us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:33.148008 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314482432}
I20260709 12:27:33.150766 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:33.151561 26128 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:33.130751  2531 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:33.153718  2531 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:33.156387  2502 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:27:33.173034 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.193902 26195 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:33.201156 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.103075  2513 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:33.210368 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3160) took 1011 ms. Trace:
I20260709 12:27:33.210807 26195 rpcz_store.cc:276] 0709 12:27:32.198664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.147339 (+948675us) service_pool.cc:224] Handling call
0709 12:27:33.147738 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.210117 (+ 62379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.210311 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.263494 26127 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:33.058373  2450 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.264842 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:33.266045 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:33.053660  2389 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.281662 26123 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:27:33.114887  2498 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:33.477979 26123 pending_rounds.cc:85] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:27:33.142948  2577 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:33.300541 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:33.143671  2377 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:33.435153  2622 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:33.353560  2407 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.302330  2502 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.134s
I20260709 12:27:33.437857  2615 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:33.441735  2567 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.465698  2389 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:33.460011  2551 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:33.295885 26786 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.002s
I20260709 12:27:33.422962  2554 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:33.727936 26125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.019s	sys 0.018s
W20260709 12:27:33.886003 26125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.024s	sys 0.020s
W20260709 12:27:33.748701 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.015s	sys 0.035s
W20260709 12:27:33.653007 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.826575 26095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.000s	sys 0.010s
I20260709 12:27:33.887524 26123 pending_rounds.cc:107] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:27:33.887938 26123 raft_consensus.cc:2889] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:33.683723 26782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.010s	sys 0.026s
W20260709 12:27:33.888795 26782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.010s	sys 0.034s
I20260709 12:27:33.888906 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4153) took 1822 ms. Trace:
I20260709 12:27:33.889479 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:33.836982 26196 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.006s	sys 0.005s
I20260709 12:27:33.886022 26786 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:33.890839 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:33.891235 26196 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.006s	sys 0.006s
I20260709 12:27:33.891743 26126 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
W20260709 12:27:33.869460 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.196s	user 0.040s	sys 0.090s
W20260709 12:27:33.893498 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.040s	sys 0.090s
W20260709 12:27:33.887542 26095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.000s	sys 0.012s
W20260709 12:27:33.893891 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.26575e+06us
W20260709 12:27:33.685588 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.011s	sys 0.010s
W20260709 12:27:33.895576 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.013s	sys 0.016s
I20260709 12:27:33.889606 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2096) took 1715 ms. Trace:
W20260709 12:27:33.896117 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 974303us
W20260709 12:27:33.887562 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.899102 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.886871 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.018s	sys 0.038s
W20260709 12:27:33.902504 26019 connection.cc:256] RPC call timeout handler was delayed by -1.1345555849998163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2268s for a response
W20260709 12:27:33.903355 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.125s (SENT) [suppressed 27 similar messages]
W20260709 12:27:33.891322 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2099) took 1716 ms (client timeout 1886 ms). Trace:
W20260709 12:27:33.908066 26786 rpcz_store.cc:269] 0709 12:27:32.178283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.236458 (+ 58175us) service_pool.cc:224] Handling call
0709 12:27:33.891270 (+1654812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.909049 26126 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:33.910241 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4156) took 1823 ms. Trace:
I20260709 12:27:33.910604 26126 rpcz_store.cc:276] 0709 12:27:32.087123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.908433 (+1821310us) service_pool.cc:224] Handling call
0709 12:27:33.908741 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.909985 (+  1244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.910190 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.897660 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:27:33.579383  2638 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.343s	user 0.004s	sys 0.002s
I20260709 12:27:33.915045 26200 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:33.923041 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3198) took 1682 ms. Trace:
I20260709 12:27:33.923707 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3163) took 1556 ms. Trace:
I20260709 12:27:33.923593 26126 rpcz_store.cc:276] 0709 12:27:32.241071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.922245 (+1681174us) service_pool.cc:224] Handling call
0709 12:27:33.922544 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.922723 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.922751 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.922797 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:33.922836 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.922888 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.922990 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.889442 26123 rpcz_store.cc:276] 0709 12:27:32.066387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.940997 (+874610us) service_pool.cc:224] Handling call
0709 12:27:32.947262 (+  6265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.888537 (+941275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.888838 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.916895 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:33.917042 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:33.918107 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:33.885054  2450 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:33.918757 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:33.822522  2647 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.504s	user 0.000s	sys 0.002s
W20260709 12:27:33.759537  2644 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.641s	user 0.000s	sys 0.002s
I20260709 12:27:33.925958 26053 raft_consensus.cc:2376] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22. Re-sending same reply.
W20260709 12:27:33.708949  2463 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.006s	sys 0.016s
I20260709 12:27:33.896143 26782 rpcz_store.cc:276] 0709 12:27:32.174041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.182133 (+  8092us) service_pool.cc:224] Handling call
0709 12:27:32.200840 (+ 18707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.201075 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:32.919453 (+718378us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.889248 (+969795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.889300 (+    52us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:33.889348 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.889403 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.889524 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1065852,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":92,"thread_start_us":1368929,"threads_started":2}
W20260709 12:27:33.860598  2645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.004s	sys 0.013s
W20260709 12:27:33.930699 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.003s	sys 0.014s
I20260709 12:27:33.926494 26045 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:33.885025  2530 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:33.931214 26093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.011s	sys 0.014s
W20260709 12:27:33.931792 26093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.011s	sys 0.014s
I20260709 12:27:33.931821 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:33.932183 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 810828us
I20260709 12:27:33.933442 26054 raft_consensus.cc:2410] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:27:33.934150 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:33.934902 26051 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:33.936054 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:33.939709 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.193s	user 0.004s	sys 0.024s
W20260709 12:27:33.940155 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.004s	sys 0.024s
I20260709 12:27:33.749636  2455 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:33.928516 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:33.943882 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3201) took 1577 ms. Trace:
I20260709 12:27:33.944304 26126 rpcz_store.cc:276] 0709 12:27:32.366997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.943074 (+1576077us) service_pool.cc:224] Handling call
0709 12:27:33.943364 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.943564 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.943588 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.943630 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:33.943670 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.943725 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.943828 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.942852 26050 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:33.889997  2626 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:33.946709 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.003s	sys 0.014s
W20260709 12:27:33.947088 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 833622us
I20260709 12:27:33.924578 26200 rpcz_store.cc:276] 0709 12:27:32.367704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.209324 (+841620us) service_pool.cc:224] Handling call
0709 12:27:33.914776 (+705452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.923292 (+  8516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.923630 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.947988 26123 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:27:33.948812 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3202) took 1579 ms. Trace:
I20260709 12:27:33.949258 26123 rpcz_store.cc:276] 0709 12:27:32.369440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.941314 (+1571874us) service_pool.cc:224] Handling call
0709 12:27:33.947316 (+  6002us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:33.948492 (+  1176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.948751 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.907500  2551 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.951100 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.018s	sys 0.007s
I20260709 12:27:33.951535 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:33.930663  2463 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.008s	sys 0.016s
I20260709 12:27:33.914470  2615 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:33.967130  2658 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:33.967090  2615 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.967703  2658 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:33.968649  2658 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.969321  2615 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:33.969748  2513 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.000s
I20260709 12:27:33.970160 26047 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:33.970256  2551 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.971263  2498 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:33.973582  2551 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:33.955056 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 447539us
W20260709 12:27:33.930971  2645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.004s	sys 0.014s
I20260709 12:27:33.956616 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1618) took 2222 ms. Trace:
I20260709 12:27:33.956643 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:33.959745 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:33.979235 26050 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:33.959486 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.018s	sys 0.007s
I20260709 12:27:33.961874 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3203) took 1592 ms. Trace:
W20260709 12:27:33.980944 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 837568us
I20260709 12:27:33.980988 26126 rpcz_store.cc:276] 0709 12:27:32.369763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.945203 (+1575440us) service_pool.cc:224] Handling call
0709 12:27:33.960059 (+ 14856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.961391 (+  1332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.961458 (+    67us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.961503 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:33.961661 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.961718 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.961822 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:33.895134  2567 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:33.963140 26045 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 29 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:27:33.964313 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:33.966192  2324 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:33.928647  2622 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.948040 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:33.942170  2630 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:33.983373 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4160) took 1605 ms. Trace:
I20260709 12:27:33.977386 26122 rpcz_store.cc:276] 0709 12:27:31.734605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.176332 (+441727us) service_pool.cc:224] Handling call
0709 12:27:32.176637 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.176977 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.177004 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.177517 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:27:32.560930 (+383413us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:32.560977 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.561045 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.956409 (+1395364us) raft_consensus.cc:1685] finished
0709 12:27:33.956433 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.956562 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3187,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":8568,"thread_start_us":383013,"threads_started":1,"wal-append.queue_time_us":852422}
I20260709 12:27:33.953083  2530 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:33.955770  2626 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.994330 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.011564  2324 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:27:33.998767 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 95448.7us
I20260709 12:27:33.908036  2407 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.003456 26123 rpcz_store.cc:276] 0709 12:27:32.377722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.957988 (+1580266us) service_pool.cc:224] Handling call
0709 12:27:33.982682 (+ 24694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.982910 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.982949 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.983008 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:33.983085 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.983161 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.983301 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:34.009606 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61643.7us
I20260709 12:27:34.028961 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.029645 26122 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:34.029786 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:34.010388 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.031880 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.011656 26051 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:34.031414 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 85095.5us
I20260709 12:27:34.013171  2567 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.013803  2630 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:33.999626 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.015332 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.040331 26126 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
W20260709 12:27:34.040815 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44084.1us
I20260709 12:27:34.018437 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.041888 26050 raft_consensus.cc:2376] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22. Re-sending same reply.
I20260709 12:27:33.970149  2513 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.032675 26054 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
W20260709 12:27:34.034238 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93637.6us
W20260709 12:27:34.017933  2338 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.906s	user 0.008s	sys 0.004s
I20260709 12:27:34.048056 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.045254 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.046756 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:34.046904 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.052954 26123 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:34.026050  2626 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.062752  2487 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:34.063063 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.064144  2626 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.064584 26126 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:34.065253 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1620) took 1053 ms. Trace:
I20260709 12:27:34.065541 26126 rpcz_store.cc:276] 0709 12:27:33.011597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.064090 (+1052493us) service_pool.cc:224] Handling call
0709 12:27:34.064407 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.064982 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.065200 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:34.067070 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1623) took 1054 ms. Trace:
I20260709 12:27:33.959367  2377 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.038388  2455 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:34.070859  2313 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:34.072584  2537 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:34.073427  2630 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:34.074297  2407 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.074681  2630 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.014281  2530 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.077105  2646 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:34.077966  2646 consensus_queue.cc:1243] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:34.075139  2664 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:34.078513  2646 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:34.079192  2664 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:34.079607  2646 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:34.079701  2664 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:34.080185  2655 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:27:34.033159  2658 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:34.025825  2551 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.086967 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.067488 26126 rpcz_store.cc:276] 0709 12:27:33.012422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.066214 (+1053792us) service_pool.cc:224] Handling call
0709 12:27:34.066495 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.066677 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.066717 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.066776 (+    59us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:34.066834 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.066902 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.067022 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:34.087620 26122 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:34.090306 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.082669 26123 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:27:34.090947 26195 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:34.091260 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1619) took 1080 ms. Trace:
I20260709 12:27:34.026170  2621 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.025887  2620 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:34.091630 26123 rpcz_store.cc:276] 0709 12:27:33.011250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.081832 (+1070582us) service_pool.cc:224] Handling call
0709 12:27:34.082285 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.090998 (+  8713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.091205 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:34.092600 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49045.9us
W20260709 12:27:34.093942 26094 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.844s (SENT)
I20260709 12:27:34.016939  2622 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.094934 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.096832 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3159) took 1901 ms. Trace:
I20260709 12:27:34.100322 26050 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:27:34.100965 26051 raft_consensus.cc:2376] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27. Re-sending same reply.
I20260709 12:27:34.099972 26196 rpcz_store.cc:276] 0709 12:27:32.195867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.128334 (+932467us) service_pool.cc:224] Handling call
0709 12:27:33.128752 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.129029 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:33.892040 (+763011us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.096447 (+204407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.096497 (+    50us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:34.096571 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.096652 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.096770 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13771,"prepare.queue_time_us":959968,"raft.queue_time_us":14527,"thread_start_us":952712,"threads_started":2}
I20260709 12:27:34.101984 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.031935  2645 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.057s
I20260709 12:27:34.103653 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.102841 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4157) took 1996 ms. Trace:
I20260709 12:27:34.105912 26122 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:34.106652 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1627) took 1093 ms. Trace:
I20260709 12:27:34.107501 26122 rpcz_store.cc:276] 0709 12:27:33.013437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.088381 (+1074944us) service_pool.cc:224] Handling call
0709 12:27:34.103643 (+ 15262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.106387 (+  2744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.106594 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:34.080178  2646 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:34.077988  2513 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.109323 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1625) took 1096 ms. Trace:
I20260709 12:27:34.106148 26200 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:34.109866 26126 rpcz_store.cc:276] 0709 12:27:33.012927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.088318 (+1075391us) service_pool.cc:224] Handling call
0709 12:27:34.108700 (+ 20382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.109010 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.109040 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.109092 (+    52us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:34.109126 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.109176 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.109272 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:34.110236 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.113809 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.114511 26050 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:34.114532 26203 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:27:34.117174  2463 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:34.117616  2463 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:34.118530  2463 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.086694  2324 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:27:34.120909  2604 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.081470  2338 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.121436  2604 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.123046  2338 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.093152  2551 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.104385  2620 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.134588 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.120056  2463 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.131505 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.155860  2551 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:34.159186  2540 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.081s	user 0.010s	sys 0.000s
W20260709 12:27:34.160877  2657 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:34.103192  2626 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
W20260709 12:27:34.168648 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74712.6us
I20260709 12:27:34.067059  2577 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:34.171547 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.153467 26050 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 21.
W20260709 12:27:34.177153 26019 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.538s (SENT)
I20260709 12:27:34.169454  2626 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:27:34.081079  2664 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.187994  2604 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:34.114041  2487 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:34.190075  2487 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.188066 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:34.207278 26019 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.574s (SENT)
I20260709 12:27:34.176787  2540 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.102916  2621 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.209619 26049 raft_consensus.cc:2368] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29.
I20260709 12:27:34.212589 26045 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:34.159977  2646 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.216960  2621 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.193665 26200 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:34.219842 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.220558 26200 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:34.224061 26195 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:34.233866  2620 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.226052  2664 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:34.225819 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.132893  2622 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.123912  2659 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:34.133536  2530 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.228559  2455 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:34.224936  2531 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.042s	user 0.012s	sys 0.002s
I20260709 12:27:34.244650 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.225507  2646 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:34.246129  2540 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.230408 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: true dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.247186 26050 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:34.252532 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:34.226343 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52622.8us
I20260709 12:27:34.253258 26126 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:34.254828 26127 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:34.255437 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.256441 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.261641 26195 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:34.255843  2577 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.269455  2614 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:27:34.270023 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.273219  2659 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.273845  2659 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:34.276327 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.278621 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.269966  2614 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:34.280308 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.282232 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.288874 26018 connection.cc:256] RPC call timeout handler was delayed by -1.4961526280003454s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:27:34.289427 26018 connection.cc:256] RPC call timeout handler was delayed by -1.463752628000293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 12:27:34.289361 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:34.289969 26018 connection.cc:256] RPC call timeout handler was delayed by -1.218952628000352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
W20260709 12:27:34.291837 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.293412 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.295545 26050 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:34.267969  2487 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.301039 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.301393 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.304033 26200 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:34.305917 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.306895 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.299106 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.316845 26195 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:34.306653 26785 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:34.318393 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.320324 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.320451 26200 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.318778 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.323407 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:34.323994 26791 raft_consensus.cc:2376] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25. Re-sending same reply.
I20260709 12:27:34.293896  2622 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.328900 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.286798  2530 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:34.286515  2531 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:34.330492 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:34.330354 26167 leader_election.cc:336] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.581s (SENT)
I20260709 12:27:34.301720  2663 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:34.328080 26756 connection.cc:256] RPC call timeout handler was delayed by -2.7243239630001881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 12:27:34.332739 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:34.334542 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.333967 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.337292 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:34.339001 26782 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:34.340667 26195 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:34.341596 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.342582  2455 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:34.342597  2577 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:34.334453  2622 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.343367  2577 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:34.344558  2664 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:34.344424  2577 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.329932  2658 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:34.338456  2678 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:34.345597  2658 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:34.345892  2678 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.346745  2658 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.348153  2678 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.340543  2630 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.342761 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:34.350018 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4154) took 2283 ms. Trace:
I20260709 12:27:34.350409 26127 rpcz_store.cc:276] 0709 12:27:32.066636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.298495 (+1231859us) service_pool.cc:224] Handling call
0709 12:27:33.322433 (+ 23938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.255337 (+932904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.349773 (+ 94436us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:34.349947 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2143751040}
W20260709 12:27:34.353842 26019 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.721s (SENT)
W20260709 12:27:34.362733 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.414505 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.450595 26756 connection.cc:256] RPC call timeout handler was delayed by -2.1096877610002593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 12:27:34.451527 26756 connection.cc:256] RPC call timeout handler was delayed by -1.9134877610003969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
I20260709 12:27:34.334558  2646 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:34.716333 26200 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:34.717532 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.718345 26200 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:34.720072 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.345108  2622 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.480729  2679 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.335457  2614 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.743961  2614 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.745644  2461 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:34.747138  2461 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:34.745025  2580 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:34.337975  2676 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.748629  2676 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:34.722990 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 238212us
I20260709 12:27:34.750298 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.750988 26201 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 21.
I20260709 12:27:34.751925 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.752566 26201 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:34.753154 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.753424 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.335582  2530 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.479135  2463 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:34.766571 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:34.727119 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.727041 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:34.738890 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 385046us
I20260709 12:27:34.758498  2339 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:34.727963 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.742115 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4146) took 3384 ms. Trace:
I20260709 12:27:34.478171  2664 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.769506  2339 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:34.770160  2664 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.771142  2622 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:34.484933  2482 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.452054 26756 connection.cc:256] RPC call timeout handler was delayed by -1.8918877610003619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 12:27:34.772595 26756 connection.cc:256] RPC call timeout handler was delayed by -1.5795877610003117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
I20260709 12:27:34.772892 26200 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:34.772957 26756 connection.cc:256] RPC call timeout handler was delayed by -1.3797877610004434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2429s for a response
I20260709 12:27:34.769770 26125 rpcz_store.cc:276] 0709 12:27:31.357376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.088824 (+731448us) service_pool.cc:224] Handling call
0709 12:27:32.089722 (+   898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.132486 (+ 42764us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:32.230274 (+ 97788us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.931954 (+701680us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.893385 (+961431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.893436 (+    51us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:33.893484 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.741538 (+848054us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b54002ca6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:34.741674 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.742032 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6043,"prepare.queue_time_us":104996,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":17133,"raft.queue_time_us":75223,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":148,"spinlock_wait_cycles":97757056,"thread_start_us":1753577,"threads_started":3,"wal-append.queue_time_us":912941}
I20260709 12:27:34.756919 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:34.778621 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.769827 26048 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:34.779217 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.718282  2455 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.763767 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.764549 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.771744  2622 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.782219 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.782564  2498 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:34.782348  2482 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.782397  2603 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.782235 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.792553 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.792877 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.793419 26195 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:34.787492 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:34.785138 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44076.6us
I20260709 12:27:34.354743  2531 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.788870 26050 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:34.768790 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:34.799744 26052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2719) took 3026 ms (client timeout 1844 ms). Trace:
I20260709 12:27:34.793905 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.723512  2660 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.783758  2664 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:34.800393 26052 rpcz_store.cc:269] 0709 12:27:31.739602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.739709 (+   107us) service_pool.cc:224] Handling call
0709 12:27:34.765843 (+3026134us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b54003593d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:34.766036 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121426688}
I20260709 12:27:34.753845  2659 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:34.797610 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.801808 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2726) took 3033 ms. Trace:
I20260709 12:27:34.812273 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.801290 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.485396  2630 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.773344  2470 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:34.829538 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.837059 26050 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:34.791648  2498 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.771327  2339 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.812842 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.812614 26048 rpcz_store.cc:276] 0709 12:27:31.751579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.793222 (+ 41643us) service_pool.cc:224] Handling call
0709 12:27:34.769626 (+2976404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.783318 (+ 13692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.784727 (+  1409us) spinlock_profiling.cc:243] Waited 2.97 s on lock 0x7b54003593d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:34.784907 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6820975488}
I20260709 12:27:34.794430 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.814111 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.814426 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.725590  2665 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 29.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.073s
W20260709 12:27:34.815658 26783 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.013s	sys 0.039s
I20260709 12:27:34.841791 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.801270  2218 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:34.843112 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:34.851821 26052 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.774497  2663 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.053s
I20260709 12:27:34.855188 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.844328 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.763258  2604 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.050s
I20260709 12:27:34.845674 26125 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:27:34.765367  2645 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.055s
I20260709 12:27:34.846702 26129 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:34.871758  2622 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.872174  2407 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:34.872608  2407 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:34.872808  2676 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:34.844463  2680 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:27:34.873600  2407 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.846884  2653 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:34.875092  2407 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:34.843410  2498 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:34.877832  2678 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:34.878834  2218 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:34.879784  2218 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.880589  2530 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.881119 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:34.801118  2455 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.843202  2664 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.883134 26197 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:34.883008  2664 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:34.785658  2463 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.846079 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.849025 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:34.849269 26120 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:27:34.808586  2544 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:34.846709 26127 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:27:34.911128 26791 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:34.911682 26126 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:34.915591 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.923604 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.924511 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:34.925980 26197 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:34.910478 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.927681 26790 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:34.928263 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2730) took 1011 ms. Trace:
I20260709 12:27:34.928639 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.928977  2552 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.929347 26790 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:34.929755  2377 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:34.930235 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:34.930826 26790 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:34.930903  2313 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.859s	user 0.013s	sys 0.000s
I20260709 12:27:34.932349 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:34.932711 26094 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.551s (SENT)
I20260709 12:27:34.933204 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:34.933537 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.934211 26787 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:34.934322  2663 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:34.934999  2499 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.935149 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.935925 26787 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:34.936630 26046 raft_consensus.cc:2410] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:34.936681 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.937649 26054 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:34.928661 26049 rpcz_store.cc:276] 0709 12:27:33.916716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.916826 (+   110us) service_pool.cc:224] Handling call
0709 12:27:34.928039 (+1011213us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:34.928185 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652400512}
I20260709 12:27:34.950014 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.950611 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:34.950920 26049 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:34.951917 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:34.954664  2555 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.955209 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.961877 26787 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:34.963920 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.966358 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.967239 26054 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:34.974917 26052 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:34.812749 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:34.848491 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:34.851265 26783 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.013s	sys 0.039s
I20260709 12:27:34.804194  2603 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:27:34.761276  2679 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:34.858563  2659 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:34.858942 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.868450  2339 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.869611  2630 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.811241  2660 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:34.904407 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.955727  2531 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:34.978189 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:34.979451 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.965581 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:34.978261 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:34.997589 26791 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:34.981946 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.000034 26199 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:27:34.984715 26201 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:35.001746 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:34.980540  2473 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.983139 26786 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:34.843705 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:34.992177 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:34.976855  2544 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:34.976655  2540 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:35.013769  2660 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.013863  2540 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.014407  2540 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:35.014533  2313 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:34.955225  2555 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:35.007524 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 261123us
W20260709 12:27:35.007985 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.008520 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:35.012871  2455 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.016079 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.016937 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:35.023561 26050 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:34.982703 26197 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:27:35.018229 26790 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:35.013685  2544 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.025441 26048 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:35.025645  2544 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.026439 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.029176 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.030269 26054 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:35.031636 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.033274 26054 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:35.037616  2463 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.040884 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:35.043463 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.043958  2482 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.013662  2679 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.049402 26052 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:35.057580 26791 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:35.071808 26790 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:35.075807 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:35.078408 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.081156 26167 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.031s	sys 0.040s
W20260709 12:27:35.081593 26167 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.032s	sys 0.040s
W20260709 12:27:35.081911 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 727361us
I20260709 12:27:35.013923  2499 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.086867 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:35.092881 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.076999 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:35.048825  2531 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:35.087237 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:35.095396 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.096544 26052 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:27:35.096697 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.097476 26200 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:35.069185  2552 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:35.099727 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.069106  2530 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.112651 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.113696 26050 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:35.091038  2680 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:35.115653 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 314681us
I20260709 12:27:35.117139 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.097321  2619 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:35.119259 26201 raft_consensus.cc:1217] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Deduplicated request from leader. Original: 18.13->[20.14-20.14]   Dedup: 20.14->[]
I20260709 12:27:35.100656  2455 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:35.122054  2622 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:35.124338 26201 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:35.095065  2531 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:35.120893 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.180577 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 384196us
I20260709 12:27:35.125873 26052 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: [27.19-28.20]
I20260709 12:27:35.181720 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:35.182381 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2735) took 1263 ms. Trace:
I20260709 12:27:35.182729 26052 rpcz_store.cc:276] 0709 12:27:33.918972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.107313 (+1188341us) service_pool.cc:224] Handling call
0709 12:27:35.125545 (+ 18232us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:35.182093 (+ 56548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.182322 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.127311 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.102989  2499 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.091495  2562 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:27:35.157186 26053 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:35.157189  2377 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:35.108392  2313 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:27:35.165107  2676 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:35.115885  2689 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:35.124053  2619 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:35.187201 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.164652 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.105201  2544 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.220474 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.186515  2531 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.111311  2530 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.177759  2455 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.221642 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:35.229928  2467 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.230949 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.232142 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:35.229928  2530 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.232673 26052 raft_consensus.cc:2376] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25. Re-sending same reply.
I20260709 12:27:35.187755  2622 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:27:35.233561 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.234643 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.235605 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:35.237460  2455 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.238286 26121 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:35.238670 26121 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:35.179960  2680 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:35.210211  2689 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.186690  2630 raft_consensus.cc:2764] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:35.102330  2552 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:35.242122 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 386515us
W20260709 12:27:35.246204 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.246778 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.248381 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.203253  2660 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:35.258914 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 157423us
I20260709 12:27:35.252840 26049 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 20: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:35.249410 26121 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.260937 26121 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:35.208226  2377 consensus_queue.cc:1243] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:27:35.283847 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 174482us
I20260709 12:27:35.264289 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:35.250823 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 298307us
I20260709 12:27:35.240548  2510 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:35.284723 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1819) took 4746 ms. Trace:
I20260709 12:27:35.285310 26787 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:35.291538 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:35.294010 26094 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.822s (SENT)
I20260709 12:27:35.292887 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:35.297185 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.257917 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2803) took 3382 ms. Trace:
I20260709 12:27:35.298136 26202 rpcz_store.cc:276] 0709 12:27:31.875892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.876478 (+   586us) service_pool.cc:224] Handling call
0709 12:27:31.905097 (+ 28619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.966942 (+ 61845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.966980 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.967028 (+    48us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:31.967076 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.257665 (+3290589us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b5400396f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:35.257708 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.257856 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141632768}
I20260709 12:27:35.304764 26121 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:35.295305 26789 rpcz_store.cc:276] 0709 12:27:30.514700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.517690 (+  2990us) service_pool.cc:224] Handling call
0709 12:27:31.767067 (+1249377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.128782 (+361715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.258568 (+3129786us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400650250. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.258706 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872570240}
I20260709 12:27:35.307781 26789 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:35.251217  2552 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:35.258193  2659 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:35.309587 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 185796us
I20260709 12:27:35.311944 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.245393  2689 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:35.306777  2471 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:27:35.309566 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4151) took 3243 ms. Trace:
W20260709 12:27:35.313660 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.324030 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.324698 26094 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.094s (SENT)
I20260709 12:27:35.325052 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:35.325498 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.287124  2614 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:35.261670  2401 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:27:35.326924 26121 rpcz_store.cc:276] 0709 12:27:32.065790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.713580 (+647790us) service_pool.cc:224] Handling call
0709 12:27:32.713863 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.238188 (+2524325us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54002a65a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:27:35.306573 (+ 68385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.309499 (+  2926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510919552}
W20260709 12:27:35.332371 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70126.6us
I20260709 12:27:35.327131 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.306075  2407 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.308947  2377 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:35.314122  2439 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.313320  2327 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:27:35.285602  2660 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.350219 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.352413  2660 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.309860  2663 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:35.355764  2499 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:34.106043 26128 rpcz_store.cc:276] 0709 12:27:32.106457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.921764 (+1815307us) service_pool.cc:224] Handling call
0709 12:27:33.942040 (+ 20276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.942266 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:34.031107 (+ 88841us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.102511 (+ 71404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.102560 (+    49us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:34.102609 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.102686 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.102790 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12733,"raft.queue_time_us":15490,"thread_start_us":131500,"threads_started":2}
I20260709 12:27:35.310534  2562 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:27:35.371330 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.289932  2653 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:35.356513  2566 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:35.345312  2645 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:35.345534  2614 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:35.361003  2699 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:35.346944 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.362226  2544 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.312410  2692 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:35.267067  2498 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.345714 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:35.400857 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55247.1us
I20260709 12:27:35.400980 26121 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: [27.21-27.21]
I20260709 12:27:35.401620 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:35.402136  2498 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.402559  2701 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:35.402966  2544 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:35.403630  2544 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:35.348193 26201 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:35.404052  2544 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.375530 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.404749 26122 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:35.405696 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.406265 26122 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:35.345418  2377 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.407107  2377 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.378779 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:35.381862 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 118204us
I20260709 12:27:35.380041  2689 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.411164 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.382479  2699 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:35.411903  2699 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:35.384254 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.412940  2689 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:35.384029  2673 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:35.414583 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:35.415215 26128 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:35.339814  2646 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:27:35.333657  2313 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:35.428385  2313 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:35.428996  2313 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.430347  2486 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:35.430634  2697 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:27:35.431021  2486 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.432479 26784 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:35.433818 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2102) took 1160 ms. Trace:
I20260709 12:27:35.434738 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.434243 26783 rpcz_store.cc:276] 0709 12:27:34.273007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.276362 (+  3355us) service_pool.cc:224] Handling call
0709 12:27:34.276738 (+   376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:34.277160 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:34.978201 (+701041us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.192460 (+214259us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:35.209594 (+ 17134us) log.cc:844] Serialized 52 byte log entry
0709 12:27:35.209796 (+   202us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:35.209843 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.209933 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.433573 (+223640us) raft_consensus.cc:1685] finished
0709 12:27:35.433607 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.433759 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4215,"prepare.queue_time_us":561385,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":7764,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":23620,"thread_start_us":875651,"threads_started":2,"wal-append.queue_time_us":330982}
W20260709 12:27:35.437203  2580 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.689s	user 0.008s	sys 0.005s
I20260709 12:27:35.437536 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.438833 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.439630 26783 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:35.440409 26195 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:35.440793  2439 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:35.441009  2451 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.441471  2439 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:35.350054  2407 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.442421  2703 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:27:35.444185  2463 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.367420  2540 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.421242 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.446440  2540 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.446678 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.447402 26122 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:35.403033  2701 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.440673  2614 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.448447  2701 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.366719  2680 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.441592  2451 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.452476  2552 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.454044 26048 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:35.454393  2658 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:35.454742  2552 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.454895  2389 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:35.454977  2658 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.455546  2389 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:35.459794 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3154) took 3274 ms (client timeout 2125 ms). Trace:
W20260709 12:27:35.460297 26194 rpcz_store.cc:269] 0709 12:27:32.185414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.648079 (+462665us) service_pool.cc:224] Handling call
0709 12:27:35.459577 (+2811498us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.459710 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257204096}
I20260709 12:27:35.444434  2407 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.462026 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.465106 26785 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:35.442186 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:35.466578 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.466733 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.372723 26782 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:35.462806  2513 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:35.470335  2513 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:35.471408  2658 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.471160  2513 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.461930  2389 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.451645  2544 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.484179 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.485749 26194 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:35.493498 26198 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:35.506520 26094 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.134s (SENT)
I20260709 12:27:35.536420 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.539197 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.472983  2439 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:35.474337  2513 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:35.544224 26094 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.151s (SENT)
I20260709 12:27:35.578693 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:35.559356 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.574766 26791 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:35.577677 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:35.578003 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.581539 26788 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
W20260709 12:27:35.582276 26788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2093) took 3409 ms (client timeout 1673 ms). Trace:
I20260709 12:27:35.580804 26202 raft_consensus.cc:2376] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25. Re-sending same reply.
W20260709 12:27:35.582773 26788 rpcz_store.cc:269] 0709 12:27:32.173116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.175656 (+  2540us) service_pool.cc:224] Handling call
0709 12:27:35.577853 (+3402197us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:27:35.582177 (+  4324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5140224}
I20260709 12:27:35.585642 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.587044 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.588089 26788 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:35.589289 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:35.589922 26199 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.590088 26194 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:27:35.591490 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 189950us
I20260709 12:27:35.592212 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:35.593294 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49108.2us
W20260709 12:27:35.593995 26094 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.159s (SENT)
I20260709 12:27:35.594199 26788 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:35.594419 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.595949 26783 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:35.463635  2531 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.599946 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.544880  2389 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.601503 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.604269 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:35.607048 26785 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:35.609584 26094 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.234s (SENT)
I20260709 12:27:35.610092 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.613047  2658 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.613633  2658 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.615300  2554 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.616415  2471 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.616945  2471 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.619563  2567 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:35.620527  2471 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.621080  2502 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.623594  2502 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:35.624784 26046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2709) took 5101 ms (client timeout 1878 ms). Trace:
W20260709 12:27:35.625213 26046 rpcz_store.cc:269] 0709 12:27:30.523585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.523715 (+   130us) service_pool.cc:224] Handling call
0709 12:27:35.624532 (+5100817us) spinlock_profiling.cc:243] Waited 4.4 s on lock 0x7b5400227c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.624686 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10112470912}
I20260709 12:27:35.625924  2502 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.620229  2567 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.628535  2567 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.622766  2658 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:35.630306 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.631134 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.631791 26783 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:35.633123  2673 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:35.626353 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.638500 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2110) took 1291 ms. Trace:
I20260709 12:27:35.638645 26046 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:35.619555  2530 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:35.635669  2544 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.636530 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.643465 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.638939 26787 rpcz_store.cc:276] 0709 12:27:34.346867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.985964 (+639097us) service_pool.cc:224] Handling call
0709 12:27:34.986369 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.305683 (+319314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.638225 (+332542us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54005c2954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:35.638437 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686949760}
I20260709 12:27:35.645126 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.595846  2498 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:35.646086 26054 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:35.649391 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.663272 26046 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:35.664300 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.664842 26128 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:35.604071  2688 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:35.665954 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.654449  2531 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.654641 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:35.661738 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:35.649657 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:35.674831 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:35.666708 26124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4159) took 3522 ms (client timeout 2146 ms). Trace:
W20260709 12:27:35.675755 26124 rpcz_store.cc:269] 0709 12:27:32.126073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.127087 (+  1014us) service_pool.cc:224] Handling call
0709 12:27:35.648200 (+3521113us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.648381 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605087488}
I20260709 12:27:35.668042 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.616040  2645 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:35.670547 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.681402 26202 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:35.645774  2530 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:35.683311  2555 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.672647 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.701561 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.702288 26054 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:35.703217 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:35.703876 26054 raft_consensus.cc:2368] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29.
I20260709 12:27:35.705466  2701 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.707938  2701 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.664953 26046 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:35.723747 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:35.724475 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.725039 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.725711 26046 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:35.726636 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.727381 26046 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:35.678596 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.680051  2672 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:35.729389  2672 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.729966  2672 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.684070 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:35.684486 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2744) took 1763 ms (client timeout 2094 ms). Trace:
W20260709 12:27:35.732681 26053 rpcz_store.cc:269] 0709 12:27:33.921396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.031625 (+110229us) service_pool.cc:224] Handling call
0709 12:27:35.684318 (+1652693us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400172d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.684427 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2585925504}
I20260709 12:27:35.683247  2530 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.734279  2530 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:35.671801 26128 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 19.
I20260709 12:27:35.736954 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.738125 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.738705 26128 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:35.739665 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.672519  2673 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:35.688793  2499 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:35.690156 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1632) took 1361 ms. Trace:
I20260709 12:27:35.740870  2499 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:35.740993 26123 rpcz_store.cc:276] 0709 12:27:34.328792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.330226 (+  1434us) service_pool.cc:224] Handling call
0709 12:27:35.689944 (+1359718us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54002cb5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.690095 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1744545152}
I20260709 12:27:35.742280 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.742961 26123 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:35.743821 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:35.696991 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:35.744500 26125 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.687636  2461 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.748413  2461 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:35.708508 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.711472 26197 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:35.675050 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.752504 26197 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:35.753049  2673 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:35.753238 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78272us
I20260709 12:27:35.753441 26197 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.754593 26197 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.754758 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.719223 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.741963  2499 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.756186  2499 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:35.758865  2645 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:35.759334  2645 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:35.759760  2645 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:35.752055 26785 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:35.756079 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.763370  2658 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:35.708451  2567 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:35.764731  2567 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.765405  2567 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:35.765267 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:35.766981  2658 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:35.767558  2658 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:35.768087  2658 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:35.772639 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.772408 26123 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:35.751933 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.779096 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.780990 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:35.783123 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:35.784451  2663 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:35.793586 26197 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:35.799335  2567 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.789908 26125 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:35.802731 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.810199 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.810956 26125 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:35.812165 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.812971 26125 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:35.813894 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:35.814585 26125 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:35.816982 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:35.803390  2645 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.806696 26123 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:35.819988 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.822037  2313 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.824270 26124 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:35.828540 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.829468 26054 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:35.830690 26054 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 28. Ops: []
I20260709 12:27:35.831348 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1207) took 1734 ms. Trace:
I20260709 12:27:35.833021 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:35.817471 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:35.831686 26054 rpcz_store.cc:276] 0709 12:27:34.096895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.830145 (+1733250us) service_pool.cc:224] Handling call
0709 12:27:35.830498 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.831100 (+   602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.831292 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.840550 26124 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:32.097098 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:35.845875 26054 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:35.846591 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1209) took 1748 ms. Trace:
I20260709 12:27:35.846908 26054 rpcz_store.cc:276] 0709 12:27:34.098021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.836068 (+1738047us) service_pool.cc:224] Handling call
0709 12:27:35.845544 (+  9476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.846329 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.846533 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:35.847932 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.850696 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.720446  2463 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:35.853775 26054 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:35.854566 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4067) took 1564 ms. Trace:
I20260709 12:27:35.854960 26054 rpcz_store.cc:276] 0709 12:27:34.290640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.853126 (+1562486us) service_pool.cc:224] Handling call
0709 12:27:35.853429 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.854269 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.854465 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.857352 26054 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:35.858076 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4068) took 1567 ms. Trace:
I20260709 12:27:35.858455 26054 rpcz_store.cc:276] 0709 12:27:34.290878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.855893 (+1565015us) service_pool.cc:224] Handling call
0709 12:27:35.856208 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.857843 (+  1635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.858024 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:35.859865 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2733) took 1941 ms (client timeout 1551 ms). Trace:
W20260709 12:27:35.860378 26047 rpcz_store.cc:269] 0709 12:27:33.918321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.918473 (+   152us) service_pool.cc:224] Handling call
0709 12:27:35.859662 (+1941189us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.859778 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100891904}
I20260709 12:27:35.862098 26047 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:35.862689 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4073) took 1570 ms. Trace:
I20260709 12:27:35.856868 26094 leader_election.cc:400] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29.
I20260709 12:27:35.864655 26094 leader_election.cc:403] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:27:35.864105 26047 rpcz_store.cc:276] 0709 12:27:34.292076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.861439 (+1569363us) service_pool.cc:224] Handling call
0709 12:27:35.861777 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.862445 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.862634 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.866214 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.866643 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.867069 26791 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:35.868822 26047 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:35.869498 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4077) took 1564 ms. Trace:
I20260709 12:27:35.869951 26047 rpcz_store.cc:276] 0709 12:27:34.305278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.868341 (+1563063us) service_pool.cc:224] Handling call
0709 12:27:35.868654 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.869259 (+   605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.869443 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.866989  2567 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:35.875646 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4078) took 1567 ms. Trace:
I20260709 12:27:35.876595 26047 rpcz_store.cc:276] 0709 12:27:34.308042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.870860 (+1562818us) service_pool.cc:224] Handling call
0709 12:27:35.871140 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.875229 (+  4089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.875258 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:35.875303 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:35.875352 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.875412 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.875589 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.860376 26054 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:27:35.879829 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.880124 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4072) took 1588 ms. Trace:
I20260709 12:27:35.882536  2313 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.870601  2461 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:35.880542 26054 rpcz_store.cc:276] 0709 12:27:34.291833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.859653 (+1567820us) service_pool.cc:224] Handling call
0709 12:27:35.859978 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.879852 (+ 19874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.880069 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:35.893136 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.867901  2672 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:35.899451 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.896461 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.900640 26786 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 12:27:35.894829 26045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2742) took 1973 ms (client timeout 1895 ms). Trace:
I20260709 12:27:35.901051 26791 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
W20260709 12:27:35.901154 26045 rpcz_store.cc:269] 0709 12:27:33.920934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.999345 (+ 78411us) service_pool.cc:224] Handling call
0709 12:27:35.894535 (+1895190us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54003589d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.894722 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396542848}
I20260709 12:27:35.902186 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:35.906379 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:35.912117 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.909598 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.917603 26045 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 20. Current term is 21. Ops: [16.12-20.14]
I20260709 12:27:35.919252 26785 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:35.919556 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2745) took 1966 ms (client timeout 2134 ms). Trace:
I20260709 12:27:35.917600 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:35.920001 26051 rpcz_store.cc:269] 0709 12:27:33.953692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.033805 (+ 80113us) service_pool.cc:224] Handling call
0709 12:27:35.919372 (+1885567us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b5400131354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:35.919497 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119747456}
I20260709 12:27:35.920666 26127 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.921454 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.899103 26054 pending_rounds.cc:85] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 16
W20260709 12:27:35.922549 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65839.9us
I20260709 12:27:35.922955 26054 pending_rounds.cc:107] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:27:35.921844 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2753) took 1580 ms. Trace:
I20260709 12:27:35.923316 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:35.923573 26045 rpcz_store.cc:276] 0709 12:27:34.341940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.902154 (+1560214us) service_pool.cc:224] Handling call
0709 12:27:35.909219 (+  7065us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:35.921547 (+ 12328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.921786 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:35.924574 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:35.926074  2482 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29.
I20260709 12:27:35.926884  2482 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:35.927286  2482 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:35.927671  2482 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.929970 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:35.923336 26054 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:35.930766 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.934053 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.933432 26123 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:35.934957 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.928478  2401 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:35.935691 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:35.897641  2461 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.891148  2663 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:35.938402 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.941720 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.943157 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.944015 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.949391 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.954267 26791 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.950232 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.951692 26197 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:35.948176 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:35.963332  2471 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:35.982030 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.964843 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:35.966451 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:35.984206 26051 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
W20260709 12:27:35.936517  2455 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.002s
I20260709 12:27:35.981173  2688 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:35.970803 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.985085 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:35.936072  2487 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:36.012373  2645 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.945710  2439 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.015179  2439 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:35.987145 26200 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:35.968251 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:35.993853 26197 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:35.982349  2471 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:36.018525  2471 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:36.019311  2467 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:27:36.019856  2467 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:36.020949 26128 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:36.022861 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:36.025672  2377 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.001s
I20260709 12:27:36.026145  2377 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.026901  2451 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.033015 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:36.033946 26126 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:36.034893 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:36.018087  2551 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:36.036641  2551 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:36.037958 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3177) took 1696 ms (client timeout 2249 ms). Trace:
W20260709 12:27:36.038396 26195 rpcz_store.cc:269] 0709 12:27:34.341473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.794100 (+452627us) service_pool.cc:224] Handling call
0709 12:27:36.037786 (+1243686us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400335c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.037900 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364289920}
I20260709 12:27:36.039089  2377 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:36.023999 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:36.039812 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4070) took 1748 ms. Trace:
I20260709 12:27:36.040252 26048 rpcz_store.cc:276] 0709 12:27:34.291340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.026138 (+734798us) service_pool.cc:224] Handling call
0709 12:27:36.039598 (+1013460us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400173c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.039748 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981635328}
I20260709 12:27:36.041465 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:36.042240 26048 raft_consensus.cc:2368] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28.
I20260709 12:27:36.043321 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:36.040222 26128 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:36.045344 26052 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 31. Ops: []
I20260709 12:27:36.045451 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:36.046209 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:36.047004 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2751) took 2003 ms. Trace:
I20260709 12:27:36.047364 26121 raft_consensus.cc:2410] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:27:36.048602  2699 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.006s
I20260709 12:27:36.044108 26048 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:36.049799 26196 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:35.993069  2498 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.000644 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:36.055919 26051 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:36.001219  2660 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:36.003422 26202 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: [27.21-27.21]
I20260709 12:27:36.006667 26127 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:35.951018  2407 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:36.009053 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:36.010866  2672 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.011430  2401 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:35.951268  2461 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.037600  2551 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.046376 26128 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:35.998407  2455 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:35.999552  2663 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:36.052520 26125 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:36.047387 26052 rpcz_store.cc:276] 0709 12:27:34.043770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.250895 (+1207125us) service_pool.cc:224] Handling call
0709 12:27:36.045151 (+794256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.045879 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.046830 (+   951us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b5400173c50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.046950 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736505728}
I20260709 12:27:36.053931 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:36.036193 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:36.105473 26126 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.105710 26121 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:27:36.067448 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:36.069535 26791 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:36.070619 26122 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:36.055438  2688 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.049026  2699 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:36.094358 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.098374  2567 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:36.099263 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 151188us
I20260709 12:27:36.101914 26203 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 12:27:36.115727 26121 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:27:36.117231 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:36.120486 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 252655us
W20260709 12:27:36.123347 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.125077 26048 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:36.126827 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.070617  2498 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.136487 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:36.137827 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.138873 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:36.117564  2551 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:36.140522 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.070822  2660 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.141980 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.111434  2663 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:36.137137 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.144579 26197 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:36.148528 26203 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:36.148828 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.146431 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:36.151233 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:36.155465 26200 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:27:36.146986 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.144310 26195 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:36.143353 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 193241us
I20260709 12:27:36.162674 26199 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:36.159112 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:36.134377  2389 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.008s
I20260709 12:27:36.168506 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:36.133745  2513 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.005s
I20260709 12:27:36.165091 26197 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:36.162403 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.168116 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.173012 26126 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:36.168519 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:36.176704 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.180368  2530 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:36.167411  2705 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:27:36.180922  2530 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.181391  2530 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:36.181692 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:36.182548  2703 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:36.182971  2703 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:36.183285 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2103) took 1910 ms. Trace:
I20260709 12:27:36.183480  2531 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:36.183995  2531 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:36.168964  2630 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:36.127187  2721 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:27:36.117163  2688 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.164697  2658 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.141170  2401 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.073s
I20260709 12:27:36.135327  2499 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 12:27:36.173096 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.146601  2555 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:36.183670 26784 rpcz_store.cc:276] 0709 12:27:34.273367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.278363 (+  4996us) service_pool.cc:224] Handling call
0709 12:27:36.183104 (+1904741us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54004ffe54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.183229 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651918080}
I20260709 12:27:36.172672  2313 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:36.184180  2482 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.229995  2471 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.231251  2486 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:36.181746 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2827) took 1078 ms. Trace:
W20260709 12:27:36.232703 26049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4081) took 1897 ms (client timeout 2146 ms). Trace:
I20260709 12:27:36.232625 26201 rpcz_store.cc:276] 0709 12:27:35.103227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.207641 (+104414us) service_pool.cc:224] Handling call
0709 12:27:35.243145 (+ 35504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.404121 (+160976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.181523 (+777402us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400301954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:36.181689 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240825600}
W20260709 12:27:36.233121 26049 rpcz_store.cc:269] 0709 12:27:34.335431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.979158 (+643727us) service_pool.cc:224] Handling call
0709 12:27:36.232504 (+1253346us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54002e2554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.232640 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588439808}
I20260709 12:27:36.142592  2451 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.091s
I20260709 12:27:36.236896 26124 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:36.185562 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.185843 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:36.237828 26196 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:36.194200 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:36.199752 26123 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:36.206545 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.244328 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:36.245151 26790 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:36.247392 26195 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:36.248023 26791 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:36.248513 26122 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:36.249531 26128 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:36.249693 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:36.194388  2603 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:36.191461  2530 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.250834  2603 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:36.188961  2690 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.883s	user 0.001s	sys 0.003s
I20260709 12:27:36.251318  2603 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:36.186196  2705 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:36.252949  2530 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.193326 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:36.253799 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 59684.9us
W20260709 12:27:36.253954 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60732.8us
I20260709 12:27:36.254500 26202 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:36.254787 26783 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.256156 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.257413 26203 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 20.
I20260709 12:27:36.188467  2439 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:36.250396 26789 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:36.210278 26194 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:27:36.212514 26120 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:36.217423 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.193885 26051 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:27:36.273409 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4082) took 1794 ms. Trace:
I20260709 12:27:36.273554 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.273820 26051 rpcz_store.cc:276] 0709 12:27:34.478707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.193285 (+1714578us) service_pool.cc:224] Handling call
0709 12:27:36.193582 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.273142 (+ 79560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.273351 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.275178 26051 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:36.275832 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4084) took 1532 ms. Trace:
W20260709 12:27:36.203675 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.276203 26051 rpcz_store.cc:276] 0709 12:27:34.743783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.274669 (+1530886us) service_pool.cc:224] Handling call
0709 12:27:36.275006 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.275589 (+   583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.275780 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.281985 26129 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:36.282300 26051 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:27:36.283344 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4086) took 1538 ms. Trace:
W20260709 12:27:36.186444 26782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2109) took 1879 ms (client timeout 2196 ms). Trace:
W20260709 12:27:36.285282 26782 rpcz_store.cc:269] 0709 12:27:34.307356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.340284 (+ 32928us) service_pool.cc:224] Handling call
0709 12:27:36.186277 (+1845993us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b5400426654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.186382 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890554752}
I20260709 12:27:36.208760 26053 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:27:36.286638 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1205) took 2195 ms. Trace:
I20260709 12:27:36.286998 26053 rpcz_store.cc:276] 0709 12:27:34.091469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.733681 (+1642212us) service_pool.cc:224] Handling call
0709 12:27:36.208147 (+474466us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b5800002798. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:27:36.208411 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.286342 (+ 77931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.286578 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113852288}
I20260709 12:27:36.284516 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.288710 26791 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:36.289746 26051 rpcz_store.cc:276] 0709 12:27:34.744440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.281707 (+1537267us) service_pool.cc:224] Handling call
0709 12:27:36.282057 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.283050 (+   993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.283280 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.220144 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.292913 26199 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.296319 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:36.297453 26128 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:36.264828 26049 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 27. Ops: []
I20260709 12:27:36.321480 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:36.321698 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67368.3us
W20260709 12:27:36.322007 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 104659us
I20260709 12:27:36.322705 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:36.324725 26756 leader_election.cc:400] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28.
I20260709 12:27:36.325065 26756 leader_election.cc:403] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:36.328013 26045 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:36.328781 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4087) took 1584 ms. Trace:
I20260709 12:27:36.329088 26045 rpcz_store.cc:276] 0709 12:27:34.744730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.284033 (+1539303us) service_pool.cc:224] Handling call
0709 12:27:36.284442 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.328529 (+ 44087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.328730 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.330850 26122 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
W20260709 12:27:36.335724 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62254.2us
I20260709 12:27:36.341635 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.342818 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.341805 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.325449 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:36.340389 26053 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:36.344753 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.345048 26045 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:36.345186 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4091) took 1402 ms. Trace:
I20260709 12:27:36.253372  2693 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:36.344764 26194 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:36.345743 26053 rpcz_store.cc:276] 0709 12:27:34.942472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.339848 (+1397376us) service_pool.cc:224] Handling call
0709 12:27:36.340179 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.344944 (+  4765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.345118 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.183321  2703 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:36.352384 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4093) took 1375 ms. Trace:
I20260709 12:27:36.352792 26045 rpcz_store.cc:276] 0709 12:27:34.976842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.344577 (+1367735us) service_pool.cc:224] Handling call
0709 12:27:36.344889 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.348899 (+  4010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.352327 (+  3428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.347512 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4094) took 1363 ms. Trace:
I20260709 12:27:36.354012 26045 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:36.353943 26053 rpcz_store.cc:276] 0709 12:27:34.984403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.346694 (+1362291us) service_pool.cc:224] Handling call
0709 12:27:36.347014 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.347199 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.347223 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.347265 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:36.347305 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.347360 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.347465 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.354651 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4096) took 1363 ms. Trace:
I20260709 12:27:36.355000 26045 rpcz_store.cc:276] 0709 12:27:34.991748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.353536 (+1361788us) service_pool.cc:224] Handling call
0709 12:27:36.353846 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.354415 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.354600 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.355707 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4098) took 1245 ms. Trace:
I20260709 12:27:36.184994  2531 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.215066  2499 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:36.340525 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:36.356179 26053 rpcz_store.cc:276] 0709 12:27:35.110347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.354945 (+1244598us) service_pool.cc:224] Handling call
0709 12:27:36.355246 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.355413 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.355438 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.355478 (+    40us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:36.355507 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.355559 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.355660 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.384279 26053 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:36.198609  2660 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:36.213542  2658 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.193423  2455 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:36.204857  2630 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:36.205524  2664 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:36.214643  2401 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:36.347896 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:36.338276 26789 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:36.361116 26790 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:36.414505 26783 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:36.225400  2487 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:36.330363  2676 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:36.372340 26045 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:36.184453  2701 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.204s
W20260709 12:27:36.432211 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 84403.9us
W20260709 12:27:36.433168 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 265109us
I20260709 12:27:36.389158  2313 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:36.223333  2498 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.210752  2688 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.190234  2567 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:36.346290  2669 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 12:27:36.325558  2513 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:36.321189  2389 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:36.409623 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.476395 26787 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:36.323091  2720 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:36.411301 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.371559 26199 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:36.345849  2693 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:36.346457  2439 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:36.351395  2555 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:36.418738  2324 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:27:36.433957 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.388185  2407 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:36.433070 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:36.449693 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4083) took 1726 ms. Trace:
I20260709 12:27:36.452642 26126 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:36.431124 26045 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:36.386423 26053 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:36.420737  2630 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.410436  2699 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:36.341303  2717 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.093s
I20260709 12:27:36.429780  2486 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28.
I20260709 12:27:36.196949  2663 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.421322  2471 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.426896  2705 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:36.419927  2401 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:36.429401  2377 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:36.478432 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.481346  2389 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:36.453881  2673 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:36.440763  2313 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:36.465350  2567 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:36.464166  2701 consensus_queue.cc:1243] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:27:36.608953 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 199921us
I20260709 12:27:36.475621  2513 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.471247  2530 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:36.518545 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.537226 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 103339us
I20260709 12:27:36.627494 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.542204 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2754) took 2200 ms. Trace:
I20260709 12:27:36.628479 26048 rpcz_store.cc:276] 0709 12:27:34.342233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.151857 (+1809624us) service_pool.cc:224] Handling call
0709 12:27:36.187128 (+ 35271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.187366 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.187391 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.187934 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.426151 (+238217us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:36.426202 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.426302 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.541930 (+115628us) raft_consensus.cc:1685] finished
0709 12:27:36.541955 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.542127 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":237866,"threads_started":1,"wal-append.queue_time_us":292611}
I20260709 12:27:36.548766  2693 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.550271  2439 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:36.552716  2724 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:27:36.557845  2603 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:36.564648 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.637399 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.563445  2630 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.567210  2461 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.568317 26049 rpcz_store.cc:276] 0709 12:27:34.722915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.233931 (+1511016us) service_pool.cc:224] Handling call
0709 12:27:36.239423 (+  5492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.321288 (+ 81865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.449356 (+128068us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54003598d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:36.449609 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57881472}
I20260709 12:27:36.607141  2313 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:36.569777 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 136786us
I20260709 12:27:36.646649 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:36.572322 26045 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:36.482457  2555 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:36.572714 26052 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:36.575385 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2752) took 2234 ms. Trace:
I20260709 12:27:36.609608  2673 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:36.602092 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 271899us
W20260709 12:27:36.679831 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 339360us
W20260709 12:27:36.602461 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.680809 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:36.346560  2721 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:36.683421 26784 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:36.603179 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 278572us
I20260709 12:27:36.604471 26125 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:36.605062 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:36.605538 26194 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:36.585018  2664 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:36.681414 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.374719 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.613476 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:36.618891 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:36.619545 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:36.620139 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:36.620816 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.619796 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:36.622879 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:36.481505  2720 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.644853  2451 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.691830 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.644721 26049 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:36.569432  2498 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:36.692781  2451 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.693208  2720 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.693445  2498 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.649925 26045 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:27:36.648208 26093 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:36.651007 26048 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 20. Current term is 22. Ops: []
I20260709 12:27:36.378943 26196 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:36.687781 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.688346 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.695930 26782 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:36.688467 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:36.697012 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.697157 26789 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27. Re-sending same reply.
I20260709 12:27:36.697791 26782 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:36.645143  2487 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.692993 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.631608  2513 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:36.698201 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.683773  2721 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:36.698865 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:36.698990 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50888.1us
I20260709 12:27:36.688879 26198 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 20: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:36.709015 26782 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:36.668691  2555 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.690577 26127 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:27:36.668318  2471 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:36.694674  2653 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:36.690871 26199 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:36.688135 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.688357 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:36.689167 26195 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:36.705090 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.631361  2717 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:27:36.630566  2680 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:27:36.707736 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.688814 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:36.691222  2676 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.700464 26788 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:36.689190  2658 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:36.693912  2498 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:36.693845  2720 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:36.705935  2324 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:36.705890  2660 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:36.689442  2530 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:36.689551  2377 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.690141  2664 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:36.700557  2727 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:36.713162  2721 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:36.720993  2693 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:36.716943  2603 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:36.682940  2401 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:36.682648 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 55230us
I20260709 12:27:36.930413 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:36.684167 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:36.936565  2461 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:36.937391 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 253097us
I20260709 12:27:36.941144  2531 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:36.942764  2531 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:27:36.866508 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 491697us
I20260709 12:27:36.689551 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4092) took 1736 ms. Trace:
I20260709 12:27:36.666667 26054 rpcz_store.cc:276] 0709 12:27:34.341287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.898531 (+1557244us) service_pool.cc:224] Handling call
0709 12:27:35.898867 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.937570 (+ 38703us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:35.937614 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.962972 (+ 25358us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.260031 (+297059us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:36.260102 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.260210 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.575163 (+314953us) raft_consensus.cc:1685] finished
0709 12:27:36.575183 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.575320 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":262060,"threads_started":1,"wal-append.queue_time_us":510325}
W20260709 12:27:36.871147 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 157714us
I20260709 12:27:36.709606 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:36.946863 26051 rpcz_store.cc:276] 0709 12:27:34.953200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.340749 (+1387549us) service_pool.cc:224] Handling call
0709 12:27:36.380868 (+ 40119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.579305 (+198437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.579338 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.580484 (+  1146us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.687221 (+106737us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:36.687254 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.687331 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.689232 (+  1901us) raft_consensus.cc:1685] finished
0709 12:27:36.689253 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.689495 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51495,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":10333,"thread_start_us":156120,"threads_started":2,"wal-append.queue_time_us":83585}
W20260709 12:27:36.949280 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 239673us
I20260709 12:27:36.951118 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:36.688989 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75625us
I20260709 12:27:36.871986 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2749) took 2850 ms. Trace:
I20260709 12:27:36.876341 26123 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:36.684686 26126 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.893888 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4180) took 1440 ms. Trace:
I20260709 12:27:36.966858 26127 rpcz_store.cc:276] 0709 12:27:35.453696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.061469 (+607773us) service_pool.cc:224] Handling call
0709 12:27:36.061791 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.713512 (+651721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.892988 (+179476us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54001f6dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:36.893609 (+   621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445467904}
I20260709 12:27:36.697252  2389 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.806084  2471 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:36.972536  2471 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.811887  2498 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.976886 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:36.743803  2451 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:36.814857  2513 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.979523  2498 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.981781  2513 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:36.808911  2555 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.794797  2486 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:36.988325  2486 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:36.988911  2486 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.992765 26053 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:36.993888 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:36.994419 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1218) took 1051 ms. Trace:
I20260709 12:27:36.994835 26053 rpcz_store.cc:276] 0709 12:27:35.943551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.575545 (+631994us) service_pool.cc:224] Handling call
0709 12:27:36.575844 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.993357 (+417513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.994102 (+   745us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400173254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:36.994347 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939819904}
I20260709 12:27:36.996376  2720 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:36.996824  2461 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:36.997202  2720 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:36.784870  2530 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.769361  2664 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.735141  2676 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:36.900856 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 189400us
I20260709 12:27:36.765309  2482 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:36.710134  2487 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.773250  2603 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.016s
I20260709 12:27:36.793284  2630 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.000182  2401 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.781985  2653 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:37.013859  2653 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:37.014483 25947 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 195) took 28141 ms (client timeout 15000 ms). Trace:
I20260709 12:27:36.745482  2660 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:37.001040 26791 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:36.983155  2531 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:37.017951  2660 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.742026  2673 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.019795  2531 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.735615  2688 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.795661  2703 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.751932  2727 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:37.077849  2727 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.716827  2731 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.040s
I20260709 12:27:36.790912  2619 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:36.799757  2567 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.804553  2721 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:36.904552  2734 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:36.902303  2740 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:27:36.910116  2693 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:36.925757 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1486) took 1009 ms. Trace:
I20260709 12:27:37.134533 26783 rpcz_store.cc:276] 0709 12:27:35.917118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.984744 (+ 67626us) service_pool.cc:224] Handling call
0709 12:27:36.925185 (+940441us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54004b0354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.925468 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600095488}
I20260709 12:27:37.135368 26128 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:36.955719 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:37.136371  2567 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:37.136845 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181567us
I20260709 12:27:37.137383  2567 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:37.138068  2567 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:36.958577 26050 rpcz_store.cc:276] 0709 12:27:34.021714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.126693 (+1104979us) service_pool.cc:224] Handling call
0709 12:27:35.919789 (+793096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.920032 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.920060 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.937807 (+ 17747us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.118213 (+180406us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:36.356703 (+238490us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.357155 (+   452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.357232 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.867840 (+510608us) raft_consensus.cc:1685] finished
0709 12:27:36.867924 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.871520 (+  3596us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b5400131350. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:36.871717 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":483926,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1798,"raft.queue_time_us":7878,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":9547,"spinlock_wait_cycles":1744926592,"thread_start_us":323443,"threads_started":2,"wal-append.queue_time_us":379582}
I20260709 12:27:36.987459  2555 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:36.997411  2461 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.141304  2461 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:37.143317  2721 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:36.759712  2728 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:27:37.012991 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 83053.8us
I20260709 12:27:37.008829 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.012076  2487 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:37.037591 25946 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.010197 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:37.041404  2688 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.042587  2513 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:27:37.044404  2603 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:37.152868  2734 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.153595  2603 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.153954 26050 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:37.154904  2603 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.050319  2482 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.051229  2717 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.052429  2498 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:37.054042  2672 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:37.167290  2672 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:37.167758  2672 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:37.026856  2720 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.169106 26051 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:37.001470  2664 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:37.074586 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 369431us
I20260709 12:27:37.173233 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.174960 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.075441 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.176004 26791 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:37.176218  2676 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.178123 26127 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:37.178313 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4181) took 1343 ms. Trace:
I20260709 12:27:36.720922  2730 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.043s
I20260709 12:27:37.178841 26121 rpcz_store.cc:276] 0709 12:27:35.835064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.131372 (+296308us) service_pool.cc:224] Handling call
0709 12:27:36.131809 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.132501 (+   692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.132545 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.161513 (+ 28968us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.449514 (+288001us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:36.449557 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.449626 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.177808 (+728182us) raft_consensus.cc:1685] finished
0709 12:27:37.177860 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.178079 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8473,"thread_start_us":262710,"threads_started":1,"wal-append.queue_time_us":436026}
I20260709 12:27:37.180716  2513 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:37.181313  2513 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.182355  2513 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.078581  2486 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:37.184432 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.184572  2732 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:37.081907  2744 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:27:37.060355  2733 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.152933 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:37.185652  2733 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.186394 26054 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.090155 26126 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:37.186605  2486 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.188251  2733 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.187120 26784 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:37.188303 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.139678  2567 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.094614  2703 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:37.014972 25947 rpcz_store.cc:269] 0709 12:27:08.896680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.625460 (+6728780us) service_pool.cc:224] Handling call
0709 12:27:37.014165 (+21388705us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:26.839853 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:26.886033 (+ 46180us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:26.886099 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:27:26.886136 (+    37us) tablet.cc:662] Decoding operations
0709 12:27:26.929818 (+ 43682us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:26.956081 (+ 26263us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:26.958350 (+  2269us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:27:27.323155 (+364805us) tablet.cc:701] Row locks acquired
0709 12:27:27.323231 (+    76us) write_op.cc:260] PREPARE: finished
0709 12:27:27.323486 (+   255us) write_op.cc:270] Start()
0709 12:27:27.323669 (+   183us) write_op.cc:276] Timestamp: P: 1783600047323396 usec, L: 0
0709 12:27:27.323710 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:27:27.324708 (+   998us) log.cc:844] Serialized 8256 byte log entry
0709 12:27:28.006315 (+681607us) op_driver.cc:464] REPLICATION: finished
0709 12:27:28.768659 (+762344us) write_op.cc:301] APPLY: starting
0709 12:27:28.768759 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:29.365468 (+596709us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:29.365600 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:29.898145 (+532545us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:29.898613 (+   468us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 12:27:29.898654 (+    41us) write_op.cc:312] APPLY: finished
0709 12:27:29.899111 (+   457us) log.cc:844] Serialized 188 byte log entry
0709 12:27:30.059109 (+159998us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:30.059308 (+   199us) write_op.cc:454] Released schema lock
0709 12:27:30.059386 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:30.123140 (+ 63754us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:30.123173 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3046716,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":754052,"apply.run_cpu_time_us":55684,"apply.run_wall_time_us":1307758,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92033,"num_ops":21,"prepare.queue_time_us":173118,"prepare.run_cpu_time_us":11005,"prepare.run_wall_time_us":522316,"replication_time_us":650002,"thread_start_us":443793,"threads_started":3,"wal-append.queue_time_us":185866,"wal-append.run_cpu_time_us":1674,"wal-append.run_wall_time_us":1099287}]]}
I20260709 12:27:37.190598  2658 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:37.191574  2676 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.191741 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.098382 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.100911  2439 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.123276  2749 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 12:27:37.124531 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:36.945344  2663 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:37.131508 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:37.149004  2407 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.576s	user 0.013s	sys 0.009s
W20260709 12:27:37.149231 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140652us
I20260709 12:27:37.149859 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:37.148886 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 138994us
I20260709 12:27:37.150660 25946 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:37.151556 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.152249 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.152630 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.159009  2693 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:37.160490  2482 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:37.161437 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.161867  2717 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.164016  2498 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.164551 26048 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:37.164825  2555 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:37.166473  2688 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:37.092686 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 98995.7us
I20260709 12:27:37.236932  2653 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.192348  2676 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.238479  2663 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.240413  2676 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:37.218914  2733 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.241147  2733 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.242158  2498 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.242895 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:37.243593  2733 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:37.193246 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.244153 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.186745  2744 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:37.246978  2744 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.245414 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.172116 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:37.195626 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50572us
I20260709 12:27:37.096715  2678 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:27:37.224861 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.248162 26051 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.228325  2439 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.230787  2482 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.215082  2693 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.258083  2732 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:37.258221  2693 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.258548  2732 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.261421  2731 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:37.233523  2746 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:37.262290  2746 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.262390  2731 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:37.262784  2731 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.263430  2746 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.232923 25946 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 12 to 24, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 24 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.261431  2703 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 12:27:37.265257  2746 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:37.265395  2703 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:37.265822  2703 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.263785  2731 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.267807  2731 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:37.221130  2407 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:37.269450 26202 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:37.198655  2734 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.234838  2730 raft_consensus.cc:2764] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election decision vote started in defunct term 28: won
I20260709 12:27:37.235175  2619 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:37.272401 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.274750  2717 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.275398  2555 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:37.238034 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:37.276306  2439 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.276623  2555 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.277168 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:37.278844  2487 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:37.279284  2487 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.279682  2487 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.172042  2603 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:37.193219  2672 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.226419 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.251614 26791 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:37.321098 26052 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:37.254185 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:37.231297 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:37.231884  2313 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.004s
W20260709 12:27:37.256647 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.257496  2663 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.259713  2732 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.248195  2749 consensus_queue.cc:1243] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:37.280503 26129 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:37.254777  2498 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:27:37.297452  2676 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:37.320984 26054 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:37.295063  2721 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:37.341501 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.338413 26093 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:37.343708 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.325752  2740 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:37.340963 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.336794  2733 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:37.325950  2439 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.358464  2486 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:37.358994  2486 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:37.360296 26167 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.917s (SENT)
I20260709 12:27:37.360896 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.344759  2313 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.363365  2313 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.364316 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:37.365000 26199 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
W20260709 12:27:37.365629 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 125226us
I20260709 12:27:37.366463 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.367452 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.368502  2660 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:37.368907 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 137747us
I20260709 12:27:37.369261  2660 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.369850 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.371627  2660 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.371963  2678 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.354067  2733 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.372687  2658 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.373467  2658 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.375211  2672 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:37.375759  2658 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.376725 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:37.377113  2721 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:37.377615  2721 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.355957 26048 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:37.377450  2744 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.379011  2744 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:37.378676  2721 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.377637 26202 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:37.372740  2733 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.384565  2313 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:37.385262  2313 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:37.386451  2313 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.388557  2653 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.384615 25946 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 15 to 18, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 18 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:37.390353  2439 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.391254 25946 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 12 to 21, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 21 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.354672  2401 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:27:37.392778  2439 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:37.393482  2467 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:37.357097  2663 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:27:37.394170  2467 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.394639 26052 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [25.15-28.17]
I20260709 12:27:37.395121  2467 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:37.395834 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58571us
I20260709 12:27:37.396278  2740 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.360081  2486 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.397549  2703 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.398386  2486 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:37.398572  2673 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:37.399283  2703 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:37.376941  2732 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:37.410383 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56670.3us
I20260709 12:27:37.422106 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.338205  2720 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:37.423878 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.400456  2721 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:37.423242 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:37.419708 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 61725.8us
I20260709 12:27:37.406165  2467 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:37.399319  2673 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:37.428313 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:37.416594  2734 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:37.470897 26195 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:37.474553 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55375us
I20260709 12:27:37.424934  2653 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.431183  2749 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:37.479511 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.433059  2658 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:37.483166 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 111145us
I20260709 12:27:37.483438 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:37.480075 26783 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:37.487175 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 119904us
I20260709 12:27:37.433663  2487 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:37.480926  2377 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.588s	user 0.004s	sys 0.014s
I20260709 12:27:37.473042  2663 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:27:37.500748  2653 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.503240  2653 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.506878  2678 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:37.508095  2717 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:37.508582  2717 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:37.509006  2324 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:37.509533  2717 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.510509  2377 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:37.496172  2658 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.514747  2658 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:37.515900 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.497061  2451 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:27:37.397541  2740 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.493060 25946 catalog_manager.cc:5697] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 13 to 25, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.499410 26046 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:37.497068  2499 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
W20260709 12:27:37.408006  2603 consensus_queue.cc:1175] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a981c9a7972c4e99a0903924c64fbb5c" responder_term: 25 status { last_received { term: 24 index: 17 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:37.497553  2673 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.522089 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:37.522905 26051 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:37.523315 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 238623us
I20260709 12:27:37.485383  2693 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.526506  2720 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:37.530867  2749 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:37.549608  2749 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.549995  2693 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.496299  2619 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.551970  2619 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.552637 26049 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:27:37.554410 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.555115 26202 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.534322  2487 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:37.535091 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 62582.6us
I20260709 12:27:37.516587  2699 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:37.557610  2619 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.558084  2699 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.558574  2699 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:37.560361 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.546509  2499 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:37.561007 26784 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:27:37.563937  2705 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.564594  2750 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:37.557699 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:37.563825  2762 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:27:37.568202 26020 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.570502  2487 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:27:37.570906  2756 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:27:37.571635  2482 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:37.571887 26783 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:27:37.572503 25946 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 9 to 17, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 17 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:37.574330 26195 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:37.575794  2461 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.566112  2717 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.608886  2676 consensus_queue.cc:1059] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:37.610293  2676 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:37.580770  2705 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.610560  2407 raft_consensus.cc:3055] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:37.613212 26790 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:37.613634  2407 raft_consensus.cc:740] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.614868  2407 consensus_queue.cc:260] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.561601  2730 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.075s
I20260709 12:27:37.612648 25946 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 14 to 23. New cstate: current_term: 23 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.566790  2499 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.568715  2750 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.625223 26202 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:37.627069  2407 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.630239  2619 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:37.630692  2688 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:37.634435 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.619846  2467 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:37.653543  2688 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.655503 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2769) took 1993 ms. Trace:
I20260709 12:27:37.655604  2750 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.658906 26047 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:27:37.661126 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:37.667476  2720 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:37.668804  2499 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:37.672000 26167 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.207s (SENT)
I20260709 12:27:37.672936 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.672082 25946 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 15 to 25, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.680958 26123 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
W20260709 12:27:37.690835 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 152388us
W20260709 12:27:37.683895 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 247017us
I20260709 12:27:37.692054 26047 pending_rounds.cc:85] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 12
I20260709 12:27:37.703529 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:37.698309 26046 rpcz_store.cc:276] 0709 12:27:35.662507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.728065 (+ 65558us) service_pool.cc:224] Handling call
0709 12:27:37.655249 (+1927184us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540022bd54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:37.655420 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3822787968}
I20260709 12:27:37.673705  2467 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.700184 26047 pending_rounds.cc:107] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:27:37.721583 26790 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:37.722247 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:37.723694 26047 raft_consensus.cc:2889] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:37.723073 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 115497us
I20260709 12:27:37.724201 26047 pending_rounds.cc:107] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:27:37.724560 26047 raft_consensus.cc:2889] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:37.691428  2720 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.732728 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.737567 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.734894 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.741140 25946 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 14 to 24. New cstate: current_term: 24 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:37.756067 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 84134.3us
I20260709 12:27:37.734964  2487 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:37.758888 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.707731  2733 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.764607 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.743486  2619 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:37.743541  2324 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:37.738585  2663 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:37.737120  2658 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.028s
W20260709 12:27:37.800844 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.756856  2313 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:37.743719  2498 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.028s
W20260709 12:27:37.802752 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.805553  2663 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:37.807652 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.808820 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.809100 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.809551 26788 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:37.809784 26783 raft_consensus.cc:2376] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22. Re-sending same reply.
I20260709 12:27:37.747479  2672 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.768381  2413 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:37.778007 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.778373 26195 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:37.778439 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45797.8us
I20260709 12:27:37.781268 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:37.773942 25946 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:37.786125  2461 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.786715  2567 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.788379 26053 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:37.788750  2499 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:37.789271 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 116452us
I20260709 12:27:37.789366  2733 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.790899 26167 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:37.791550  2487 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.792039  2750 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.792430  2720 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.793277  2688 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.793385 26196 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:27:37.794018 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:37.794814  2324 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.789392  2619 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:37.847652  2324 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.848867  2720 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:37.849161  2619 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.797842  2705 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.802317 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.768944  2673 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
W20260709 12:27:37.853529 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51318.8us
I20260709 12:27:37.812412 26199 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:37.843838 26196 pending_rounds.cc:85] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 16
I20260709 12:27:37.854725 26196 pending_rounds.cc:107] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:27:37.855033 26196 raft_consensus.cc:2889] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:37.855090 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.855963  2451 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:37.856833  2451 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:37.857590  2451 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:37.857963 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 67214.3us
I20260709 12:27:37.826452  2413 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.858917  2413 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.859026 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:37.860265  2487 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.862886  2487 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:37.863481  2487 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.864200 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:37.814954 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.856549  2769 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:37.876224 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.846541 26202 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.851977  2467 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.887328 26195 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:37.827764  2732 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:37.896799 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.898770 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4080) took 3574 ms. Trace:
I20260709 12:27:37.899312 26047 rpcz_store.cc:276] 0709 12:27:34.324323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.877581 (+1553258us) service_pool.cc:224] Handling call
0709 12:27:37.658283 (+1780702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.755341 (+ 97058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.898433 (+143092us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b540022bd50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:37.898692 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4094301568}
I20260709 12:27:37.857990  2451 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.907315 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.908457 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.909211 26784 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:37.896773 25946 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 9 to 20, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:27:37.917776 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:37.877149  2672 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.911342  2451 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.912809  2721 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:37.925072 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.906389  2673 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:37.928287 26757 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:37.930624 25946 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 13 to 24, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 24 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.843783  2749 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.932778 26121 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:37.888288  2567 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.938505 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.939626  2663 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.927817  2467 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:37.925010  2732 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.941370  2749 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.942847 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:37.943235  2732 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.943852 26790 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:37.944636 26789 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 21. (term mismatch)
I20260709 12:27:37.945004 26789 pending_rounds.cc:85] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 20
I20260709 12:27:37.944955  2775 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:37.945539 26789 pending_rounds.cc:107] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 27.21
I20260709 12:27:37.945740  2775 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.945948 26789 raft_consensus.cc:2889] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:37.946327  2775 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.943573 26126 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:37.949010 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:37.950260 26785 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.944815 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:37.953351 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50101.6us
I20260709 12:27:37.953459 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:37.953931  2731 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:37.954433 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.932124  2705 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:37.956954 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 58716.7us
I20260709 12:27:37.964085 26783 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:37.966796 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:37.968555  2731 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.956595  2733 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:37.969762  2731 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.972373  2401 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:37.972132  2673 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:37.972409  2769 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:37.972831  2401 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.973277  2401 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:37.974560  2673 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.976049 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:37.976677 26791 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.977547  2676 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:37.979045 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.979262 26202 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:37.979815  2407 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:37.929994  2721 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:37.980552 26195 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:37.992019  2721 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:37.993037  2567 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:37.993474  2721 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:37.995893  2672 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:37.997130 26782 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:37.997501 26782 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:37.998492 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:37.999028  2688 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:37.999406  2699 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.000793 26054 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:38.001405 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:38.001546 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:38.001842  2699 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:38.002045 26123 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:38.003804 26196 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:27:38.004948 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.005932 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:38.008477  2746 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:27:38.010286  2746 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:38.010862  2746 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:38.008495 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.922786  2471 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:27:37.956202 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:37.998283 26782 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.033984 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.034284 26788 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
W20260709 12:27:38.044312 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61926us
I20260709 12:27:38.044497 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.049746 26791 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:38.052755 26782 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:38.058625 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:38.065344 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:38.067025 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:38.068068 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:38.072796 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:38.069536 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:38.011855  2746 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.080979 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:38.077962 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 79559.8us
I20260709 12:27:38.047668  2471 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.074748 26196 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
W20260709 12:27:38.069020  2784 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.059s	user 0.004s	sys 0.001s
I20260709 12:27:38.053723  2732 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:38.093959 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:38.093334 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:38.096490 26788 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:38.074051  2688 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:38.100658 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.060094  2749 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:38.107024 26126 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
W20260709 12:27:38.109681 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.081843  2744 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:38.115278 26783 raft_consensus.cc:1240] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:38.115569 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:38.337891 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 240477us
I20260709 12:27:38.086920  2673 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.161724 26126 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:38.169646 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 114156us
W20260709 12:27:38.170360 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.339933 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.117352  2789 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 12:27:38.162405 26790 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:38.093458  2471 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.096946  2678 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:38.097545  2732 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.086738  2699 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:38.119086 26785 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:38.099709  2730 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.064s
I20260709 12:27:38.108517  2688 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.217586 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:38.219902 26124 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:38.177608  2795 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:38.166373  2744 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:38.230068 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:38.231745 26784 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:38.351796 26129 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:38.352126 26791 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:27:38.212458  2733 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.234807  2672 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.236490 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:38.172943  2717 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.246215 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:38.242120  2413 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.247815  2401 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.249099 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:38.256031  2793 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:38.263149  2705 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:38.263816  2407 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.380422  2740 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:38.380589  2407 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:38.265286  2783 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.290s	user 0.002s	sys 0.000s
W20260709 12:27:38.266718  2787 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 12:27:38.269363 26786 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:27:38.382769  2407 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:38.271555 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:38.385576 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 114129us
W20260709 12:27:38.271759  2788 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.275s	user 0.000s	sys 0.002s
W20260709 12:27:38.280853 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 324729us
I20260709 12:27:38.276470  2786 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:38.388105 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:38.390733 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:38.335580  2794 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.392288  2794 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:38.393534  2487 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 12:27:38.393987  2487 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:38.394796  2794 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:38.338124 26123 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:38.394961  2705 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.397424  2705 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:38.396998  2744 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:38.232849 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.180353  2451 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:38.357658 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.344612  2750 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:38.345774  2732 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:38.361091 26122 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:38.364661 26120 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:38.363924 26125 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:38.374748  2401 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:38.376875  2413 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:38.378791 26121 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:38.358080  2486 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:27:38.360925  2672 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.385651  2663 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:38.291285 26782 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:27:38.418105 26782 pending_rounds.cc:85] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 18
I20260709 12:27:38.418351 26782 pending_rounds.cc:107] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:27:38.418576 26782 raft_consensus.cc:2889] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:38.391126 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.391372  2619 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:38.341375 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 369438us
I20260709 12:27:38.233214  2769 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:27:38.399264 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.406419 26129 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:38.409802 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:38.420463 26123 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:38.412191 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:38.421053 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81603.4us
I20260709 12:27:38.421300 26791 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:38.362264  2678 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:38.430409 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:38.354041  2673 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.421828 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:38.431466 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:38.408370  2699 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:38.464263 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.433825 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.364110  2733 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.502422 26124 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 30 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
W20260709 12:27:38.432026 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68952.5us
I20260709 12:27:38.434132 26757 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:38.504839 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.366017  2717 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.505859  2750 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:38.427645  2663 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:38.533093 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 99063.2us
I20260709 12:27:38.409569  2775 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.536085 26128 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:38.492086  2749 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.417697  2794 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.100s
I20260709 12:27:38.539814 26757 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.432619  2720 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.108s
W20260709 12:27:38.550998 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 334517us
I20260709 12:27:38.574472 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:38.540575  2487 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.572024 26123 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:38.584295 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:38.540333  2731 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:38.600068 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60366.3us
I20260709 12:27:38.542955  2663 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:38.625221 26199 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:27:38.640621 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:38.641541 26198 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:38.646221  2672 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:38.582657  2699 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.544724  2750 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.588979  2720 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 29.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:27:38.621356  2703 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.639869  2688 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:38.686287 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.695068  2781 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.695812  2746 raft_consensus.cc:2764] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:27:38.564666  2749 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.706856 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:38.711755 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 310093us
I20260709 12:27:38.716799 26199 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:38.623337  2487 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.740731 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.741701 26169 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.742471 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.743224 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.696650  2799 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:38.696193  2733 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:38.757807 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 360121us
W20260709 12:27:38.766587 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 188905us
I20260709 12:27:38.767073 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:38.774968 26198 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:38.780311 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:38.758432  2499 raft_consensus.cc:2764] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:38.740377  2673 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:38.760224  2471 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:38.781278 26199 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
W20260709 12:27:38.785571 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.758293  2407 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:38.700706  2699 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.708415  2781 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.767529  2721 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:38.715926  2703 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.773602  2676 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:38.773422  2795 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:38.685815  2678 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.795818 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:38.800087 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:38.800361 26202 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:38.804169 26791 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:38.789527  2733 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:38.791335  2407 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.821885 26199 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 12:27:38.824885 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 331551us
I20260709 12:27:38.827359 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:38.829083 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 407323us
I20260709 12:27:38.802695  2795 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:38.829600 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.830783 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:38.811296  2733 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.807463  2721 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:38.811678  2676 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:38.850983 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:38.851192 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 72713us
I20260709 12:27:38.807866  2619 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:38.827184  2806 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:27:38.813447  2407 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:38.840572 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.866861 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:38.871299 26199 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:38.853077  2603 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:38.837067  2814 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:38.889518 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 103215us
I20260709 12:27:38.837152  2795 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:38.812136  2471 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:27:38.862486  2733 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:38.892223 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:38.892275 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:38.870105  2451 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:38.866600  2813 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.898794 26195 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:38.828558  2818 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:27:38.905066 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:38.905741 26198 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:38.886521  2731 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:38.897459  2804 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:27:38.888384  2603 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:38.918027 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:38.896400  2749 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:38.952081 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.932513 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.922868  2805 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:27:38.922365  2451 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:38.914608  2750 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:38.971596 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:38.972707 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.970075 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 90022.2us
I20260709 12:27:38.974979  2811 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:38.978758 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 90240us
I20260709 12:27:38.922580  2814 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:38.985241 26195 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:38.953996  2821 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:27:38.989872  2808 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:27:38.922040  2731 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.009662 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:39.014617 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.012050  2818 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:39.021986  2731 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:39.026104  2820 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:39.027884  2820 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:38.976159  2699 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:38.972750  2487 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:38.972954  2717 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.033428  2814 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:39.028569  2812 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:27:39.029531 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:39.036762 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:39.037606  2451 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:39.030109 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.032435  2413 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:39.039464  2413 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:39.039569 26199 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:39.040473  2413 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.041752  2413 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:39.042145  2401 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.042737  2401 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.044687 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:39.045545 26123 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:39.046429 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:38.955404  2721 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.048079 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:39.048893 26128 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
W20260709 12:27:39.049096  2824 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:27:39.050854  2487 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.048333  2720 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:39.052170  2720 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:39.052626  2781 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:39.053294  2720 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.047019  2451 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:39.056422 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:39.057140 26122 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:27:39.059018 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.917845  2813 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.060873 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:39.065135 26199 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:39.062299  2750 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:27:39.067610 26120 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:27:38.984760  2603 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.051460  2487 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.069021  2603 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:39.069067  2407 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.070952  2487 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:38.997856  2676 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.071614  2407 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:39.072394  2834 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:39.072829  2834 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:38.998142  2811 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.013296  2703 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:39.064154 26122 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:39.074827 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:39.076953 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:39.077369 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:39.087419 26120 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:39.088598 26129 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:39.061421  2821 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:39.079275 26094 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.186s (SENT) [suppressed 41 similar messages]
I20260709 12:27:39.086854 26121 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:39.091858 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.098418 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:39.078454  2732 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
I20260709 12:27:39.119032  2732 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.120317  2821 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.121625  2732 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:39.091868  2699 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:39.076843  2401 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.078769  2819 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:39.122606  2699 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.123211  2819 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:38.968854  2619 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.135388 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.079557  2499 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:27:39.139544  2499 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:39.140064  2499 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:39.141131  2499 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.141831 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2164) took 1108 ms. Trace:
I20260709 12:27:39.142232 26788 rpcz_store.cc:276] 0709 12:27:38.033254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.097355 (+ 64101us) service_pool.cc:224] Handling call
0709 12:27:38.114232 (+ 16877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.545548 (+431316us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:38.804283 (+258735us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.956145 (+151862us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.968368 (+ 12223us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.968574 (+   206us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:38.968604 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.968685 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.141598 (+172913us) raft_consensus.cc:1685] finished
0709 12:27:39.141621 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.141769 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":357262,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":44054,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":10488,"thread_start_us":738589,"threads_started":3,"wal-append.queue_time_us":325691}
I20260709 12:27:39.073765  2834 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.145684  2834 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:39.142529  2499 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.148427  2795 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.155696 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:39.156901 26123 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:27:39.160414 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:39.161571 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.138414 26120 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:39.164157 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:39.164863 26199 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:39.172760 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.182185 26123 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:39.196789 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.200838 26786 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:39.200985 26129 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:39.174660 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:39.215128 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.121626  2813 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.222616 26783 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:39.224926 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.180354  2795 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:39.174078  2819 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:39.239715 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 223796us
I20260709 12:27:39.174580  2832 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:39.234835 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:39.235409 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.225724 26195 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:39.219385  2782 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:39.267622  2782 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.268412  2782 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:39.221972  2820 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:39.239229  2813 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:27:39.274271  2820 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.274158 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:39.281277 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.274493 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.272579 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.281810 26195 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:39.229732  2818 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:39.277082 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.280618 26199 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:39.280640 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.229391  2769 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.280097 26049 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:27:39.287526 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:39.289808 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:39.276203  2721 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.291661 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:39.271922  2471 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.300757  2401 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:39.301256  2401 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:39.271816  2832 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.302250  2401 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.303654  2734 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:39.270462  2819 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:39.276957  2821 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:27:39.299878  2471 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.313019 26202 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:27:39.304349 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.288401  2820 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:39.305471 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:39.279018  2720 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:39.328258 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.317047 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.315321 26198 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:27:39.326135 26046 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:39.350236 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.289062  2699 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.316509 26200 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:27:39.362171 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 56771.8us
W20260709 12:27:39.363763 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 59513.2us
I20260709 12:27:39.363001 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:39.369042 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:39.369323 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:39.369828 26048 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:39.304128  2734 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:39.367234 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:39.371726 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.304524  2842 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.306267  2819 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:39.309185  2471 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.375881 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:39.385056 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 102021us
I20260709 12:27:39.377326 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:39.377488 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.393288 26198 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:39.377555  2818 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.378597 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:39.379245  2486 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:39.365024 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:39.314630  2703 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.304188  2832 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.372550  2407 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:39.397521 26202 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:39.397719 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.361282  2705 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:39.308364  2821 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:27:39.402143  2836 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.112s	user 0.001s	sys 0.001s
I20260709 12:27:39.366678  2672 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.430102  2672 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.374015  2678 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:39.432461  2678 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.432904  2775 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:39.433007  2678 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:39.402069 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:39.433988 26052 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:39.435103 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:39.397373 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:39.437055  2833 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 12:27:39.437425 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.392076 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:39.438177 26046 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:39.393930  2663 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.710s	user 0.005s	sys 0.010s
I20260709 12:27:39.416810 26048 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:39.420037 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 143151us
I20260709 12:27:39.440764 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.440833  2663 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.415266 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:39.306782  2732 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.089s
I20260709 12:27:39.304320  2721 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.399502  2486 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.436909 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:39.490335 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.490444 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53626.9us
W20260709 12:27:39.490993 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.440192  2794 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:39.395462  2810 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.426972 26195 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
W20260709 12:27:39.447047 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.459232 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:39.394218  2819 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.615834 26048 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:39.297603  2769 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.441478 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.497807 26124 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:39.374073  2733 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:39.529464 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.560149 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.563802 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:39.419992  2407 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.625558 26052 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:39.469543  2821 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:39.612463 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.443821  2401 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.559412 26054 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:39.460453 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.623191  2819 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.497800  2486 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:39.649235 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.525420  2749 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:39.443980  2815 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:39.664556 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 349241us
W20260709 12:27:39.666442 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.638556 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:39.646838 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 245601us
I20260709 12:27:39.648785 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:39.525610  2775 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:39.491520 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:39.671224 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 179759us
I20260709 12:27:39.611485  2825 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:39.655527 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 214117us
I20260709 12:27:39.399119  2842 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.659612 26202 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:39.639374  2407 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:39.672164 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.497629  2794 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:39.669598 26196 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:39.676554 26203 raft_consensus.cc:2368] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:39.677299 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:39.627236  2705 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:39.427922  2703 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.671773 26048 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:39.684747 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.443732  2800 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.268s
I20260709 12:27:39.638912  2451 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.682614 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.654867  2834 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.692013 26194 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:39.670956  2775 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.641295  2603 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
W20260709 12:27:39.736213 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.735455 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.717900 26049 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:39.668660  2845 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:39.680465  2487 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:39.671449  2838 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:39.748005 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 65504.7us
I20260709 12:27:39.749286 26019 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:39.755672 26201 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:39.703779  2794 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:39.764879 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.765733 26194 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:39.613520  2734 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.673158  2749 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:39.781502 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.745592  2775 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:39.710772  2744 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.731002 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:39.688373  2810 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.810240 26200 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:27:39.752846  2672 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:39.813931 26203 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:27:39.687477  2825 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.820331 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:39.814774 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.826328 26198 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:39.828251 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.829276 26194 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:27:39.830129  2705 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.770820  2795 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:39.834717  2795 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:39.677480  2815 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.836524  2499 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:39.842334  2830 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:39.843014  2830 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:39.718222  2451 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.770360  2487 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:39.844056  2830 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.845499  2487 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.846221  2451 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:39.847394  2734 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:39.851241  2749 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:39.852828 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:39.853650 26127 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:39.854646  2672 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:39.855607  2672 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:39.855975  2672 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:39.856989  2672 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.858356  2672 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:39.735687  2834 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:39.866097 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.931018 26049 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:39.770555  2845 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:39.936906 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:39.938616  2499 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.938262  2486 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:39.835758  2795 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:39.941322  2851 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:27:39.769933  2848 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:39.947108 26046 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.003s
I20260709 12:27:39.994427 26046 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:39.948333 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 276264us
I20260709 12:27:39.951015 26198 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:39.952787 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.954996 26052 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:27:39.959697 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.968456 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45127.7us
W20260709 12:27:39.957798 26199 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.001s	sys 0.016s
W20260709 12:27:39.997102 26199 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.001s	sys 0.017s
W20260709 12:27:39.997234 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.947789  2451 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.977535 26194 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:39.969952  2733 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:39.786262  2821 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.824671  2744 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:39.998383 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.011327 26046 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [25.15-28.17]
I20260709 12:27:39.972148  2775 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:39.974181  2786 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:27:39.985776  2663 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:40.030395 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.039662 26783 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:40.012058  2451 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.002310  2848 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.049556 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:39.986454  2499 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.011078  2721 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:40.050298 26788 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:40.038055 26791 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:40.021256  2769 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:40.055847 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 378627us
I20260709 12:27:40.049371 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.064811 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.065447  2775 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:40.065840 26791 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:40.066380 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.068287  2451 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:40.068868 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.069705 26789 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24.
I20260709 12:27:40.071452  2499 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:40.047489  2800 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.082836 26127 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:40.051950  2810 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.051080  2848 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.093578 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.104465 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:40.108166 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.111390  2619 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.112021  2619 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.115453  2619 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.117179 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.118111 26045 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:40.086294  2841 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.653s	user 0.005s	sys 0.000s
W20260709 12:27:40.122704 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 64092.9us
I20260709 12:27:40.127122 26788 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:40.128105 26053 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:40.132369 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:40.090454  2855 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.100s	user 0.005s	sys 0.005s
I20260709 12:27:40.136613 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:40.140475  2499 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:40.141119  2499 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:40.142139 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.142105  2499 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.144537  2451 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.145078  2451 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.057121  2407 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.149685 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.157153 26045 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:40.148015 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.102684  2842 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.158399 26197 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:27:40.165280 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.166070 26054 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:40.145967  2851 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:40.169348 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.173005 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.173866 26201 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.174784 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.175372 26201 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:40.102207  2845 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.175899  2848 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.185724 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.186383  2848 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:40.186512 26047 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:40.187649 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:40.188664  2854 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
W20260709 12:27:40.189742  2856 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:27:40.180737 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.188189 26047 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:40.196563 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.197693 26201 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:40.129765  2703 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:40.198004 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.145474  2451 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.232758 26124 raft_consensus.cc:1217] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 17.12->[19.13-21.14]   Dedup: 19.13->[21.14-21.14]
I20260709 12:27:40.233618 26124 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:27:40.164390  2744 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.088579  2800 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:40.201436  2857 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:27:40.237521 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:40.166486  2834 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.191493  2486 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:40.241036 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 212276us
I20260709 12:27:40.222572  2838 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:40.135048  2810 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
W20260709 12:27:40.235250  2852 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
W20260709 12:27:40.242412  2401 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.594s	user 0.006s	sys 0.006s
I20260709 12:27:40.269788 26124 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:40.239418  2732 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:40.284076 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.286324 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181960us
I20260709 12:27:40.310012 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.264267  2860 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.235946  2703 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.045s
I20260709 12:27:40.317675 26124 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:27:40.318738 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:40.322108 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.322971 26123 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.390726 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4128) took 1094 ms. Trace:
W20260709 12:27:40.334677 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.367200 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:40.392496 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.360996 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2178) took 1151 ms. Trace:
I20260709 12:27:40.355016 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.388927 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4124) took 1111 ms. Trace:
I20260709 12:27:40.401189 26053 rpcz_store.cc:276] 0709 12:27:39.278371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.280537 (+  2166us) service_pool.cc:224] Handling call
0709 12:27:39.296614 (+ 16077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.135348 (+838734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.388696 (+253348us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b5400171954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:40.388865 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911648128}
I20260709 12:27:40.362454 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1477) took 1048 ms. Trace:
I20260709 12:27:40.364034 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.404470 26054 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
W20260709 12:27:40.405683 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.392534 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.397534 26049 rpcz_store.cc:276] 0709 12:27:39.296980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.326996 (+ 30016us) service_pool.cc:224] Handling call
0709 12:27:40.390524 (+1063528us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400172d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:40.390656 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791900416}
I20260709 12:27:40.402943 26196 rpcz_store.cc:276] 0709 12:27:39.314114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.316724 (+  2610us) service_pool.cc:224] Handling call
0709 12:27:40.362217 (+1045493us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:40.362375 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718828544}
I20260709 12:27:40.399112 26782 rpcz_store.cc:276] 0709 12:27:39.209416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.210872 (+  1456us) service_pool.cc:224] Handling call
0709 12:27:39.230249 (+ 19377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.265859 (+ 35610us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:39.399261 (+133402us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.772834 (+373573us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.773948 (+  1114us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.774093 (+   145us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:39.774129 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.774217 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.360582 (+586365us) raft_consensus.cc:1685] finished
0709 12:27:40.360612 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.360882 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2873,"prepare.queue_time_us":91627,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":28394,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":38549,"thread_start_us":483899,"threads_started":2,"wal-append.queue_time_us":570666}
I20260709 12:27:40.413146 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:40.415452 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:40.332031 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.411180 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.417650 26123 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.415688 26121 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:40.418778  2794 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.608s	user 0.003s	sys 0.007s
I20260709 12:27:40.420089 26124 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:40.425943 26018 leader_election.cc:400] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24.
I20260709 12:27:40.426448 26018 leader_election.cc:403] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:40.427053 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 105479us
W20260709 12:27:40.439199 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.440568 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:40.360405  2672 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.000s
I20260709 12:27:40.309508  2732 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:40.382798  2782 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.108s	user 0.014s	sys 0.000s
W20260709 12:27:40.387607  2678 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.954s	user 0.011s	sys 0.002s
I20260709 12:27:40.444667 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2864) took 1130 ms. Trace:
I20260709 12:27:40.444517 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2861) took 1109 ms. Trace:
I20260709 12:27:40.452189 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.450836 26197 rpcz_store.cc:276] 0709 12:27:39.311064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.384284 (+ 73220us) service_pool.cc:224] Handling call
0709 12:27:39.391764 (+  7480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.158911 (+767147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.417966 (+259055us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:40.420557 (+  2591us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732532992}
I20260709 12:27:40.450764 26199 rpcz_store.cc:276] 0709 12:27:39.317604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.364966 (+ 47362us) service_pool.cc:224] Handling call
0709 12:27:39.392387 (+ 27421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.392606 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:39.997494 (+604888us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.442386 (+444892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:40.442458 (+    72us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:40.442521 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.442621 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.442768 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28289,"prepare.queue_time_us":595139,"prepare.run_cpu_time_us":1866,"prepare.run_wall_time_us":7314,"raft.queue_time_us":29250,"thread_start_us":999114,"threads_started":2}
I20260709 12:27:40.374794  2815 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:40.373715  2676 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:40.420187  2819 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.001s
I20260709 12:27:40.454293 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.459767 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.462797 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.464876 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.409531  2810 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:40.464072 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.441632  2401 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.471457 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.475374 26168 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:40.479334 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:40.479178 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:40.469014 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.419437  2794 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.341022  2848 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.118s
I20260709 12:27:40.466408 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:40.533495 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.534202 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:40.534685 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.521736 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:40.524204 26049 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:40.528548 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.481812 26201 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:40.529857 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 337333us
I20260709 12:27:40.532307 26123 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:27:40.478833  2699 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:27:40.532599  2827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.000s	sys 0.008s
I20260709 12:27:40.539711 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:40.506220  2820 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.109s	user 0.013s	sys 0.000s
I20260709 12:27:40.536198 26053 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:40.535347 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:40.547710 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 81389.3us
I20260709 12:27:40.547817 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.548833 26199 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
W20260709 12:27:40.522262  2814 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.222s	user 0.010s	sys 0.032s
I20260709 12:27:40.388190  2838 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.548909 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:40.550632 26054 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:40.520040  2487 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:40.559216 26120 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:27:40.543650  2827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.000s	sys 0.009s
I20260709 12:27:40.555640 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.340839  2860 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.553009 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.557818 26202 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.555227 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.566887 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.567148 26048 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.572136 26052 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.459196  2782 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.572964 26054 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:40.567617 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:40.459040  2678 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.501827  2672 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.051s
I20260709 12:27:40.600323 26120 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:40.557029  2769 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.480571  2676 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.608129 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:40.520622  2401 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:40.554114  2814 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.010s	sys 0.033s
I20260709 12:27:40.577466 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.480811  2819 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.584810 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.479158  2810 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.590178 26123 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:40.626947 26048 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:40.591419 26049 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:40.592639 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.618078 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.628659 26121 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:40.622663 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.580291  2830 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:40.553261  2820 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.627703 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.640170 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.641106 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.602512  2887 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.610152  2769 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.644446 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.636152 26045 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22.
I20260709 12:27:40.616367  2721 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:40.567641  2880 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:27:40.653436 26049 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:40.651823 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.667647 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.667407 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.668295 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:40.656958 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.669183 26199 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:40.668470 26045 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.656473 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.668944 26196 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:40.672493 26050 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:40.675805 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.676901 26051 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.679436 26045 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:40.672880 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 327163us
I20260709 12:27:40.682443 26054 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:40.693115 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.693481 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.689560 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:40.694577 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.652570  2862 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.652500  2769 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.708478 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:40.697789 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:40.716212 26199 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:40.716367 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.710462 26047 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:40.714581 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:40.670902  2749 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:40.653894  2869 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:40.717687 26789 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:40.670905  2721 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.724557 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.725668 26054 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:40.726534 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.674707  2800 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.729187 26054 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:40.727684 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4130) took 1401 ms. Trace:
I20260709 12:27:40.730499 26050 rpcz_store.cc:276] 0709 12:27:39.326520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.326762 (+   242us) service_pool.cc:224] Handling call
0709 12:27:40.727425 (+1400663us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:40.727610 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2921670400}
I20260709 12:27:40.734334 26051 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:40.736356 26045 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.712731  2877 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:40.748526 26199 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:40.746666 26054 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:27:40.697093  2487 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:40.747433 26048 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:27:40.743556 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.744182 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.749205 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:40.757236 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.706382  2814 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:40.721413  2811 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:40.762916 26054 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:27:40.643512  2732 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.730834  2721 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:40.671280  2887 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.775465 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.721925  2885 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:27:40.785219 26788 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:40.786916 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.787729 26788 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:40.791786 26121 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:40.723495  2862 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:40.801167 26050 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:40.802304 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:40.804069 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:40.795684 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.808624 26784 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:40.809906 26790 raft_consensus.cc:2410] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:40.796819 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:40.810809 26791 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:40.763579  2814 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:40.729108  2749 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.813882 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:40.814893 26785 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:27:40.800904  2871 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 12:27:40.779914  2750 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.798152  2794 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:40.795162  2818 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.759440  2877 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:40.870689  2451 consensus_meta.cc:287] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.000s
W20260709 12:27:40.895036  2883 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 12:27:40.895692  2896 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:27:40.896116  2874 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 12:27:40.895107  2890 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:27:40.817088  2815 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:40.802888  2897 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.909205 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:40.825618 26121 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:40.824018 26788 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:40.761089  2882 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.074s
I20260709 12:27:40.822778  2782 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:40.818276  2814 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.922184 26791 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:40.927336 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:40.930835  2451 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:40.945281 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.975346 26120 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:40.918262  2750 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:40.946444 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 118225us
I20260709 12:27:40.959101  2811 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.823545  2886 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.114s
I20260709 12:27:40.981478 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:40.957863 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:40.976130 26791 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:40.981204 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:41.001243 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:41.002218 26790 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.001322 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:40.983222 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.003610 26788 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:40.944048  2861 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:41.006371 26194 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.957340  2800 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.007020 26050 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:40.955638  2875 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:40.959697  2401 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:40.955410  2860 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:41.020895 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:41.025799  2895 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.002s
I20260709 12:27:41.026052 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2878) took 1193 ms. Trace:
I20260709 12:27:41.026508 26195 rpcz_store.cc:276] 0709 12:27:39.833020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.177790 (+344770us) service_pool.cc:224] Handling call
0709 12:27:41.020722 (+842932us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b58001a1f98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:27:41.025967 (+  5245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42222976}
I20260709 12:27:41.027611 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.028510 26198 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:41.026072  2908 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:41.030012 26198 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:41.017655 26791 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:41.031204 26052 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:41.023350 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:41.023456 26785 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:40.977421  2820 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.029735  2908 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.022812 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.039251 26053 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25. Re-sending same reply.
W20260709 12:27:41.000958  2883 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:27:41.005853 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:41.041232 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:40.953070  2815 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.979997  2811 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:41.030979 26195 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:41.032009 26121 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:41.045665 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:41.046074 26195 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:41.047497 26789 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
W20260709 12:27:41.048034 26201 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 12:27:41.048436 26201 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:40.985740  2819 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.046433 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:41.053589 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.054306 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:41.056388 26198 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:41.058501  2825 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:41.059584  2888 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 12:27:40.932173  2818 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.000586  2889 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.186s
I20260709 12:27:41.068753 26790 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:40.915436  2487 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.008186  2782 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:27:40.961756  2897 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.030982  2800 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.005652  2861 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.193257 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:41.201292 26094 leader_election.cc:336] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.931s (SENT)
I20260709 12:27:41.201918 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:41.202096 26125 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:41.061486  2860 raft_consensus.cc:2764] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:41.142127 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1225) took 1025 ms. Trace:
I20260709 12:27:41.204412 26046 rpcz_store.cc:276] 0709 12:27:40.117002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.117123 (+   121us) service_pool.cc:224] Handling call
0709 12:27:40.117505 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.118001 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:40.145750 (+ 27749us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.453216 (+307466us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.468534 (+ 15318us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.468762 (+   228us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:40.468811 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.468906 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.141892 (+672986us) raft_consensus.cc:1685] finished
0709 12:27:41.141915 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.142059 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16325,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":319,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":12890,"thread_start_us":275502,"threads_started":1,"wal-append.queue_time_us":390037}
I20260709 12:27:41.207705 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.042551  2886 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:41.217011 26788 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:41.123108 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.085421 26018 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:41.105588 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:41.218499 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 133155us
W20260709 12:27:41.227104 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.182545 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:41.166643 26123 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:41.233390 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:41.233731 26122 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:41.208957 26046 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:41.136915 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.104187 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.192379 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.128845 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.116312 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.165267 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.055706  2885 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:41.049748  2815 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:41.235761 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.236929 26045 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:41.238199 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.238905 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.208678  2451 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.258962  2911 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.141934  2749 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.259543  2911 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.050827  2811 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.085606  2862 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:41.267612 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66351.9us
W20260709 12:27:41.269825 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.061201  2603 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:41.240154 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.277141 26052 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:41.223345  2901 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:41.122560  2879 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.244s	user 0.003s	sys 0.004s
I20260709 12:27:41.217249  2886 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.106761  2820 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.148s
W20260709 12:27:41.248176  2905 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:27:41.256670  2499 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:41.256990  2818 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.257319 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.257593  2786 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:41.257491 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:41.293522 26790 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:41.258515 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.260548 26093 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:41.194538  2795 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.249620  2897 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.248937 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.265831  2750 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.253810 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.256392 26054 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:41.257022 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.304783 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:41.304347 26047 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:41.248190 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:41.122332  2899 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 12:27:41.151175  2904 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:41.132026  2810 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.082057  2882 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.278896 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:41.253542  2699 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.710s	user 0.006s	sys 0.006s
I20260709 12:27:41.235528  2821 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:41.240885  2781 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:41.241176  2885 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.237339  2898 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:41.160436  2703 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.115s
I20260709 12:27:41.335304  2898 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.338912  2898 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.085078  2848 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.191s
I20260709 12:27:41.288898  2901 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.290354 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.104630  2825 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.171s
I20260709 12:27:41.254030  2861 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.249191 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.110749  2909 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.200s
I20260709 12:27:41.305027 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.249540 26125 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:27:41.305707 26053 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:41.281733  2889 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.287635  2886 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:41.362108 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.321532  2908 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.313622  2904 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:41.319082 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.319991 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.369853 26124 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:41.321357 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.281049  2814 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.255961  2401 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:27:41.303150  2688 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:41.376602  2688 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:41.377694  2688 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.348284  2821 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:41.380870  2886 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.381394  2821 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.383163  2886 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.384091 26127 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:41.322077 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.389019  2678 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:41.389816 26047 raft_consensus.cc:2410] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:41.323305 26052 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:41.326702 26054 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
W20260709 12:27:41.322280 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 172034us
I20260709 12:27:41.327677  2781 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:41.397213 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:41.327710  2699 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 23. Current term is 24.
I20260709 12:27:41.397990  2800 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.399040  2800 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.399478  2830 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:41.400048  2830 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:41.332979  2885 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:41.332376 26790 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
W20260709 12:27:41.404800 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2876) took 1560 ms (client timeout 1944 ms). Trace:
I20260709 12:27:41.327796  2882 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.347456 26050 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:41.306043  2877 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:41.350493 26046 raft_consensus.cc:2410] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 12:27:41.316798 26094 leader_election.cc:336] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.986s (SENT)
I20260709 12:27:41.409626 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:41.304965  2451 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:27:41.304313  2795 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.416584  2901 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.399490  2800 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.375000  2814 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.401927 26120 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:41.404028 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.423444 26122 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:41.411255  2499 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:41.412488  2918 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:41.416361  2678 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:41.439548 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:41.363987  2889 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.419641 26128 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:41.421093 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:41.405340 26203 rpcz_store.cc:269] 0709 12:27:39.810733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.827929 (+ 17196us) service_pool.cc:224] Handling call
0709 12:27:41.369163 (+1541234us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400217fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:41.371455 (+  2292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1777425792}
I20260709 12:27:41.424750 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.422621 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.416733  2795 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.422441 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.433593 26121 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:41.371470  2904 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:41.360857 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.460654 26126 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:41.423302 26196 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
W20260709 12:27:41.454798 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 138031us
I20260709 12:27:41.452711 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.452709 26197 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:41.457751 26122 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
W20260709 12:27:41.462361 26094 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.090s (SENT)
I20260709 12:27:41.462815 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:41.473420 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.474383 26121 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:27:41.461846 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.478376 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.482503 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.484670 26756 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:41.456269  2810 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:41.486768 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:41.454362  2663 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:41.401077  2830 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.487434 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.612061 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:41.617539 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:41.612501  2721 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.817s	user 0.010s	sys 0.003s
I20260709 12:27:41.621400  2721 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.623193  2676 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.624541  2721 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:41.503428  2749 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:27:41.634011  2870 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.096s	user 0.003s	sys 0.000s
I20260709 12:27:41.502905  2703 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:41.635349  2703 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:41.635732  2703 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:41.636160  2749 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.636765  2749 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:41.638329 26194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1483) took 1252 ms (client timeout 1525 ms). Trace:
W20260709 12:27:41.638747 26194 rpcz_store.cc:269] 0709 12:27:40.385873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.440619 (+ 54746us) service_pool.cc:224] Handling call
0709 12:27:41.638136 (+1197517us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54002175d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:41.638268 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236243840}
W20260709 12:27:41.627664 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 165312us
W20260709 12:27:41.642328 26094 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 1.763s (SENT)
I20260709 12:27:41.636270 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1239) took 1457 ms. Trace:
I20260709 12:27:41.642783 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:41.643057 26049 rpcz_store.cc:276] 0709 12:27:40.179216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.640801 (+461585us) service_pool.cc:224] Handling call
0709 12:27:41.636066 (+995265us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5400229a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:41.636191 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25977600}
I20260709 12:27:41.644215  2821 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.645188 26094 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:41.505708 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.643774 26126 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:41.648047 26120 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:41.645951 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:41.503124 26203 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:41.646492  2810 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:41.513190 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:41.644856  2821 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.480010  2407 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:41.516031  2814 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:41.731455 26121 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:41.762480  2809 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:27:41.768328  2925 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 12:27:41.660787  2923 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:27:41.771198  2887 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:41.773241  2928 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:27:41.533185  2848 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:41.763996  2810 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.765300  2909 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:41.774376  2909 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:41.774165  2848 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.775554  2909 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.613425  2486 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.562410  2873 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.829385 26125 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:41.863660 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.849025  2486 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.864893 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:41.830016  2407 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:41.875211  2848 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:41.918284  2930 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.273s	user 0.000s	sys 0.002s
W20260709 12:27:41.920581 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 274699us
W20260709 12:27:41.921643 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 279612us
I20260709 12:27:41.923213 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:41.925097  2603 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:41.925755  2603 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:41.950938 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:41.955667 26199 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:41.848001  2873 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:41.959720 26202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2869) took 2431 ms (client timeout 1873 ms). Trace:
W20260709 12:27:41.951673  2815 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.702s	user 0.005s	sys 0.006s
W20260709 12:27:41.960278 26202 rpcz_store.cc:269] 0709 12:27:39.525725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.689586 (+163861us) service_pool.cc:224] Handling call
0709 12:27:41.955332 (+2265746us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54002189d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:41.956539 (+  1207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865361152}
W20260709 12:27:41.960889 26094 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.186s (SENT)
I20260709 12:27:41.964113 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:41.965461 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:41.966594 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.949345  2821 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:41.962013 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:41.967319 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:41.968743  2932 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:41.969079  2889 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.970816 26126 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:41.972097 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:41.952502  2672 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:27:41.978488 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:41.972645  2885 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.561s	user 0.005s	sys 0.005s
I20260709 12:27:41.972697  2834 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:41.952920  2749 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:41.973744  2486 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:41.957410  2413 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.022s
I20260709 12:27:41.978647 26782 raft_consensus.cc:2393] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:41.979140 26120 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:41.922484  2407 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.979169 26790 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:41.980037 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:41.992445 26126 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:41.982612 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:41.984284 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:41.984968 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.993544 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.987535 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.994683 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.976840 26789 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:41.988159 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:41.995461 26048 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 28 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:41.958613  2872 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:27:41.990351 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:41.991779 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:41.980417 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:41.987780 26052 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:41.984434  2919 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:41.994575 26045 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:41.984680  2663 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:41.983335  2672 consensus_queue.cc:1243] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:42.003017 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:41.998833 26047 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:42.003571 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.000502 26202 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:27:42.001164 26126 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:42.001288 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.004895 26045 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:41.966279  2815 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:41.989902  2795 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.006036 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.013167 26045 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.011938  2800 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.012048  2821 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.002429  2933 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:41.988823  2885 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.031754 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:42.044050 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51232us
W20260709 12:27:42.034148 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 73762.4us
W20260709 12:27:42.045083 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.047935 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.048138 26045 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:42.034879  2944 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:42.008694  2721 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.037268  2943 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.057317  2943 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.058313  2749 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.059566  2943 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.038936  2948 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:42.049495  2935 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:42.060724  2948 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:42.062711  2811 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.000s
I20260709 12:27:42.063189  2948 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:42.040632 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:42.010200  2769 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.279s	user 0.007s	sys 0.008s
W20260709 12:27:42.036084 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55755.7us
I20260709 12:27:42.055685 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:42.056679  2944 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:42.065634 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:42.070935 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:42.063158  2811 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.081961 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.052265  2663 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.052310  2487 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.084971 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.086818 26052 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:42.087652 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.088840 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.090047 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.026324  2795 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.031926  2703 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.076974  2821 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.048995  2919 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:27:42.094365 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.095204 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.064034  2603 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.105424 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.106288  2815 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.106371 26046 raft_consensus.cc:2376] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23. Re-sending same reply.
I20260709 12:27:42.108484  2944 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.109604 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.066321  2942 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:27:42.082239  2933 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:27:42.111425 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.079442  2676 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:42.112661  2676 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:42.113758  2676 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.115437  2827 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:42.115917  2827 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:42.098635 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.116798 26053 raft_consensus.cc:2376] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24. Re-sending same reply.
I20260709 12:27:42.117962 26122 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.097498 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.118927  2407 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:42.119331 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.119985 26048 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.121733 26124 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:42.122315  2881 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.456s	user 0.004s	sys 0.010s
I20260709 12:27:42.123656 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1673) took 1954 ms. Trace:
I20260709 12:27:42.123906 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.124317 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3231) took 1847 ms. Trace:
I20260709 12:27:42.095938 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.124727 26199 rpcz_store.cc:276] 0709 12:27:40.276494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.752084 (+475590us) service_pool.cc:224] Handling call
0709 12:27:41.955481 (+1203397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.956118 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.124108 (+167990us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002189d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:42.124256 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696889600}
I20260709 12:27:42.125528 26045 raft_consensus.cc:2376] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24. Re-sending same reply.
I20260709 12:27:42.094480 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.109933  2885 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.081670  2948 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:42.131283 26047 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:42.123991 26129 rpcz_store.cc:276] 0709 12:27:40.169320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.169453 (+   133us) service_pool.cc:224] Handling call
0709 12:27:40.169819 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.170267 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.170312 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.170909 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.417579 (+246670us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:40.417630 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.417706 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.999770 (+1582064us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:42.123387 (+123617us) raft_consensus.cc:1685] finished
0709 12:27:42.123410 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.123601 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5445,"raft.queue_time_us":71050,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8745,"spinlock_wait_cycles":11782400,"thread_start_us":246293,"threads_started":1,"wal-append.queue_time_us":413969}
I20260709 12:27:42.117986 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.121656  2795 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.112020  2933 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:42.068389  2769 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.126335 26125 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:42.097164 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.139909 26052 raft_consensus.cc:2376] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 22. Re-sending same reply.
I20260709 12:27:42.141119 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.141847 26052 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:42.142854 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.144251 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:42.108018 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.129822  2703 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.132256 26053 raft_consensus.cc:2376] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27. Re-sending same reply.
I20260709 12:27:42.132323 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.148921 26129 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 20.
I20260709 12:27:42.133822 26048 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:42.065460  2721 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.145643 26046 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:42.152084 26050 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:42.152168 26122 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:42.144047 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.095152  2487 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.117957  2811 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.154227  2795 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.164531  2795 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.135552  2933 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:42.166366  2933 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.167502  2795 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.167685  2933 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:42.167914  2926 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.396s	user 0.002s	sys 0.000s
I20260709 12:27:42.158149 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:42.174145  2769 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:42.176807 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.180274 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:42.190997 26126 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:27:42.191435 26126 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 14
I20260709 12:27:42.191679 26126 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:27:42.191929 26126 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.194368 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.194936 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:42.180708 26094 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Timed out: RequestConsensusVote RPC to 127.24.71.129:44989 timed out after 2.143s (SENT)
I20260709 12:27:42.195822 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.151260  2948 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:42.207592 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49523.7us
I20260709 12:27:42.209059 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:42.161672  2942 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.228659 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:42.230742 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50083.3us
W20260709 12:27:42.227290  2876 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.560s	user 0.024s	sys 0.020s
I20260709 12:27:42.231921 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:42.230396  2904 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.750s	user 0.012s	sys 0.000s
I20260709 12:27:42.116818  2827 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.241742 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:42.226871  2795 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:42.241910 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:42.227149  2814 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.242964 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.227962  2769 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:42.207151  2401 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 12:27:42.241240  2876 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.585s	user 0.024s	sys 0.020s
I20260709 12:27:42.252087 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.253104 26094 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:42.249051  2942 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:42.255900  2942 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:42.255697  2948 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.251091  2781 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.248942  2795 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.260743  2781 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:42.261471 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.263027 26125 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:42.263684  2911 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.247553  2943 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:42.264545  2943 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:42.265060  2943 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:42.248428  2499 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 12:27:42.256446  2942 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.249023  2904 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.286996  2904 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.246784  2949 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.260762  2795 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.283330 26126 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.292956 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:42.290781 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52167.6us
W20260709 12:27:42.289907 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 54237.6us
I20260709 12:27:42.295742 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:42.251120  2769 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:42.302665 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:42.267119  2815 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.338981 26124 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:42.280843  2827 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.340678 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.292173  2814 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:42.293560  2721 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.293881  2401 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:42.348861 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1676) took 1166 ms. Trace:
I20260709 12:27:42.349291 26124 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:42.349488 26128 rpcz_store.cc:276] 0709 12:27:41.182303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.182810 (+   507us) service_pool.cc:224] Handling call
0709 12:27:41.189478 (+  6668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.448726 (+259248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.346145 (+897419us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400287654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:42.348790 (+  2645us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528580992}
I20260709 12:27:42.341437  2811 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:42.353188 26124 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:42.352835 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:42.307276  2769 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:42.351774  2845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.804s	user 0.013s	sys 0.013s
W20260709 12:27:42.358386 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.351416  2911 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.349319  2814 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:42.391486 26126 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:42.356328  2815 raft_consensus.cc:2764] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:42.354152  2401 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:42.383505  2845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.840s	user 0.014s	sys 0.014s
I20260709 12:27:42.355651  2811 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:42.396703  2938 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.451s	user 0.000s	sys 0.003s
I20260709 12:27:42.397733 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2187) took 1157 ms. Trace:
I20260709 12:27:42.358748  2959 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 12:27:42.449378 26784 rpcz_store.cc:276] 0709 12:27:41.240580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.240725 (+   145us) service_pool.cc:224] Handling call
0709 12:27:41.260546 (+ 19821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.277026 (+ 16480us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:41.291470 (+ 14444us) log.cc:844] Serialized 14 byte log entry
0709 12:27:41.641724 (+350254us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b180038bba0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:27:41.924792 (+283068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:41.924851 (+    59us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:41.924934 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.397452 (+472518us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b540057a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:42.397504 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.397666 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14121,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":77,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"spinlock_wait_cycles":62164864,"thread_start_us":282526,"threads_started":1,"wal-append.queue_time_us":631780}
W20260709 12:27:42.451534 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 155886us
W20260709 12:27:42.398824 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4226) took 1289 ms (client timeout 1676 ms). Trace:
I20260709 12:27:42.452358 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:42.407639 26121 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:27:42.356514  2949 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.414208 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2185) took 1652 ms. Trace:
I20260709 12:27:42.453203  2821 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:42.453507 26787 rpcz_store.cc:276] 0709 12:27:40.717603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.717755 (+   152us) service_pool.cc:224] Handling call
0709 12:27:40.739654 (+ 21899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.244107 (+504453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.244138 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:41.244184 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:41.244251 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.356763 (+1112512us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:42.356814 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.356919 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159591808}
I20260709 12:27:42.454274  2935 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:42.418501 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2183) took 1737 ms. Trace:
I20260709 12:27:42.454726  2935 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:42.392411  2960 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:27:42.454825 26783 rpcz_store.cc:276] 0709 12:27:40.680716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.680859 (+   143us) service_pool.cc:224] Handling call
0709 12:27:40.681145 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.910336 (+229191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.910367 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:40.910414 (+    47us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:40.910458 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.418259 (+1507801us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54005ce254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:42.418304 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.418440 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433961728}
I20260709 12:27:42.432655 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.432138 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.456629 26050 raft_consensus.cc:2410] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:27:42.457082 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.457696 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:42.457803 26050 pending_rounds.cc:85] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 14
I20260709 12:27:42.458132 26050 pending_rounds.cc:107] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:27:42.458424 26050 raft_consensus.cc:2889] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.458694 26053 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:42.459108 26053 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:42.431310 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.434223 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.434113 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.434863 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:42.436165 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.460981 26054 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:42.354753  2721 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.453047 26121 pending_rounds.cc:85] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 14
I20260709 12:27:42.461736 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:42.462134 26121 pending_rounds.cc:107] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 27.15
W20260709 12:27:42.452225 26127 rpcz_store.cc:269] 0709 12:27:41.109751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.109940 (+   189us) service_pool.cc:224] Handling call
0709 12:27:42.398577 (+1288637us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:42.398730 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339123584}
I20260709 12:27:42.462582 26121 raft_consensus.cc:2889] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.433430 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.463366 26046 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:42.392751  2814 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.464262 26052 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:42.464984 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81223.2us
I20260709 12:27:42.464826 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:42.469280 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.470173 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:42.468405 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.477993 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:42.479054 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.482015 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.484880 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.485589 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:42.480948 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:42.486456 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:42.487015 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.487581 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:42.454385  2811 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.496428 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.497092 26198 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:42.497876 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.492898 26127 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:42.460114 26053 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.489986  2732 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:42.496227 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:42.499698 26053 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:42.455642  2935 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:42.505030 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52787.6us
I20260709 12:27:42.446214  2919 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:27:42.505726 26782 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:27:42.473763  2904 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:42.508481 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.511265 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.512039 26200 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.517649 26784 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:42.458828  2942 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.459427  2909 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 12:27:42.485984  2827 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.518759 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:42.544958 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.550215 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.551513 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.520879 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.549628 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.553303 26201 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:42.520476 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.555102 26127 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:27:42.556830 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.560312 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.560674 26053 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 20. (term mismatch)
I20260709 12:27:42.561215 26053 pending_rounds.cc:85] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 19
I20260709 12:27:42.561556 26053 pending_rounds.cc:107] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 28.20
I20260709 12:27:42.561918 26053 raft_consensus.cc:2889] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.554350 26199 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:42.495800  2809 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:42.572799 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:42.551601 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.502890  2732 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.565532 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.566787 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:42.503901  2860 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:42.493325  2936 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.588161 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.589126 26194 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:42.573843 26127 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:42.520295  2960 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:42.582476 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.585546 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:42.599530 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:42.600865 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.521534  2959 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:42.612545 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:27:42.572949  2935 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:42.571015  2672 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:42.574118  2732 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.580981  2860 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.517306  2749 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.081s
I20260709 12:27:42.570806  2809 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.629885 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.639765 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:42.655501 26094 leader_election.cc:336] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.641s (SENT)
I20260709 12:27:42.656023 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:42.549834  2401 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.657140  2827 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:42.657126 26194 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:42.666494 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.667351 26196 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:42.668752 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3259) took 1414 ms. Trace:
I20260709 12:27:42.669627  2942 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.669776 26787 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:42.639750 26199 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:42.658051  2401 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.639024 26201 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:27:42.669274 26120 rpcz_store.cc:276] 0709 12:27:41.254698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.684446 (+429748us) service_pool.cc:224] Handling call
0709 12:27:41.978070 (+293624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.993486 (+ 15416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.668481 (+674995us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b54001f86d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:42.668672 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124831616}
I20260709 12:27:42.676103 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.676607 26203 raft_consensus.cc:2368] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:42.640525  2821 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.645263  2860 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.691368 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.673019  2809 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.618242  2918 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:27:42.629496  2959 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.678380 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.697811 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:42.704191 26094 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.696s (SENT)
I20260709 12:27:42.728777 26199 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:42.693187  2407 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:42.744773 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.696887  2949 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.760161 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.841495 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.843114 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.843859 26203 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.839320 26202 raft_consensus.cc:2410] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 32 because replica has last-logged OpId of term: 31 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 14.
I20260709 12:27:42.747079 26198 raft_consensus.cc:2368] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31.
W20260709 12:27:42.739302  2967 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:27:42.690651  2943 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:42.847779  2966 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.382s	user 0.002s	sys 0.000s
I20260709 12:27:42.861630 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:42.863812 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.745512  2407 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.698029 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:42.867202 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.867496 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.872965  2407 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.874640 26120 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:42.875041 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.875640 26198 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:42.876539  2909 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.876714 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.877357  2909 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.878003 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.878362  2897 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.878842 26198 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 30 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:27:42.878901  2897 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.879751 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:42.889423 26128 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:42.889725  2864 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:42.889494  2909 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.896591 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:42.897477 26201 raft_consensus.cc:2376] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28. Re-sending same reply.
W20260709 12:27:42.897938  2937 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.974s	user 0.003s	sys 0.000s
I20260709 12:27:42.898957 26201 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: [27.18-29.19]
I20260709 12:27:42.906217  2809 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:42.909142  2882 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.913559  2809 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:42.913990 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3260) took 1659 ms. Trace:
I20260709 12:27:42.914539 26121 rpcz_store.cc:276] 0709 12:27:41.255027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.768811 (+513784us) service_pool.cc:224] Handling call
0709 12:27:41.956204 (+187393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.485352 (+529148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.902170 (+416818us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54001c4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:42.913901 (+ 11731us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037229824}
I20260709 12:27:42.910090  2407 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.918071  2407 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:42.698781  2821 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:42.737834  2965 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 12:27:42.731992  2795 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:42.926676 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 422708us
I20260709 12:27:42.938776 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2181) took 2251 ms. Trace:
I20260709 12:27:42.937840 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:42.959232 26120 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:27:42.960865 26198 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 21: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:42.939183 26786 rpcz_store.cc:276] 0709 12:27:40.671066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.671277 (+   211us) service_pool.cc:224] Handling call
0709 12:27:40.687420 (+ 16143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.687663 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:27:40.931676 (+244013us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.404179 (+1472503us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b18004568e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:27:42.922453 (+518274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:42.922524 (+    71us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:42.922598 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.922694 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.922853 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36015,"prepare.queue_time_us":243873,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":102760,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":54,"spinlock_wait_cycles":29324032,"thread_start_us":719663,"threads_started":3,"wal-append.queue_time_us":1726632}
W20260709 12:27:42.963405 26125 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.009s	sys 0.008s
I20260709 12:27:42.855593  2968 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 12:27:42.852953  2949 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:42.963852 26125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.009s	sys 0.008s
I20260709 12:27:42.980634 26195 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 23: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:43.043337 26200 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.515s	user 0.015s	sys 0.000s
I20260709 12:27:42.865809  2975 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:43.047183 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 461725us
W20260709 12:27:43.050665 26050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.009s	sys 0.008s
W20260709 12:27:43.051030 26050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.009s	sys 0.008s
I20260709 12:27:43.049634  2975 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:43.051319 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:43.063884 26199 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:43.060343 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:43.046638  2983 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:43.127455 26049 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:27:43.048262 26200 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:42.926170  2885 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:43.052090 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 359407us
I20260709 12:27:43.144865 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:43.145370 26200 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:43.147724 26200 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:27:43.026290  2936 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.962930  2877 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:42.979584  2904 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:43.003723 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 437006us
I20260709 12:27:43.148842 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:43.177707 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:43.119899 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69247.1us
I20260709 12:27:43.181605 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:43.172950  2993 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:43.180889  2877 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:43.192015 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 396718us
W20260709 12:27:43.194279 26094 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.867s (SENT)
W20260709 12:27:43.250475 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 72887.1us
W20260709 12:27:43.250007 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68511.7us
I20260709 12:27:43.325451 26168 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:43.251691 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60499.2us
W20260709 12:27:43.197095 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 136852us
I20260709 12:27:43.326994 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.329016  2877 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:43.329312 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.186200  2974 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:27:43.329828  2949 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:43.295570  2993 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:27:43.120921  2947 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:43.120893  2975 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:43.172886  2749 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.329707  2877 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.337590 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.338397 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:43.348965 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.068418  2732 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:43.349668  2966 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.172s	user 0.004s	sys 0.000s
I20260709 12:27:43.333355 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:43.195611  2983 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:43.172796  2885 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.199673  2959 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:43.370409  2988 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.370741  2904 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.372449 26789 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
W20260709 12:27:43.374704 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 180470us
I20260709 12:27:43.334358  2976 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:43.380355 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:43.387863 26094 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.873s (SENT)
I20260709 12:27:43.349972  2993 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.341786  2977 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:43.388794 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55561.8us
W20260709 12:27:43.382296 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44807.5us
I20260709 12:27:43.349972  2960 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:43.389375 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.401595 26048 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:43.391582 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:43.402856 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:43.404661 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:43.364084  2982 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.412768 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:43.400394 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:43.368332  2943 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:43.415882 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:43.461829 26094 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.007s (SENT)
I20260709 12:27:43.423975  2809 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:43.462708 26094 leader_election.cc:336] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.081s (SENT)
I20260709 12:27:43.463166 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:43.464622 26094 leader_election.cc:336] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.096s (SENT)
I20260709 12:27:43.465157 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.427517  2821 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:43.432375  2980 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.423506 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:43.434206  2989 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:43.467016  2955 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 1.118s	user 0.002s	sys 0.004s
I20260709 12:27:43.467182  2989 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:43.467582  2809 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:43.468348  2989 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.469983  2873 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:43.436178 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4259) took 1226 ms. Trace:
I20260709 12:27:43.470801 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3269) took 1203 ms. Trace:
I20260709 12:27:43.470715 26126 rpcz_store.cc:276] 0709 12:27:42.209652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.232121 (+ 22469us) service_pool.cc:224] Handling call
0709 12:27:43.435928 (+1203807us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b54001f6dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:43.436108 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41643264}
I20260709 12:27:43.436890  2982 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:43.471263 26127 rpcz_store.cc:276] 0709 12:27:42.267136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.608675 (+341539us) service_pool.cc:224] Handling call
0709 12:27:42.609520 (+   845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.622816 (+ 13296us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:42.622866 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.674187 (+ 51321us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.061124 (+386937us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:43.061164 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.061242 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.470550 (+409308us) raft_consensus.cc:1685] finished
0709 12:27:43.470584 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.470743 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":430,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11860,"thread_start_us":360387,"threads_started":1,"wal-append.queue_time_us":771367}
I20260709 12:27:43.473757  2982 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:43.441294 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:43.443883 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52396.1us
I20260709 12:27:43.373997  2910 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.447252 26045 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:43.475744 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:43.476163 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:43.450189 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:43.476648  2910 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:43.477200 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:43.452939  2979 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:43.477636  2980 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:43.454196  2885 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:43.478139  2979 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:43.460626  2966 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:27:43.478653  2979 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:43.479010  2811 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.973s	user 0.009s	sys 0.000s
I20260709 12:27:43.451553 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2830) took 1020 ms. Trace:
W20260709 12:27:43.479530 26094 leader_election.cc:336] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.159s (SENT)
I20260709 12:27:43.479450  2811 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:43.476401 26045 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:43.477533 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:43.458144  2992 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:27:43.474701 26046 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:43.480111 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.479919 26049 rpcz_store.cc:276] 0709 12:27:42.431620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.433860 (+  2240us) service_pool.cc:224] Handling call
0709 12:27:43.451298 (+1017438us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:43.451472 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532229504}
I20260709 12:27:43.467252 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:43.423385 26203 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.001s
I20260709 12:27:43.495215 26203 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:43.388475  2993 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:43.499709 26094 leader_election.cc:336] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.170s (SENT)
I20260709 12:27:43.500403 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.376310  2986 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:43.502869 26094 leader_election.cc:336] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.173s (SENT)
I20260709 12:27:43.499275 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:43.515874 26124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.021s	sys 0.009s
W20260709 12:27:43.525207 26124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.022s	sys 0.010s
W20260709 12:27:43.523499 26121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.022s
I20260709 12:27:43.522156 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:43.527503 26121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.006s	sys 0.022s
W20260709 12:27:43.519271 26194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.017s	sys 0.014s
W20260709 12:27:43.530409 26194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.018s	sys 0.014s
I20260709 12:27:43.535627 26045 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:43.522454 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:43.491693  2401 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.782s	user 0.006s	sys 0.002s
I20260709 12:27:43.539942 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:43.544122 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.022s	sys 0.038s
W20260709 12:27:43.545037 26784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.023s	sys 0.012s
I20260709 12:27:43.501258  2910 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.373314  2877 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.400926  2977 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:43.587657 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.022s	sys 0.049s
W20260709 12:27:43.587695 26784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.025s	sys 0.013s
I20260709 12:27:43.589886 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2829) took 1158 ms. Trace:
I20260709 12:27:43.548959 26199 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:43.495889  2980 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:43.590507 26050 rpcz_store.cc:276] 0709 12:27:42.431340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.457217 (+ 25877us) service_pool.cc:224] Handling call
0709 12:27:42.457574 (+   357us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:27:42.459108 (+  1534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.459135 (+    27us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:27:42.474617 (+ 15482us) log.cc:844] Serialized 104 byte log entry
0709 12:27:43.066262 (+591645us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:43.126142 (+ 59880us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.428398 (+302256us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b1800074e60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:43.542761 (+114363us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b1800074e60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b77d9d3c7 00007f5b743af9a7 00007f5b743bf0bf 00007f5b743bf038
0709 12:27:43.589051 (+ 46290us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.589379 (+   328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.589486 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.589551 (+    65us) raft_consensus.cc:1685] finished
0709 12:27:43.589583 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.589806 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245620,"prepare.queue_time_us":478789,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":6502,"raft.queue_time_us":49630,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":25472,"spinlock_wait_cycles":117915136,"thread_start_us":548781,"threads_started":1,"wal-append.queue_time_us":872584}
I20260709 12:27:43.519095  2809 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:43.496124  2985 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:43.593015  2992 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:43.545106  2749 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:43.546924  2897 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:43.644441  2678 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:43.647071 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1685) took 2237 ms (client timeout 2227 ms). Trace:
W20260709 12:27:43.647564 26122 rpcz_store.cc:269] 0709 12:27:41.410139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.478592 (+ 68453us) service_pool.cc:224] Handling call
0709 12:27:43.646808 (+2168216us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54001c3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:43.646951 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155872512}
I20260709 12:27:43.591578  2977 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:43.638701  2910 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:43.653133  2910 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:43.653254 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.654312  2910 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:43.656224 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2891) took 1144 ms. Trace:
I20260709 12:27:43.656699 26202 rpcz_store.cc:276] 0709 12:27:42.512252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.551079 (+ 38827us) service_pool.cc:224] Handling call
0709 12:27:43.656007 (+1104928us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:43.656161 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658180864}
W20260709 12:27:43.657326 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.657213  2811 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:43.658910 26052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:27:43.659814 26052 raft_consensus.cc:606] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.660192 26052 raft_consensus.cc:614] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:43.660106  2795 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:43.662214  2821 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:43.663419  2818 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:43.663820  2818 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:43.538211  2885 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:27:43.664887  2818 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:43.667719  2487 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:43.670117  2877 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:43.673596 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1501) took 1182 ms. Trace:
I20260709 12:27:43.674105 26195 rpcz_store.cc:276] 0709 12:27:42.491081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.497662 (+  6581us) service_pool.cc:224] Handling call
0709 12:27:43.673380 (+1175718us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:43.673529 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093122816}
W20260709 12:27:43.678040  2981 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 12:27:43.678479 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2889) took 1166 ms. Trace:
I20260709 12:27:43.678949 26198 rpcz_store.cc:276] 0709 12:27:42.511604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.879496 (+367892us) service_pool.cc:224] Handling call
0709 12:27:43.678234 (+798738us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b5400302854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:43.678408 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132860288}
I20260709 12:27:43.681332  2603 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:43.682763  2976 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:43.683192  2818 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:43.683534 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:43.685077  2980 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.692749 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:43.693774 26047 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:43.695362 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:43.696113 26052 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:43.712304 26120 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.008s	sys 0.023s
W20260709 12:27:43.712872 26120 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.008s	sys 0.023s
I20260709 12:27:43.607420  2995 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:43.741353 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:43.753518 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:43.791481 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.795747 26047 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:43.800781 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4260) took 1316 ms. Trace:
I20260709 12:27:43.740283  2676 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:43.740177  2845 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:27:43.804728  2838 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:43.805151  2845 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.805508 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2900) took 1290 ms. Trace:
I20260709 12:27:43.805411  2838 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.805738  2845 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.806007 26194 rpcz_store.cc:276] 0709 12:27:42.515187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.911988 (+396801us) service_pool.cc:224] Handling call
0709 12:27:42.912323 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.912679 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.912707 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.913166 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.531549 (+618383us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:43.531622 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.531724 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.805286 (+273562us) raft_consensus.cc:1685] finished
0709 12:27:43.805308 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.805447 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":617952,"threads_started":1,"wal-append.queue_time_us":880452}
W20260709 12:27:43.732252  2830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.018s	sys 0.019s
I20260709 12:27:43.697794  2809 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.811290 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:43.819525 26052 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:43.740011  2864 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:43.833076 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:43.807972  2838 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:43.837721 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:43.801640 26128 rpcz_store.cc:276] 0709 12:27:42.484489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.890344 (+405855us) service_pool.cc:224] Handling call
0709 12:27:42.890626 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.890824 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:43.383151 (+492327us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.800279 (+417128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.800358 (+    79us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:43.800432 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.800523 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.800697 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":549486,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":48,"thread_start_us":882616,"threads_started":2}
I20260709 12:27:43.755528  2965 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:43.779896  2886 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:43.829808  2814 consensus_meta.cc:287] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.319s	user 0.006s	sys 0.007s
W20260709 12:27:43.876458 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 354034us
I20260709 12:27:43.877242 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:43.843536  2769 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.452s	user 0.013s	sys 0.001s
I20260709 12:27:43.880761 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:43.827328  2676 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:43.811359  2915 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.042s	user 0.004s	sys 0.013s
I20260709 12:27:43.780083  2603 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.073s
I20260709 12:27:43.895618 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:43.907634 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:43.912864 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 372998us
I20260709 12:27:43.839036  2864 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:43.914938 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:43.921366 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.937055 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:43.751636  2995 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.602485  2983 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:43.982775 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.836649  2809 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:43.910450  2915 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.141s	user 0.004s	sys 0.015s
W20260709 12:27:43.990612 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.987533 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:43.980597  2975 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.003s
I20260709 12:27:44.004154 26785 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:27:43.914394  2986 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:43.887369  2814 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:43.983440  2897 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:43.902125  2769 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.028312 26126 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:44.028946 26198 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:44.030206 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.051383 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.052258 26198 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
I20260709 12:27:44.053731 26198 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:44.031127 26203 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:27:44.054915 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:44.051486 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.031352 26199 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:44.004405  2845 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.061617 26194 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 18. (term mismatch)
I20260709 12:27:44.011811  2968 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:44.062249 26194 pending_rounds.cc:85] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 17
I20260709 12:27:43.966161  2979 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.017975  2989 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:44.029268  2897 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:44.025000  2799 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:44.076478 26197 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:44.057974  2864 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.076728  2799 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.077203  2864 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.079071  2799 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.067348 26194 pending_rounds.cc:107] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 27.18
I20260709 12:27:44.072386  2968 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.082872  2968 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:44.083868  2960 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.000s
W20260709 12:27:44.084412  3009 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 12:27:44.084713  2860 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.386s	user 0.013s	sys 0.000s
I20260709 12:27:44.085276  2860 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.007830  2977 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.064517 26756 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:44.068653  3023 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:44.025251  2975 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.048707  2948 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:44.080415  3002 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:27:44.080791 26194 raft_consensus.cc:2889] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:44.068698 26202 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:44.170467 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.091190 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.092764 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.173972 26198 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:44.075035 26199 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:44.072054  3003 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 12:27:44.161804 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 97388.8us
I20260709 12:27:44.175845 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:44.177985 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 282452us
I20260709 12:27:44.162467 26790 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:44.073809  3026 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.084440  2960 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:44.179800 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:44.180727 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.183781 26195 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:44.181370 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2911) took 1625 ms. Trace:
I20260709 12:27:44.206622 26791 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:44.208421 26202 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:44.200850 26200 rpcz_store.cc:276] 0709 12:27:42.556459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.148918 (+592459us) service_pool.cc:224] Handling call
0709 12:27:43.149325 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.249368 (+100043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.249401 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.249953 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.740917 (+490964us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:43.740961 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.741040 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.181056 (+440016us) raft_consensus.cc:1685] finished
0709 12:27:44.181088 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.181281 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99458,"raft.queue_time_us":100759,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":46826,"thread_start_us":490613,"threads_started":1,"wal-append.queue_time_us":658669}
I20260709 12:27:44.212177 26129 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:44.214171 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.215071 26129 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:44.217651 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4265) took 1472 ms. Trace:
I20260709 12:27:44.218197 26120 rpcz_store.cc:276] 0709 12:27:42.745687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.042862 (+297175us) service_pool.cc:224] Handling call
0709 12:27:43.053220 (+ 10358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.053441 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:43.716746 (+663305us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.217222 (+500476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.217309 (+    87us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:44.217378 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.217445 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.217577 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":387134,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"thread_start_us":1147975,"threads_started":3,"wal-append.queue_time_us":364024}
I20260709 12:27:44.220249 26120 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:44.222105 26120 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:44.220695 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 237016us
I20260709 12:27:44.223824 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:44.224128 26019 connection.cc:256] RPC call timeout handler was delayed by -2.002662247999524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:27:44.224714 26019 connection.cc:256] RPC call timeout handler was delayed by -1.9385791610002343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 12:27:44.225099 26019 connection.cc:256] RPC call timeout handler was delayed by -1.8551793279998492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 12:27:44.225461 26019 connection.cc:256] RPC call timeout handler was delayed by -1.5758003870000721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 12:27:44.226038  2845 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.226573  2845 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.227730 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.227916 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:44.230952 26129 raft_consensus.cc:1240] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:44.231738 26194 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:27:44.232321 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.236394 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:44.236438 26194 pending_rounds.cc:85] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 16
I20260709 12:27:44.237109 26194 pending_rounds.cc:107] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:27:44.237353 26194 raft_consensus.cc:2889] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:44.168818  2800 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:27:44.238557 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 58835.2us
I20260709 12:27:44.241128 26126 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:44.241951 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:44.246913 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.248064 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:44.179436  3027 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:44.186403  2703 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:44.259409 26129 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:44.053020  2676 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.242479 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:44.213075  2811 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:44.182266  3023 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:44.173462 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.185863  2968 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.077s
I20260709 12:27:44.282876 26126 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:44.252102  2960 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:44.294868 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.181167  2948 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.266316  2811 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.310544 26120 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:27:44.244087  3032 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:44.277635  2976 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:44.277515  2603 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.282825  2915 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:44.283560  2821 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:44.211563  2986 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:44.325186 26167 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.767s (SENT) [suppressed 48 similar messages]
I20260709 12:27:44.353816  3023 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.355260 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2899) took 1840 ms. Trace:
I20260709 12:27:44.355530 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1519) took 1848 ms. Trace:
I20260709 12:27:44.295589  3035 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:44.207697  3026 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.289177  2800 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:44.381356  2800 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.296571  2830 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:44.383322  2965 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:44.384616  2993 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.000s
I20260709 12:27:44.385308  2993 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.386580 26203 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
W20260709 12:27:44.387549  3004 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 12:27:44.392787 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4262) took 1891 ms. Trace:
I20260709 12:27:44.393321  3026 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:44.393774  3008 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.411s	user 0.000s	sys 0.004s
I20260709 12:27:44.395315  2915 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.397234  2979 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:44.398072  2915 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:44.323490  2811 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:44.345974  3044 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:44.268211  3027 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.413705 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4239) took 3009 ms. Trace:
I20260709 12:27:44.347707  2977 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.243659  2897 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.313476  2795 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.349483  3020 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:44.350623  3029 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:27:44.350710  2769 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:44.351264  3007 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 12:27:44.328903  2989 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:44.328186  2976 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.353261  2814 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.354038  2821 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:44.363204  2935 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.368505  2809 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:44.369580  2997 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.469s	user 0.002s	sys 0.000s
I20260709 12:27:44.370464  2975 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:44.372609  2818 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.006s
I20260709 12:27:44.345405  3025 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:44.496301 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2828) took 2065 ms. Trace:
I20260709 12:27:44.496760 26051 rpcz_store.cc:276] 0709 12:27:42.431051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.439607 (+  8556us) service_pool.cc:224] Handling call
0709 12:27:42.459944 (+ 20337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:42.490444 (+ 30500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.490493 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.491306 (+   813us) log.cc:844] Serialized 52 byte log entry
0709 12:27:43.590052 (+1098746us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:43.907178 (+317126us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.003132 (+ 95954us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.290316 (+287184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.290409 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.362810 (+ 72401us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b18003dc540. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:44.368718 (+  5908us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:44.496018 (+127300us) raft_consensus.cc:1685] finished
0709 12:27:44.496040 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.496213 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110604,"prepare.queue_time_us":221259,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":5440,"raft.queue_time_us":235638,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":11292,"spinlock_wait_cycles":30355584,"thread_start_us":1615636,"threads_started":3,"wal-append.queue_time_us":1277285}
I20260709 12:27:44.499755 26051 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:27:44.504356  2994 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:27:44.507275  3027 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.507508 26202 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:44.355937 26784 rpcz_store.cc:276] 0709 12:27:42.507452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.518487 (+ 11035us) service_pool.cc:224] Handling call
0709 12:27:42.566858 (+ 48371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:42.935759 (+368901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.935791 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.936420 (+   629us) log.cc:844] Serialized 52 byte log entry
0709 12:27:43.593353 (+656933us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:43.593406 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.593487 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.355319 (+761832us) raft_consensus.cc:1685] finished
0709 12:27:44.355341 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.355475 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84998,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":934740,"threads_started":2,"wal-append.queue_time_us":837210}
I20260709 12:27:44.393338 26121 rpcz_store.cc:276] 0709 12:27:42.501814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.915534 (+413720us) service_pool.cc:224] Handling call
0709 12:27:42.915862 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.920324 (+  4462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.920384 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.921002 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.529908 (+608906us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:43.529976 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.530082 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.386819 (+856737us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:44.392400 (+  5581us) raft_consensus.cc:1685] finished
0709 12:27:44.392433 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.392710 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32149,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":20389,"spinlock_wait_cycles":34402176,"thread_start_us":608429,"threads_started":1,"wal-append.queue_time_us":930324}
I20260709 12:27:44.408773 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:44.511858 26052 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:44.409015 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:44.409614 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:44.355686 26201 rpcz_store.cc:276] 0709 12:27:42.514871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.899837 (+384966us) service_pool.cc:224] Handling call
0709 12:27:42.900196 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.921086 (+ 20890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.921124 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.921679 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.501269 (+579590us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:43.852988 (+351719us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.880550 (+ 27562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.880657 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.355008 (+474351us) raft_consensus.cc:1685] finished
0709 12:27:44.355032 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.355185 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27882,"prepare.queue_time_us":550300,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":43789,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":53097,"spinlock_wait_cycles":41238400,"thread_start_us":825652,"threads_started":2,"wal-append.queue_time_us":900925}
I20260709 12:27:44.347021  3038 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.383984  2965 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:44.415513 26123 rpcz_store.cc:276] 0709 12:27:41.399246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.864567 (+465321us) service_pool.cc:224] Handling call
0709 12:27:41.864884 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.893292 (+ 28408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.893337 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.225342 (+1332005us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b2c00010828. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:27:43.307928 (+ 82586us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.763861 (+455933us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:43.763905 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.766668 (+  2763us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.366884 (+600216us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:44.408500 (+ 41616us) raft_consensus.cc:1685] finished
0709 12:27:44.408520 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.408645 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27935,"raft.queue_time_us":28367,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":243,"spinlock_wait_cycles":43336832,"thread_start_us":390719,"threads_started":1,"wal-append.queue_time_us":523229}
W20260709 12:27:44.431457 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 189579us
I20260709 12:27:44.434473 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:44.436271 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.544260 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.387959  2993 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:44.447695 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:44.356637  2603 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.489548  2821 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.492681  2935 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.558549 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:44.559243 26050 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:44.459931  2976 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:44.452384  2989 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:44.435082  3035 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.445996  2795 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.558161  2769 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:44.447227  3020 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.533794  2499 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.580111 26049 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
W20260709 12:27:44.606141 26197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2882) took 2100 ms (client timeout 2546 ms). Trace:
W20260709 12:27:44.606580 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48132.7us
W20260709 12:27:44.606732 26197 rpcz_store.cc:269] 0709 12:27:42.506152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.506311 (+   159us) service_pool.cc:224] Handling call
0709 12:27:44.605875 (+2099564us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54002b5f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:44.606055 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3443104768}
W20260709 12:27:44.616797 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.633338 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 393567us
I20260709 12:27:44.635061 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.636034 26121 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 24 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 19.
I20260709 12:27:44.639956 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.641881 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:44.644749 26123 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:44.646977 26120 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 12:27:44.607967  2838 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.011s	sys 0.023s
I20260709 12:27:44.672544  2845 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:44.577317  2877 consensus_queue.cc:1059] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:27:44.673570  2915 raft_consensus.cc:3055] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:44.674113  2915 raft_consensus.cc:740] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:44.675277  2915 consensus_queue.cc:260] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.677093 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1516) took 2182 ms. Trace:
I20260709 12:27:44.677160  2915 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:44.677696 26789 rpcz_store.cc:276] 0709 12:27:42.494708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.494830 (+   122us) service_pool.cc:224] Handling call
0709 12:27:42.495168 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.387564 (+892396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.676784 (+1289220us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54005708d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:44.677022 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016039680}
I20260709 12:27:44.680325  2989 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.681691  2989 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.623773  2499 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.690095 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.690737 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.690986 26784 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:27:44.690881  3045 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:44.691562 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:44.692129 26783 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:44.691999  3045 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.692019 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.693517  3039 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.693843 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.694015  3039 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.696441  2830 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:44.697131  2830 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.688970 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.703954 26791 raft_consensus.cc:2410] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:27:44.652654 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:44.735960 26789 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:44.563711  3038 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.637609  3027 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:44.671936  2838 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.012s	sys 0.023s
I20260709 12:27:44.744426 26789 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:44.741794 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:44.745844 26791 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 23: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:44.593145  2769 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.660813  2818 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:44.748769 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 96214.2us
I20260709 12:27:44.752810 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:44.760850 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.758276 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.709628  3026 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:27:44.748179  2948 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:44.762068 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 436885us
I20260709 12:27:44.818496 26784 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:44.739777  3047 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:44.739905  2982 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
W20260709 12:27:44.824347 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.824170 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.824909 26788 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
W20260709 12:27:44.772436  2910 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.101s	user 0.022s	sys 0.033s
W20260709 12:27:44.821188 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2213370429999486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
W20260709 12:27:44.774542  2864 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.695s	user 0.013s	sys 0.004s
I20260709 12:27:44.737143  2845 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.032s
I20260709 12:27:44.832607 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.826686 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:44.833669 26200 raft_consensus.cc:2376] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24. Re-sending same reply.
I20260709 12:27:44.757668  2830 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.827319 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.820706 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.828181 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:44.760028  3033 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.825664 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.838215 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:44.838953 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:44.839442 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:44.748742  3027 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.844794 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:44.845638 26200 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:44.847549 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:44.830761  2910 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.023s	sys 0.033s
I20260709 12:27:44.854246 26194 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:27:44.838626 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:44.820122  2809 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:44.823393  3047 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:44.866936 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:44.872342  2977 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:44.873067  2977 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:44.828716  2948 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:44.874207  2948 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.876521  3039 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:44.877714  3039 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:44.878132  3039 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:44.879014  3039 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.828439  3036 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:44.880450  3036 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.865658  3045 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.882951  3036 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:44.885171  3050 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.885717  3050 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.866130  2795 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.870399  2676 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:44.870918  2864 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:44.860080 26203 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:44.885950  2809 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.898015  2809 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.889338  2864 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:44.919435 26783 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:44.892652  3039 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:44.887861  3050 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.887502  2769 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.886829  2910 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.925535  2910 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.892133  3036 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:44.928786  3036 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.930986  3036 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.891101  2948 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.920550  2864 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:44.886942  2814 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:44.932670  2814 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.932755  2864 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:44.934397  2864 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.935590  2814 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:44.940749  3047 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:44.950877 26200 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:44.927894  2910 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.950593  2935 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.974987 26791 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
W20260709 12:27:44.996439 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 333138us
I20260709 12:27:44.967563  2821 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:44.972036  3020 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.017445 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.018901 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.015614 26053 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:27:45.022198 26053 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 16
I20260709 12:27:45.022518 26053 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:27:45.022820 26053 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:45.016804 26048 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:27:45.024451 26048 pending_rounds.cc:85] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 14
I20260709 12:27:45.024783 26048 pending_rounds.cc:107] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:27:45.025061 26048 raft_consensus.cc:2889] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:45.026726  3026 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:45.027693  2818 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:45.028069  2818 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:45.029081  2818 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.030359  2818 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:45.033730  3052 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:27:45.034061 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.035061 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3276) took 1021 ms. Trace:
I20260709 12:27:45.027976 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.035310 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.039922 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.042846 26784 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:45.046417 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:44.968375  2976 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.004415  2499 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.035841 26125 rpcz_store.cc:276] 0709 12:27:44.013194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.264076 (+250882us) service_pool.cc:224] Handling call
0709 12:27:44.270570 (+  6494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.303471 (+ 32901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.303526 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.304189 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.608214 (+304025us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:44.608273 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.608339 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.034856 (+426517us) raft_consensus.cc:1685] finished
0709 12:27:45.034877 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.035004 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32301,"raft.queue_time_us":78189,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":259236,"threads_started":1,"wal-append.queue_time_us":495925}
I20260709 12:27:45.057817 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.058768 26051 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:45.016819  3039 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:27:45.070478  3038 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.098425 26093 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.101930  3026 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:45.102409  3026 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:45.104072  2965 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.007s
W20260709 12:27:45.112941  3001 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.466s	user 0.008s	sys 0.006s
I20260709 12:27:45.111941  3026 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.116044  3026 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.116773 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2902) took 2601 ms. Trace:
I20260709 12:27:45.117219 26196 rpcz_store.cc:276] 0709 12:27:42.515794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.914162 (+398368us) service_pool.cc:224] Handling call
0709 12:27:42.914554 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.915001 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:42.915351 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.485346 (+569995us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.543255 (+ 57909us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.543473 (+   218us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:43.831697 (+288224us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.850889 (+ 19192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.850972 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.112312 (+1261340us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:45.116496 (+  4184us) raft_consensus.cc:1685] finished
0709 12:27:45.116531 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.116685 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119124,"prepare.queue_time_us":532970,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":65465,"raft.queue_time_us":94963,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":28745,"spinlock_wait_cycles":74010496,"thread_start_us":781191,"threads_started":2,"wal-append.queue_time_us":730648}
I20260709 12:27:45.125659  2860 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.129426 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1697) took 1393 ms. Trace:
I20260709 12:27:45.129891 26129 rpcz_store.cc:276] 0709 12:27:43.735881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.241859 (+505978us) service_pool.cc:224] Handling call
0709 12:27:44.259220 (+ 17361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.264658 (+  5438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.129178 (+864520us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b540013ef50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:45.129353 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38772992}
I20260709 12:27:45.105155  3027 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:45.140311 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 309316us
W20260709 12:27:45.142119 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 526077us
I20260709 12:27:45.146901 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.147719 26125 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:45.145364 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.152446 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2199) took 1237 ms. Trace:
I20260709 12:27:45.153049 26790 rpcz_store.cc:276] 0709 12:27:43.915388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.936635 (+ 21247us) service_pool.cc:224] Handling call
0709 12:27:45.152176 (+1215541us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400715a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:45.152356 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422749952}
W20260709 12:27:45.143597 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 508173us
I20260709 12:27:45.161648 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.163722 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.166275 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.169965 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.170933 26121 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
W20260709 12:27:45.202687 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.203166 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.203452 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.213105 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.204963  2999 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.209s	user 0.006s	sys 0.006s
I20260709 12:27:45.137902  2965 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:45.234997  3054 log.cc:927] Time spent T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 12:27:45.228798  2975 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:45.236501  3058 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.241930  2769 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.267094 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 45335us
I20260709 12:27:45.271324 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.247038  3026 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 12:27:45.294456 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3271) took 2804 ms. Trace:
I20260709 12:27:45.294675 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.295807 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.297297 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.294910 26124 rpcz_store.cc:276] 0709 12:27:42.489839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.905397 (+415558us) service_pool.cc:224] Handling call
0709 12:27:42.905802 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.906125 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.906167 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.906837 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.525644 (+618807us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:43.525687 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.525761 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.131914 (+1606153us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:45.294175 (+162261us) raft_consensus.cc:1685] finished
0709 12:27:45.294195 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.294394 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17405696,"thread_start_us":617366,"threads_started":1,"wal-append.queue_time_us":927210}
W20260709 12:27:45.304060 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85270.1us
I20260709 12:27:45.272756  2976 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.255604  2975 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.314944 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.315675 26050 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:45.320101 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:45.272903  3058 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.322464 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.322135 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:45.286672  3059 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:27:45.325886 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.326607 26050 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:45.330622  2965 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:45.308485  3045 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:45.332160  3062 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:27:45.346222  2603 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.349648 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4280) took 1165 ms. Trace:
I20260709 12:27:45.375300 26120 rpcz_store.cc:276] 0709 12:27:44.184166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.323850 (+139684us) service_pool.cc:224] Handling call
0709 12:27:44.405614 (+ 81764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.647373 (+241759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.349319 (+701946us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001f86d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:45.349566 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554352384}
I20260709 12:27:45.353663  3061 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:27:45.354626  3011 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 1.260s	user 0.000s	sys 0.012s
W20260709 12:27:45.357229 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.361110 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4277) took 1289 ms. Trace:
I20260709 12:27:45.361907  2732 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:45.364702  2897 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:45.367551  2811 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.937s	user 0.017s	sys 0.000s
I20260709 12:27:45.369617 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:45.288172  2769 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.371006 26053 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:45.409349 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:45.449505 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.385866 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:45.449750 26126 rpcz_store.cc:276] 0709 12:27:44.071619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.286449 (+214830us) service_pool.cc:224] Handling call
0709 12:27:44.308704 (+ 22255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.356873 (+ 48169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.356921 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.356982 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:44.357044 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.360797 (+1003753us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:45.360860 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.361040 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110142848}
I20260709 12:27:45.386277 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:45.445263 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.323942  2976 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.450484 26128 raft_consensus.cc:2376] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26. Re-sending same reply.
I20260709 12:27:45.409777 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:45.464355 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:45.466333 26124 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:45.469544 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.462929  2703 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:45.471586  2703 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.468307  3046 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:27:45.466588  3066 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:27:45.480513  2818 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:45.480811 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.481791 26052 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:27:45.468829  2976 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.482648 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.483217 26052 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.480995  2818 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.485100  2818 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:45.485546 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.486063  2800 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:45.463248  3077 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:45.486975  3077 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:45.487385  3077 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:45.487768  2915 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:45.488304 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3279) took 1285 ms. Trace:
I20260709 12:27:45.488692 26122 rpcz_store.cc:276] 0709 12:27:44.203347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.358318 (+154971us) service_pool.cc:224] Handling call
0709 12:27:44.358622 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.358826 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:44.414327 (+ 55501us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.121449 (+707122us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b1800324e80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b77d9d3c7 00007f5b743af9a7 00007f5b743bf0bf 00007f5b743bf038
0709 12:27:45.121662 (+   213us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.487787 (+366125us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b1800324e80. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:27:45.487944 (+   157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:45.487994 (+    50us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:45.488047 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.488112 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.488250 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":679979,"prepare.queue_time_us":779039,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":1236,"spinlock_wait_cycles":76797312}
I20260709 12:27:45.468665  2811 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.494822  3060 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:45.495561  2811 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:45.497256 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 355701us
W20260709 12:27:45.497563 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 399229us
I20260709 12:27:45.499298 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.500809 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3280) took 1268 ms. Trace:
I20260709 12:27:45.501197 26127 rpcz_store.cc:276] 0709 12:27:44.232791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.506121 (+273330us) service_pool.cc:224] Handling call
0709 12:27:44.506463 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.506696 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:45.123141 (+616445us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.500431 (+377290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:45.500516 (+    85us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:45.500565 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.500639 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.500754 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":745518,"prepare.queue_time_us":629867,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":247529,"threads_started":1}
I20260709 12:27:45.502128 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.502466 26050 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:45.435179 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:45.509531 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 187487us
I20260709 12:27:45.498903 26093 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.508723 26122 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:45.512601 26120 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:45.513370 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.510668 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.513517 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:45.514176 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 194012us
I20260709 12:27:45.514513 26050 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.516518 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.517266 26052 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:45.520550 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:45.525158 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.381295  2965 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.495364  3060 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:45.533090 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.513468  2935 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:45.523891  3045 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:27:45.551692 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.548269 26050 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:45.530818  3027 raft_consensus.cc:2764] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:27:45.532110  2603 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:45.557274  2976 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:45.527297  3023 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:27:45.559121  3058 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.549221  2910 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:45.561554  3033 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.715s	user 0.006s	sys 0.006s
I20260709 12:27:45.543551  3049 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.019s
I20260709 12:27:45.562343  2811 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:45.563390 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.581943 26048 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:45.566388  3045 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:45.560015  3077 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.532370  2860 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:45.589411 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.563889  3035 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.570752 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:45.516543  2809 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:27:45.578068 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.546216  3020 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:45.578187  2814 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:45.591828 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.542901  2965 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.596619 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.579577  3048 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:45.579887  2976 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.587313  3047 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.584106 26120 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:27:45.609203 26122 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:45.616864 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.579476  2821 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:27:45.584391  3033 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.576306  3081 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:45.612739 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84481.9us
I20260709 12:27:45.607349 26048 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:45.610167 26050 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.625602 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:45.626257 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.584014  3045 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:45.627185 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.627839 26788 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:45.628147 26046 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:45.627928  3045 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 14, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:45.628770 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 129969us
I20260709 12:27:45.629410  3045 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:45.599215  2811 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:45.583751  3025 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:45.630565  2811 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.631202  3025 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.582661  2910 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.633711  2910 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.634099  3025 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.645535 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:45.579648  3084 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:45.637183 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.634840 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.615623  2976 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.659262  2976 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:45.629462 26046 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:45.638548 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.664481 26791 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:45.640758  3059 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:45.666570  3059 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:45.667079  2864 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.732s	user 0.016s	sys 0.000s
I20260709 12:27:45.641139 26123 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:45.667649  3079 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:27:45.667862  2864 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:45.668586 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.669138  3090 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.669878 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.670336 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.670526 26788 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:45.670957 26045 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:45.671625 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.672708  3087 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.673465  3087 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.675796  3087 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.625769 26048 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:27:45.663652  2910 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
I20260709 12:27:45.677275 26048 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:45.678682 26048 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:45.679491 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.638700 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.667033  3059 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:45.680475 26048 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:45.681021 26788 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:45.644690 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:45.681751 26787 raft_consensus.cc:2376] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25. Re-sending same reply.
I20260709 12:27:45.681977 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.672737  3023 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.682657 26785 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:45.682731  3023 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.615919  3020 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:45.676410 26789 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:45.677166 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.668727  2864 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:45.669665  3090 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.678916 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:45.685130 26790 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:45.685199 26788 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:45.616901  2814 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.619045  2860 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:45.623144 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 52505.5us
I20260709 12:27:45.644845 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:45.643529 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.685590 26048 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:45.694819  2732 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.695647  3023 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.696148 26791 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:45.697218 26045 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:45.688611 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.684664  3020 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.701164 26202 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:27:45.620280  2886 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:45.678905 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:45.635835 26050 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:45.616961  3047 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.779392  2732 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.780393  2995 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.769099  3034 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:45.724004 26200 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:45.781829  3026 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:45.781647  3034 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.782272  3026 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:45.704797 26197 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:27:45.702759 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59603.3us
I20260709 12:27:45.719591 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:45.703980 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:45.707100 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:45.719661 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 217613us
I20260709 12:27:45.779639 26121 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:45.739567 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.786111 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:45.783828 26196 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:45.788349 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.665514  2769 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.703296  2864 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.801898 26019 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.705665  3077 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:45.793164 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 89284.3us
I20260709 12:27:45.725898  3036 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.708268  2800 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:45.791837 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 148374us
I20260709 12:27:45.817600 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.819365 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.821817 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.825639 26048 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:45.705706  3032 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:45.809056 26782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:46996
I20260709 12:27:45.785303 26046 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:45.806010 26790 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:27:45.840207 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.840528 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.841387 26046 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:45.821556 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:45.823810 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1274) took 1390 ms. Trace:
I20260709 12:27:45.781661  3047 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.828716 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.842294 26049 rpcz_store.cc:276] 0709 12:27:44.433651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.435864 (+  2213us) service_pool.cc:224] Handling call
0709 12:27:45.823578 (+1387714us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:45.823729 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79655296}
I20260709 12:27:45.843658 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.756470  2676 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:27:45.832607 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:45.847294 26202 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:45.803227  2769 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:45.764712  2818 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.752833  3092 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 12:27:45.831207 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:45.859673 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.848124 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.804045  3087 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:45.861937 26196 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 29 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:45.844868 26790 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:45.845773 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:45.845963 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.824720  2800 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:45.835209 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:45.826776  3038 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.826558  3089 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.850818 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.827107  3049 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.791925  2845 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:45.704928  3093 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.120s
I20260709 12:27:45.812577  3082 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.801048  3025 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:45.859612 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.881291 26049 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:45.882262 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.849187  3077 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:45.796593  2821 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:27:45.822305  3083 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.828377  2995 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:27:45.836778  3023 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:27:45.838963 26782 raft_consensus.cc:606] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:45.865150  3047 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:45.908380 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 206068us
I20260709 12:27:45.915855 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1504) took 1734 ms. Trace:
I20260709 12:27:45.865440 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:45.865667  2809 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:27:45.851955 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 206476us
I20260709 12:27:45.822068  3090 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:45.790004  2860 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:45.868482 26791 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:45.845868  2989 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:27:45.875622  2769 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:45.924692 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1883) took 1164 ms. Trace:
I20260709 12:27:45.925107 26789 rpcz_store.cc:276] 0709 12:27:44.760451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.760564 (+   113us) service_pool.cc:224] Handling call
0709 12:27:45.924529 (+1163965us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b5400565754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:45.924639 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2104051968}
I20260709 12:27:45.872071  2800 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:45.876747  2818 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.876930 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.872980 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.878873 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:45.838229  3032 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:45.883381 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:45.860785 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.812498  3078 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:27:45.860366  2814 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.859339  3081 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:45.827390  3036 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.867676  3035 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:45.863517 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:45.861797  3087 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:45.916338 26198 rpcz_store.cc:276] 0709 12:27:44.181653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.192005 (+ 10352us) service_pool.cc:224] Handling call
0709 12:27:44.192388 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.359488 (+167100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.359522 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.359567 (+    45us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:44.359608 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.915610 (+1556002us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400398d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:45.915665 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.915801 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383845632}
I20260709 12:27:45.917076 26783 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:45.921716 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:45.878830  3048 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 12:27:45.879367  3038 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.883198  3049 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:45.882627  3089 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.131585 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.012125 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.121069 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.135457 26203 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:46.135882 26201 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
W20260709 12:27:46.012183 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 177078us
W20260709 12:27:46.020092 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 148858us
W20260709 12:27:45.928649 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104162us
I20260709 12:27:46.138692 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:46.139024 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:45.991335 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68348.5us
I20260709 12:27:45.998196 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.115309 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1525) took 1293 ms. Trace:
I20260709 12:27:46.141361 26049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28"
dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:60878
I20260709 12:27:46.141284 26784 rpcz_store.cc:276] 0709 12:27:44.821663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.821804 (+   141us) service_pool.cc:224] Handling call
0709 12:27:46.115101 (+1293297us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54003101d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:46.115234 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417571200}
I20260709 12:27:46.142012 26049 raft_consensus.cc:606] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:46.142378 26049 raft_consensus.cc:614] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:46.117074 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.143193 26050 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:46.013330 26202 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: [27.18-29.19]
I20260709 12:27:46.133013  2860 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.932587 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.145133 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:46.145442 26790 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:45.914491  3047 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:46.146534  3047 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.149549  3050 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:45.971493  3077 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.150697  3077 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.151670 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.150159  3050 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.154024 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:46.141459 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.154551  3050 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:45.893151  3025 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:46.155206  3050 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:46.155653  3050 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.156435 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.157411 26198 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:46.156325  3025 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.158365 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:46.159137  3026 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.159441  3023 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:46.129122 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.161137 26194 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:27:46.151764 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.166320 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:46.172202 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.156026  3050 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:45.997857 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:46.173235 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:46.174350 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:46.168730 26046 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:46.195039 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:46.196720 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:46.197393 26128 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:46.198478 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:46.199632 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:46.200938 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 203158us
I20260709 12:27:46.206709 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.159193 26202 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:46.210585 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:46.211237 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:45.919422  2965 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:46.222029 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:46.214668 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:46.215377 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 336558us
I20260709 12:27:46.214808 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.222733 26123 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.216562 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:46.215971 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:46.223318 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:46.219619 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:46.217921 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.224108 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:46.223016 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:46.140658  3081 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.225777 26127 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:46.133780  3033 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:46.227084 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:46.172216  3087 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.995940  2795 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.919550  3097 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:45.970562  3090 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:46.220906 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.037352  3034 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.157413  2975 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:46.177480  3035 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:46.185689  2989 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:46.196290  2860 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:45.909281  3083 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.056963  2676 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.176s
I20260709 12:27:46.084767  3093 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.146s
I20260709 12:27:45.991137  3098 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.265s
I20260709 12:27:46.225728  2845 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:45.969905  3059 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.235023  3026 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:46.283176 26127 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:46.248510  3033 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.285463  3036 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.264608  3105 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:46.287922  3089 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.288620  2965 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:46.288798  3049 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.267143  2795 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.215103  3023 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 12:27:46.290529  2985 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:46.267653  2821 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:46.233888  3081 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:46.291627  3104 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:46.251260  2818 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.263651  3032 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:46.299532 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.214676  3045 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:46.312534 26123 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:46.232196  2897 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 12:27:46.328671 26128 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:46.325778  3035 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:46.361480 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 188009us
I20260709 12:27:46.470633 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:46.400381 26048 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:46.327720  2860 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:46.327461  3108 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:46.333745  2845 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:46.336390  2989 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:46.462900 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 267925us
I20260709 12:27:46.463158 26122 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.17-28.17]
W20260709 12:27:46.474488 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 335551us
W20260709 12:27:46.507676 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 345108us
I20260709 12:27:46.405280  2821 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:46.549829 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:46.550799 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 80267.4us
I20260709 12:27:46.410368  3106 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:46.554783 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 388491us
I20260709 12:27:46.462234  3045 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:46.557611 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 384455us
I20260709 12:27:46.560570 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:46.409883  3049 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.144s
I20260709 12:27:46.566311 26093 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:46.514787  3032 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:46.568897 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:46.511324  3103 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:46.577587 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 134811us
I20260709 12:27:46.553711  2821 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:46.518350  3108 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:46.461704  2800 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:46.580758 26200 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.009s	sys 0.018s
I20260709 12:27:46.550105  3028 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:46.586621 26200 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.009s	sys 0.018s
W20260709 12:27:46.582576 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62335.8us
I20260709 12:27:46.466502  2818 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.584411 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:46.588188 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:46.557919  2897 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:46.556960  3106 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:46.583420  3038 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.591795  3097 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:46.593252  2860 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.597146  2732 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.519315  2845 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:46.601212  3033 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.605603  2818 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.606369  3116 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:46.606045  3028 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.609450  2975 raft_consensus.cc:993] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: : Instructing follower a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:27:46.609802  3122 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:46.610415  3098 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:27:46.610970  3110 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
W20260709 12:27:46.626032 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.669461 26788 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:46.669844 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.579757  2965 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:46.614095  2897 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:46.722287 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 135041us
W20260709 12:27:46.593438  3109 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.321s	user 0.000s	sys 0.000s
W20260709 12:27:46.675490 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92127.1us
I20260709 12:27:46.677433 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:27:46.726817 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49534.9us
I20260709 12:27:46.727118 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:46.682433 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94343.4us
I20260709 12:27:46.589515  3107 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:46.590111  3108 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.705415 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.736629 26199 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:46.706915 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.707850 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:46.699443  3116 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.716814 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.678483  3106 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.719481 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.742835 26194 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:46.721499 26195 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
W20260709 12:27:46.675390 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 125751us
W20260709 12:27:46.729413 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.711481  3122 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.717206  2975 raft_consensus.cc:1081] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Signalling peer a981c9a7972c4e99a0903924c64fbb5c to start an election
I20260709 12:27:46.680176  3034 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:27:46.722848  2965 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:46.773006 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47397us
I20260709 12:27:46.768420 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:46.725617  2860 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:46.774582 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:46.728384  2897 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:46.779834 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.677934  3045 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.756480  2732 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:46.678352  2985 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.736913  3107 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.743902  3116 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:46.766852  3122 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:46.784832  3092 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.008s	sys 0.005s
I20260709 12:27:46.777081  3048 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:46.785025  2864 consensus_meta.cc:287] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.969s	user 0.008s	sys 0.014s
I20260709 12:27:46.777462  2965 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.801514 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.803653 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:46.717041  2989 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.812081 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.816499 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.776130  2603 consensus_queue.cc:1059] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.030s
I20260709 12:27:46.794404  3059 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:46.810664 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 81394.7us
I20260709 12:27:46.819948 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.824138  2800 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:46.814147  2864 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.824966  2800 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:46.825513  2800 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:46.825917  3047 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.003s
I20260709 12:27:46.826444  3047 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.788312  3049 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:27:46.828348  2795 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.801599  2732 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:46.832469  3115 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:46.795677  3107 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:46.833206  2732 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.807444  2936 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:46.836416  3119 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:46.837049  3119 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:46.838023  3119 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.839641  3119 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.801416  3083 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.802472  2975 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:46.840898  2975 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:46.841296 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:46.797729  2985 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.842545 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:46.843465  3114 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:27:46.843561 26202 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:46.804736  3092 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.009s	sys 0.005s
I20260709 12:27:46.850468  3125 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:46.851425 26199 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:46.836544  2936 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.852186 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:46.817881  3082 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.926s	user 0.007s	sys 0.013s
I20260709 12:27:46.843776  2985 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.861994  2936 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:46.867843  3028 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:46.853207  3087 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
I20260709 12:27:46.869098  3087 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.870148 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.870489 26789 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:46.871150 26786 raft_consensus.cc:2410] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:27:46.871742  3087 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:46.826454  2800 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.872694  3117 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:46.873574  2800 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:46.873456  3135 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:46.872611 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:46.876891 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2204) took 1063 ms. Trace:
I20260709 12:27:46.877329 26789 rpcz_store.cc:276] 0709 12:27:45.808560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.925901 (+117341us) service_pool.cc:224] Handling call
0709 12:27:45.926227 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.870863 (+944636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.872007 (+  1144us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:46.872214 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171147136}
W20260709 12:27:46.887360 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 142531us
I20260709 12:27:46.841331  2975 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:46.902983 26167 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.767s (SENT)
I20260709 12:27:46.903698 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:46.899338 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260709 12:27:46.887434  3109 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:27:46.877010  3082 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:46.916225  3097 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:46.896275 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:46.925880  3087 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:46.926398  3087 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.926617 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.928282 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.930553 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.916812  3047 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:46.926865  3087 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:46.876622  2986 consensus_queue.cc:1175] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 29 status { last_received { term: 29 index: 21 } last_committed_idx: 21 last_received_current_leader { term: 29 index: 21 } }
I20260709 12:27:46.934285  3137 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:46.931896 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.935407  3044 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:46.935549 26046 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:46.935168  3137 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.937868  3060 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:46.939214  3060 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.939689  3137 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.940054 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:46.940974 26046 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:46.941452  3137 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.941900 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:46.943267 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.948732 26168 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:46.941447 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.892375  2915 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:46.953384 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:46.851203  3125 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:46.954952 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.935978  3044 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:46.956779 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.961952 26783 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:46.962911 26129 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 31 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:27:46.958004 26052 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:27:46.964076 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1285) took 1312 ms. Trace:
I20260709 12:27:46.964133 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.965715 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:46.964497 26052 rpcz_store.cc:276] 0709 12:27:45.651985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.957428 (+1305443us) service_pool.cc:224] Handling call
0709 12:27:46.957772 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.963816 (+  6044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.964014 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:46.966714 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:46.970122 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.971359 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.974987 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:46.977061 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.977942 26047 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:46.980871 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.980924 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:46.982685 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:46.984879 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:46.986853 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:46.987514 26047 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:46.992425  3133 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:46.963645  2499 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:46.942132  3137 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.001897 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.001941 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:47.004427  3123 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.008473  3047 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:47.009145  3047 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.011510  3047 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.014787  3077 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.017221  3125 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:46.974555  3059 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:46.970726  3082 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.023701  2703 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:47.025548  3100 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:47.025988  3100 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:47.026311  3026 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.003s
I20260709 12:27:47.026492  3116 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.027379 26046 raft_consensus.cc:2410] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:27:47.030575 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:47.030398  3107 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.031229 26203 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:47.031133  2897 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.033705  3137 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.033453  3044 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.000429  3032 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:47.034837  3044 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.034759  3032 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.038657 26052 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:47.038976 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:47.026870  3100 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.000013 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:47.032017  3048 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:47.042265 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:47.067008 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:46.981659  2915 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:27:47.068681 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:47.043432  3035 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.002s
I20260709 12:27:47.036623  3059 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:47.080178 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:47.041317  2499 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:47.048609  3119 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:47.002214  3097 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:27:47.084492 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:47.008092  3050 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.791s	user 0.008s	sys 0.000s
W20260709 12:27:47.088454 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 118231us
I20260709 12:27:47.078459  3047 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:47.083765  3034 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:47.042526  2845 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.101455 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.102190 26783 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:47.088617 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.086860  3100 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.084087  3059 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.041478  3082 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.088410  3119 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.067705  3106 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.097894 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.085835  2795 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:47.103598 26201 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:47.103063 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.094390  3028 raft_consensus.cc:2764] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:27:47.088115  2910 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.104127 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:47.083941  3035 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.092875  3133 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:47.093043  3107 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.139588 26125 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:47.075714  3122 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.115851 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.118088 26783 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.089159  3048 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:47.146633  3048 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.147112  3048 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:47.147714  3035 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.148702 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.149391  2845 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.150012 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.119844  2897 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.090585  3108 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.150795 26129 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:47.151391  2936 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.151907 26129 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:27:47.113619  3119 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.127736  2769 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:47.098400  3116 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.078914  2821 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.152340  3108 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.152555  2499 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.158093  3082 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.135250  3089 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.158700  2821 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.128420  3129 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:47.159130  3089 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.159607  3129 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:47.160625  3129 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.161586  3089 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.161885  3129 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:47.133482  2814 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:47.153829 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1536) took 1475 ms. Trace:
I20260709 12:27:47.116954  3047 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:47.096964  2983 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.000s	sys 0.016s
I20260709 12:27:47.098760  3034 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.104346 26782 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:47.094172  3050 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:47.145869  3081 consensus_meta.cc:287] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.679s	user 0.006s	sys 0.004s
I20260709 12:27:47.146101  3113 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:47.120213  3097 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:47.133484  2910 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.133279  2886 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.169543 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.181747 26201 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:47.183009 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.184469  2910 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.163213 26785 rpcz_store.cc:276] 0709 12:27:45.678308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.683524 (+  5216us) service_pool.cc:224] Handling call
0709 12:27:45.683794 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.998933 (+315139us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:46.060689 (+ 61756us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.548391 (+487702us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.577113 (+ 28722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:46.577168 (+    55us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:46.577221 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.033552 (+456331us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b18003daf20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1
0709 12:27:47.034983 (+  1431us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b18003daf20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:47.153582 (+118599us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400650254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:47.153636 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.153773 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56233,"prepare.queue_time_us":64401,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":88,"raft.queue_time_us":31846,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":72185,"spinlock_wait_cycles":884102656,"thread_start_us":406118,"threads_started":1,"wal-append.queue_time_us":546287}
I20260709 12:27:47.162688  2769 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:47.174897 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:47.190868 26202 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:47.176054  3081 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.153863  2936 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:47.179181  2975 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.202303  3133 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.162519  2814 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.215515  2814 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:47.220275 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.224651 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2202) took 1423 ms. Trace:
I20260709 12:27:47.225301 26788 rpcz_store.cc:276] 0709 12:27:45.801020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.802703 (+  1683us) service_pool.cc:224] Handling call
0709 12:27:45.825923 (+ 23220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.722652 (+896729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.224331 (+501679us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b540063abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:47.224573 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938099968}
I20260709 12:27:47.228283  3106 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.228915  3032 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.229388 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.231173  3048 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.195605  3087 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.235491 26790 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26.
I20260709 12:27:47.200268  2886 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.176266  2965 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.238605  3104 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:47.239068  3104 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:47.239177  2886 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.195859  2769 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.204552  3119 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.240121  3087 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.199916  3081 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.169364  3034 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.240883  2769 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.231547  2864 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.237354 26789 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:27:47.199262  3122 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:47.169046  2983 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.000s	sys 0.017s
I20260709 12:27:47.243624  3048 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.238565  3045 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:47.240202  2965 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.242879  3050 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.242990  3119 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.239943  3104 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.256780  3104 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:47.262147  3059 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.264034  3059 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.129432 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:47.401660 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:47.403085 26045 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:47.106988 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:27:47.325534 26053 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:47.330355 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:47.409300 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:47.410220 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 80441.8us
I20260709 12:27:47.142346 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:47.143157 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.412685 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.414435 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.415337 26120 raft_consensus.cc:2393] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:47.144668 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.136339 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.143851 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.141398 26124 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:47.138985 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.342554 26125 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:47.416942  3122 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.426321  3122 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.426931  3122 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:47.418633 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 273917us
I20260709 12:27:47.428450  2811 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:47.343513 26129 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:47.432812 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.433269 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.434006 26791 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.361644 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:47.184157 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.436057 26201 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
W20260709 12:27:47.436564 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75422.7us
I20260709 12:27:47.437948 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:47.438535 26201 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:47.420053 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.440496 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.441766 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.442950 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.365486 26196 raft_consensus.cc:2368] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29.
I20260709 12:27:47.373775 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.376204 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.378005 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.444166 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.475775 26787 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.380343 26789 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:47.381023  2936 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:47.467862  2975 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:47.449610 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:47.382687 26790 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:47.389986  3083 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.482215  3083 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.390329  3076 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.126464 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:47.318126 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.493986 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:47.410542 26045 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:47.134436 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:47.409302 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 302220us
I20260709 12:27:47.416209 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.418319 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.444329  2886 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:47.379336  2499 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.448446  3156 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.513145  3156 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.513258  3092 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:47.513827  3156 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.513675 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:47.514248 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 69522.4us
I20260709 12:27:47.515079 26120 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:47.446472 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.516645  3025 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:47.517778 26124 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:47.479701 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.519213 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.481295 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.520658  2811 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:47.521313  2811 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:47.522908  2811 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:47.459688 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4286) took 1263 ms (client timeout 1587 ms). Trace:
I20260709 12:27:47.525143 26045 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
W20260709 12:27:47.487046 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.484537 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:47.465375 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1891) took 1817 ms (client timeout 1767 ms). Trace:
I20260709 12:27:47.454892 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.459136 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:47.432024 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:47.502101 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72674.5us
I20260709 12:27:47.444525  3059 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:47.517222 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.518321  2886 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.520110  2499 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.434171 26129 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:47.475804 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
W20260709 12:27:47.530742 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.572382 26785 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:47.526013 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.542701 26200 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:47.540398  3133 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:47.574887 26046 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:47.519270  3025 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.056s
I20260709 12:27:47.576509  3025 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:47.576965  3025 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:47.577059 26046 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:47.578050 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1288) took 1744 ms. Trace:
W20260709 12:27:47.526093  3132 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.659s	user 0.004s	sys 0.001s
I20260709 12:27:47.578577 26046 rpcz_store.cc:276] 0709 12:27:45.833834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.575870 (+1742036us) service_pool.cc:224] Handling call
0709 12:27:47.576510 (+   640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.577659 (+  1149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.577944 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:47.532421  3076 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.581030 26784 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:47.581971 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2212) took 1695 ms. Trace:
I20260709 12:27:47.582207  3076 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.583027  2982 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.582401 26784 rpcz_store.cc:276] 0709 12:27:45.886316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.142181 (+255865us) service_pool.cc:224] Handling call
0709 12:27:46.206120 (+ 63939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.580562 (+1374442us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54005cea20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:27:47.581603 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.581882 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455563136}
I20260709 12:27:47.583321  3059 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:47.584004  2982 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.584771 26126 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:47.526067 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2849) took 1704 ms. Trace:
W20260709 12:27:47.587646 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.582820 26195 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.526882 26049 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
W20260709 12:27:47.569240 26786 rpcz_store.cc:269] 0709 12:27:45.638308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.638443 (+   135us) service_pool.cc:224] Handling call
0709 12:27:47.455587 (+1817144us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54004f4554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:47.455931 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2736961792}
I20260709 12:27:47.586637 26045 rpcz_store.cc:276] 0709 12:27:45.821192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.524293 (+1703101us) service_pool.cc:224] Handling call
0709 12:27:47.524848 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:47.525689 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.525962 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:47.591648 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 75084us
W20260709 12:27:47.524430 26127 rpcz_store.cc:269] 0709 12:27:46.195557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.198145 (+  2588us) service_pool.cc:224] Handling call
0709 12:27:47.428225 (+1230080us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b54002ca1d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:47.459411 (+ 31186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61036928}
I20260709 12:27:47.566975 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:47.599114 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.545446 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.599150 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2848) took 1791 ms. Trace:
I20260709 12:27:47.542613  3083 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:47.541486  3138 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.675s	user 0.003s	sys 0.000s
I20260709 12:27:47.531220  3119 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:47.602847 26049 rpcz_store.cc:276] 0709 12:27:45.805192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.500832 (+1695640us) service_pool.cc:224] Handling call
0709 12:27:47.526437 (+ 25605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.595318 (+ 68881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.596641 (+  1323us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:47.609489 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.604779 26046 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:47.615222 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.624485 26783 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.625464 26200 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:47.628556  2975 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:47.629591 26125 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 24: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:47.533502  2936 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.096s
I20260709 12:27:47.630045 26120 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:47.630538  2936 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:47.630955  2936 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:47.631805 26120 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:27:47.633224 26045 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 16. (term mismatch)
W20260709 12:27:47.636916 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 187530us
I20260709 12:27:47.644428 26045 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 15
I20260709 12:27:47.624567 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:47.647171 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2851) took 1474 ms. Trace:
I20260709 12:27:47.647552 26046 rpcz_store.cc:276] 0709 12:27:46.143677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.579834 (+1436157us) service_pool.cc:224] Handling call
0709 12:27:47.598726 (+ 18892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.616806 (+ 18080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.617533 (+   727us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:47.640591 26049 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:27:47.646044 26200 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:47.641532 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.644953 26045 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 29.16
I20260709 12:27:47.653148 26045 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:47.638834  2499 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.654696 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:47.533502  3033 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.114s
I20260709 12:27:47.658859  3033 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:47.659441  3033 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:47.660995  3033 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.662882  3033 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:47.667157 26201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:53816
I20260709 12:27:47.668202 26201 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:47.668632 26201 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.668855  2499 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:47.669781  2499 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.632201  2936 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:47.675784 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46127.9us
I20260709 12:27:47.578433  3025 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.625483  2811 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.667057  3092 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:47.632304  2886 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.693238  2936 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:47.693845 26202 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.011s
I20260709 12:27:47.694288 26202 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:47.625972  3122 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.673599  2982 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.702965 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2853) took 1558 ms. Trace:
I20260709 12:27:47.705411 26045 rpcz_store.cc:276] 0709 12:27:46.160915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.601694 (+1440779us) service_pool.cc:224] Handling call
0709 12:27:47.632683 (+ 30989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.701416 (+ 68733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.702905 (+  1489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:47.696681  2499 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.641134  2910 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.717455 26199 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: [22.19-24.20]
I20260709 12:27:47.718082  3035 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.718755 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1526) took 1099 ms. Trace:
I20260709 12:27:47.718863  3035 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.641413  3059 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.719188 26198 rpcz_store.cc:276] 0709 12:27:46.619622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.678273 (+ 58651us) service_pool.cc:224] Handling call
0709 12:27:46.679542 (+  1269us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:46.725337 (+ 45795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.725379 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:46.762073 (+ 36694us) log.cc:844] Serialized 78 byte log entry
0709 12:27:46.818956 (+ 56883us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:46.877711 (+ 58755us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.878276 (+   565us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.879153 (+   877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.879292 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:47.003841 (+124549us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b48000b1b58. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:27:47.473067 (+469226us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18002e1740. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:47.477523 (+  4456us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b18002e1740. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:47.489759 (+ 12236us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:47.718244 (+228485us) raft_consensus.cc:1685] finished
0709 12:27:47.718271 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.718540 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46310,"prepare.queue_time_us":53905,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":58120,"raft.run_cpu_time_us":2731,"raft.run_wall_time_us":151594,"spinlock_wait_cycles":54590080,"thread_start_us":25242,"threads_started":1,"wal-append.queue_time_us":86506}
W20260709 12:27:47.722985 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73211.4us
I20260709 12:27:47.723683  3083 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:47.724908  3083 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:47.724812 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.725404 26122 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:47.726410  3078 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:47.727365 26122 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:27:47.729988 26167 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26.
I20260709 12:27:47.730367 26167 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:47.731611  2975 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.731956  3106 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.713763  3034 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:47.733948 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 137953us
I20260709 12:27:47.734965  3034 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.736187  3076 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:47.736869 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.737519 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.737995  2982 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:47.737804 26785 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.738422  2982 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:47.738730 26127 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: [26.21-26.21]
I20260709 12:27:47.739668  3081 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.732287  2975 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.738467 26786 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.735917  3034 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:47.716679 26201 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.739905 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.739718  2982 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:47.750982  3129 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.588s	user 0.005s	sys 0.019s
I20260709 12:27:47.751199 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:47.750448  3081 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:47.748805 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 59641.7us
I20260709 12:27:47.750488 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.755755 26122 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:27:47.758564  2975 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.754706 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.760429 26128 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:47.760564 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.751032  2982 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.693614  3044 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.071s
I20260709 12:27:47.761790 26783 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.762408  3044 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:47.762828  3044 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:47.764392  3081 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.764587  3038 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.766101 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.766498  3122 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.767674 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.755949 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.770762  3122 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:47.771103 26195 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:47.771320  3122 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:47.772493 26122 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:47.774082 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.774708 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.776700  3090 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.776472  3156 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.777760 26198 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:47.779057  3156 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:47.725396  3083 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:47.779263  3122 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.774989 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.779848 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.691712  3119 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.790954 26122 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:47.792606 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:47.764153 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:47.780804  3038 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.796139 26045 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:27:47.762845  3045 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.012s
I20260709 12:27:47.794085 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.803787 26783 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:47.804782 26201 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:47.806722 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.809417 26200 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:27:47.809835 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.810474 26195 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:47.811278 26047 raft_consensus.cc:2410] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 30 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:27:47.811493 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.795879  3081 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:47.809161 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:47.806186 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 261044us
I20260709 12:27:47.818312  2982 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:47.818969  2982 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.819743  2982 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:47.763980  3044 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.812168  3034 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:47.823244 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.834267 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:47.834520  3092 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:47.835353 26045 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:47.836017  2983 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:47.836573 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:47.836856  3092 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:47.837265  3092 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:47.837435 26045 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:47.838407  3092 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:47.839438 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 59660.1us
I20260709 12:27:47.840894 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:47.841893  3090 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.842644 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:47.843664  3106 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 25. Current term is 26.
I20260709 12:27:47.780856  3083 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.815377  3081 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:47.818944  2830 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.863624 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.846261 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:47.839993  3092 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:47.829818  3045 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.816794  3035 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.872176 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:47.833308  3034 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.875286  3125 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.875442  3034 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:47.875599  3044 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:47.869825  3090 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:47.869508  3081 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:47.879870 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:47.906402 26045 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.908877 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2943) took 1183 ms. Trace:
W20260709 12:27:47.881822  3162 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:27:47.909943 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.910586 26784 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.888427  3156 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:47.912194  3156 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:47.912051  3045 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.912596  3156 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:47.913565  3156 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.914673  3045 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.914976  3156 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:47.889616  3090 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:47.891806 26195 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:47.922866 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:47.923682 26195 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:47.896521 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:47.925163 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45529.1us
I20260709 12:27:47.925333 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.926002 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:47.892511 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 128500us
W20260709 12:27:47.896701 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 157074us
I20260709 12:27:47.828544  3038 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.899070  2910 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.899919 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.900827  3083 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:47.901785  2769 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.017s
I20260709 12:27:47.904912 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.905400 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:47.926079 26127 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:47.892576 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
W20260709 12:27:47.935063 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 60971.4us
I20260709 12:27:47.922816  3090 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.936731 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:47.937525 26127 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:47.938356  3106 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:47.924116  3119 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:47.909344 26194 rpcz_store.cc:276] 0709 12:27:46.725269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.576321 (+851052us) service_pool.cc:224] Handling call
0709 12:27:47.576893 (+   572us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:47.577607 (+   714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.577639 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:47.578535 (+   896us) log.cc:844] Serialized 52 byte log entry
0709 12:27:47.713828 (+135293us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:47.755231 (+ 41403us) log.cc:844] Serialized 14 byte log entry
0709 12:27:47.780732 (+ 25501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.780825 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:47.908037 (+127212us) raft_consensus.cc:1685] finished
0709 12:27:47.908067 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.908227 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25216,"prepare.queue_time_us":41349,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":67,"raft.queue_time_us":33762,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":4801,"thread_start_us":127337,"threads_started":1,"wal-append.queue_time_us":299207}
W20260709 12:27:47.942827 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.944751 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:47.948055  3119 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.948704  3119 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.950881  3119 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:47.952889  3087 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:47.953321  3087 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:47.953698  3087 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.954468 26123 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:47.957118 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1284) took 2306 ms. Trace:
I20260709 12:27:47.957490 26048 rpcz_store.cc:276] 0709 12:27:45.651009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.214771 (+563762us) service_pool.cc:224] Handling call
0709 12:27:46.235700 (+ 20929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.507519 (+271819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.956622 (+1449103us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540006fc54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:47.956921 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378228992}
I20260709 12:27:47.958844 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:47.962961  3081 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:47.966141 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4202) took 1353 ms (client timeout 1599 ms). Trace:
W20260709 12:27:47.966634 26053 rpcz_store.cc:269] 0709 12:27:46.613090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.869785 (+256695us) service_pool.cc:224] Handling call
0709 12:27:47.965924 (+1096139us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b540022c754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:47.966058 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043998592}
I20260709 12:27:47.967303  3034 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:47.969550 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:47.970422 26783 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:47.582049 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:47.976393 26786 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:47.971571 26195 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:47.972854 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.973932 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:47.974025 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:47.973079 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:48.001901 26784 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:47.972648 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2855) took 1011 ms. Trace:
I20260709 12:27:48.006503 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:48.014026 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:48.004671 26047 rpcz_store.cc:276] 0709 12:27:46.961592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.999751 (+ 38159us) service_pool.cc:224] Handling call
0709 12:27:47.972408 (+972657us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400030054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:47.972569 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762996608}
I20260709 12:27:48.015379 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:47.972020  2965 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:48.019234  3081 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.019706 26201 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:27:48.019599  3045 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:48.020537  3045 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.020984  3045 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:48.021317 26201 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:48.023672 26194 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:48.025902 26194 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:27:48.026399  2965 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:48.026933  2965 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.027313  2965 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:48.028818  3048 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:48.029641  3038 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.029821  3048 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:48.030282  3048 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:48.031343  3048 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.033174  3048 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:48.035913 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.975510  3116 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:48.036652 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.036825 26048 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.967605  2982 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:47.972311  3125 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:48.044727 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 118898us
I20260709 12:27:48.054706 26200 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:27:48.017727  3119 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:48.057534 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:48.060942 26123 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:48.039670  3125 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:48.065987 26054 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:48.066748 26046 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:27:48.070122 26195 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:27:48.074324 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1534) took 1029 ms. Trace:
I20260709 12:27:48.074831 26195 rpcz_store.cc:276] 0709 12:27:47.041733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.008791 (+967058us) service_pool.cc:224] Handling call
0709 12:27:48.058570 (+ 49779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.070657 (+ 12087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.071095 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.066357  3125 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:48.079756 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 134267us
W20260709 12:27:48.083043 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.082983 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:48.084398 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:48.090504 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.092772 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.105121  3081 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.108781  3081 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.111043  3081 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.112085  3119 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:48.112385  3044 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.113082  3044 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.085608  3059 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.114660  3059 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.117627  3059 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:48.118115 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.119715 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.120342 26123 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:27:48.107872 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 166153us
I20260709 12:27:48.123494 26045 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:48.124671 26053 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
W20260709 12:27:48.129987 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47163.3us
I20260709 12:27:48.102929  2830 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:48.132701  2830 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.135746  2830 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.153517 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:48.141585 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:48.164484 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.154168 26790 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:48.066474  3116 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.183343 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:48.174366 26126 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:48.185491 26048 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:48.150184  3156 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.185811 26168 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:48.202273 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:48.188023  3092 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:27:48.205350 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56752.8us
W20260709 12:27:48.204056 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 332473us
W20260709 12:27:48.208191 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.209661 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:48.213160 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 145382us
W20260709 12:27:48.219096 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.216492 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:48.199371  3034 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.204927  2982 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.222630 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3267) took 1346 ms. Trace:
I20260709 12:27:48.208912  2769 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:48.226771 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:48.204631  2830 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:48.237895 26048 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:48.238826 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1293) took 1279 ms. Trace:
I20260709 12:27:48.239248 26048 rpcz_store.cc:276] 0709 12:27:46.959331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.194701 (+1235370us) service_pool.cc:224] Handling call
0709 12:27:48.221324 (+ 26623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.238503 (+ 17179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.238770 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.223050  3092 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:48.260049 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 117374us
I20260709 12:27:48.235605  2499 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:48.226243 26197 rpcz_store.cc:276] 0709 12:27:46.718872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.537433 (+818561us) service_pool.cc:224] Handling call
0709 12:27:47.538116 (+   683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:47.539161 (+  1045us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:47.539680 (+   519us) log.cc:844] Serialized 14 byte log entry
0709 12:27:47.728545 (+188865us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:47.733258 (+  4713us) log.cc:844] Serialized 26 byte log entry
0709 12:27:47.733443 (+   185us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:48.062768 (+329325us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.065014 (+  2246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.065116 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.065158 (+    42us) raft_consensus.cc:1685] finished
0709 12:27:48.065173 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.065322 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40740,"prepare.queue_time_us":35464,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":77,"raft.queue_time_us":22034,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":373,"thread_start_us":475947,"threads_started":2,"wal-append.queue_time_us":260735}
W20260709 12:27:48.258234  3160 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:27:48.303311 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44519.4us
I20260709 12:27:48.314193 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4209) took 1444 ms. Trace:
I20260709 12:27:48.273032  3119 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:48.322875 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.325038 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.326584 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:48.240924  3156 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.313130  2499 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:48.312601  3129 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:48.344570 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 145999us
I20260709 12:27:48.317010 26049 rpcz_store.cc:276] 0709 12:27:46.859917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.660994 (+801077us) service_pool.cc:224] Handling call
0709 12:27:47.661577 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.662095 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:48.184288 (+522193us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.303565 (+119277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:48.303626 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:48.303686 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.303780 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.303919 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52098,"prepare.queue_time_us":600952,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":63,"raft.queue_time_us":9049,"thread_start_us":539480,"threads_started":2}
I20260709 12:27:48.349725  3101 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.350422  3101 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.352352 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4184) took 3233 ms. Trace:
I20260709 12:27:48.352840 26051 rpcz_store.cc:276] 0709 12:27:45.026748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.067787 (+ 41039us) service_pool.cc:224] Handling call
0709 12:27:45.083077 (+ 15290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:45.089868 (+  6791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:45.089916 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.090537 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:27:47.494020 (+2403483us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b180039b500. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:27:47.649969 (+155949us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:48.103039 (+453070us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.125442 (+ 22403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.187560 (+ 62118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.187609 (+    49us) raft_consensus.cc:1685] finished
0709 12:27:48.214886 (+ 27277us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.260315 (+ 45429us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75338,"prepare.queue_time_us":568126,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":25252,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":12744,"spinlock_wait_cycles":6812032,"thread_start_us":478870,"threads_started":2,"wal-append.queue_time_us":2547880}
I20260709 12:27:48.359974 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:48.360890 26051 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:48.360956  3176 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.319698  3039 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:48.368974 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2859) took 1348 ms. Trace:
I20260709 12:27:48.369431 26051 rpcz_store.cc:276] 0709 12:27:47.020197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.360179 (+1339982us) service_pool.cc:224] Handling call
0709 12:27:48.360633 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.368660 (+  8027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.368912 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:48.370157 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 465051us
I20260709 12:27:48.368965 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:48.379253  3116 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.380631  3178 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:48.376596 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: true dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:48.385426 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1532) took 1509 ms. Trace:
I20260709 12:27:48.386087 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:48.387133 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:48.385951 26203 rpcz_store.cc:276] 0709 12:27:46.875542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.687215 (+811673us) service_pool.cc:224] Handling call
0709 12:27:47.781278 (+ 94063us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:47.828791 (+ 47513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.828831 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:47.871637 (+ 42806us) log.cc:844] Serialized 52 byte log entry
0709 12:27:48.215550 (+343913us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:48.215610 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.215712 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.384935 (+169223us) raft_consensus.cc:1685] finished
0709 12:27:48.384958 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.385181 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12767,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":1403,"thread_start_us":314076,"threads_started":1,"wal-append.queue_time_us":504764}
I20260709 12:27:48.392791  2499 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.394133  3116 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:48.394609  3116 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:48.400823 26049 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:48.401633 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2857) took 1429 ms. Trace:
I20260709 12:27:48.401984 26049 rpcz_store.cc:276] 0709 12:27:46.971903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.352703 (+1380800us) service_pool.cc:224] Handling call
0709 12:27:48.400615 (+ 47912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.401355 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.401573 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.406486  2910 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:48.407119  2910 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:48.408044  2910 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.363754  3034 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:48.412597  3034 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:48.413076  3034 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:48.416049  3092 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.421280 26127 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:48.427114 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:48.434919 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:48.327761  3090 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
W20260709 12:27:48.445822 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.395618  3116 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.480842 26127 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:27:48.484014 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.427532  2769 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:48.486140 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 58319.2us
W20260709 12:27:48.491707 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 188485us
I20260709 12:27:48.492588 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:48.501766 26127 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:48.504814 26051 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:27:48.485237  3025 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.376516  3176 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.505270 26051 pending_rounds.cc:85] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 12
W20260709 12:27:48.510047 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.514427 26051 pending_rounds.cc:107] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:27:48.514739 26051 raft_consensus.cc:2889] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:48.496699  3087 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.004s
I20260709 12:27:48.492295  3101 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:48.497982  2811 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:48.508935  3083 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.003s
I20260709 12:27:48.534917 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2861) took 1514 ms. Trace:
I20260709 12:27:48.535497 26051 rpcz_store.cc:276] 0709 12:27:47.020863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.437637 (+1416774us) service_pool.cc:224] Handling call
0709 12:27:48.494029 (+ 56392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.529034 (+ 35005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.534833 (+  5799us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:48.542990 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50496.6us
I20260709 12:27:48.543325  3189 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.526085  3125 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.521643  2965 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:48.556842 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3257) took 1852 ms (client timeout 1704 ms). Trace:
I20260709 12:27:48.557590  2975 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:48.558090  2975 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.558480  2975 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.559957 26201 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: [26.17-28.18]
W20260709 12:27:48.547654 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 102604us
W20260709 12:27:48.557335 26196 rpcz_store.cc:269] 0709 12:27:46.704544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.707533 (+  2989us) service_pool.cc:224] Handling call
0709 12:27:48.556511 (+1848978us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54002b4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:48.556734 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440772992}
I20260709 12:27:48.528784  3122 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.522895  3087 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.574362 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:48.561932  3189 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:48.576629 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61094.4us
I20260709 12:27:48.578163  3036 consensus_queue.cc:1059] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:27:48.578704  3189 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.585929  3044 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.597451  3122 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:48.651571 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.652467 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4207) took 1950 ms. Trace:
I20260709 12:27:48.664551 26046 rpcz_store.cc:276] 0709 12:27:46.702215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.675787 (+973572us) service_pool.cc:224] Handling call
0709 12:27:47.714301 (+ 38514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.069359 (+355058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.652142 (+582783us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400228654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:48.652407 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803985152}
W20260709 12:27:48.653771 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.608362  3188 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.659286  2995 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.563406  2769 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.573319 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:48.575325 26051 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:27:48.662943  3133 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:48.660091  3187 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.555958  3034 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.680224 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:48.680583 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:48.681453 26201 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:48.676584 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:48.681453 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:48.683058 26046 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:48.681282 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:48.683782 26200 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:48.677399  3045 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.656s	user 0.016s	sys 0.000s
I20260709 12:27:48.715839 26197 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:48.718811 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:48.719516 26128 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:48.676826  3048 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.686762  2995 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.723531 26203 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:48.683837  3125 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.682669  2965 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:48.683609  3197 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.728322 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:48.687043  2830 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:48.730399 26196 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:27:48.684134  3156 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.697649  3199 raft_consensus.cc:2764] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:48.678061  3188 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.733764  3198 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:27:48.730698  3087 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.720623  3045 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.754622 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:48.683732  3187 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.756373 26203 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:27:48.759455  3093 raft_consensus.cc:3055] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:48.759714  3122 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.759941  3093 raft_consensus.cc:740] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:48.764155 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:48.765158 26050 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:48.766541 26050 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:27:48.767125 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2862) took 1730 ms. Trace:
I20260709 12:27:48.767442 26050 rpcz_store.cc:276] 0709 12:27:47.037047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.765800 (+1728753us) service_pool.cc:224] Handling call
0709 12:27:48.766207 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.766850 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.767050 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.768383  3201 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:48.769935  3185 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.770642 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2941) took 2045 ms. Trace:
I20260709 12:27:48.770488  3185 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.771008 26199 rpcz_store.cc:276] 0709 12:27:46.724834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.543724 (+818890us) service_pool.cc:224] Handling call
0709 12:27:47.703291 (+159567us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:47.717939 (+ 14648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.770417 (+1052478us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400335750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:48.770561 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350570240}
I20260709 12:27:48.772868  2995 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:48.773203  3093 consensus_queue.cc:260] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.774608  3093 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:48.778395 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3301) took 1641 ms. Trace:
W20260709 12:27:48.779176  3194 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:48.779881 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1296) took 1819 ms. Trace:
I20260709 12:27:48.780359 26052 rpcz_store.cc:276] 0709 12:27:46.960289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.001062 (+ 40773us) service_pool.cc:224] Handling call
0709 12:27:48.779490 (+1778428us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b54001b3fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:48.779703 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81592704}
I20260709 12:27:48.781080  2975 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.778880 26121 rpcz_store.cc:276] 0709 12:27:47.137095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.137351 (+   256us) service_pool.cc:224] Handling call
0709 12:27:47.145738 (+  8387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.341323 (+195585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.341380 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:47.341445 (+    65us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:47.341490 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.778018 (+1436528us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54000b95d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:48.778064 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.778202 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447963520}
I20260709 12:27:48.783629 26050 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:48.785447  3185 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.786712 26046 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
W20260709 12:27:48.789724 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.790578 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.784870 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2864) took 1747 ms. Trace:
I20260709 12:27:48.793066 26050 rpcz_store.cc:276] 0709 12:27:47.037589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.768188 (+1730599us) service_pool.cc:224] Handling call
0709 12:27:48.782933 (+ 14745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.784470 (+  1537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.784788 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.793886 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:48.797334 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 229378us
I20260709 12:27:48.783512  2975 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.801659 26756 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:48.793062 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:48.728915  3133 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.748978  3048 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.793227  3034 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.809906 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:48.812952 26203 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:27:48.811908 26789 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:48.839969 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:48.811951 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:48.841043 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4217) took 1118 ms. Trace:
I20260709 12:27:48.849398 26050 rpcz_store.cc:276] 0709 12:27:47.722891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.794379 (+1071488us) service_pool.cc:224] Handling call
0709 12:27:48.806109 (+ 11730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.836641 (+ 30532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.836689 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:48.840746 (+  4057us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:48.840790 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.840858 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.840987 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.845300  3045 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:48.852459 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 62941us
I20260709 12:27:48.811272 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:48.840891  2975 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.748894  3197 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.847200  3187 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:48.812806  3106 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:48.861843  2936 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:48.863169 26050 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:27:48.865628 26126 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:48.858999  2975 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.868856  2975 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.870177  3133 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.837821  3198 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:48.812278  2811 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:48.808306  3199 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:48.873908 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1300) took 1102 ms. Trace:
I20260709 12:27:48.877425  3048 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:48.878266  3093 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.878423  2975 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:48.880686 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 82797.7us
I20260709 12:27:48.885850 26050 rpcz_store.cc:276] 0709 12:27:47.771486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.850096 (+1078610us) service_pool.cc:224] Handling call
0709 12:27:48.859577 (+  9481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.871919 (+ 12342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.873839 (+  1920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:48.889040 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
W20260709 12:27:48.889878 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78044.1us
I20260709 12:27:48.897892 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:48.898675 26785 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:48.899742  3093 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:48.905457  3093 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:48.871654  3188 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:48.907061 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.864707  3106 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:48.900930  3185 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:48.901688  2936 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:48.912914  3185 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.914333  2936 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.921308  3202 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:48.922135  3202 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.922595 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4212) took 2021 ms. Trace:
I20260709 12:27:48.906603  3198 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:48.925045  3198 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.20, Last appended by leader: 20, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.923214 26054 rpcz_store.cc:276] 0709 12:27:46.901619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.964852 (+1063233us) service_pool.cc:224] Handling call
0709 12:27:47.965254 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.070376 (+105122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.922219 (+851843us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:48.922505 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228987136}
I20260709 12:27:48.922641  3202 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:48.928673 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55550.1us
W20260709 12:27:48.930876 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.909727 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:48.932176 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:48.933063  3191 log.cc:927] Time spent T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:27:48.934484 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1734) took 1387 ms. Trace:
I20260709 12:27:48.935482 26756 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:48.934854 26123 rpcz_store.cc:276] 0709 12:27:47.547028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.215583 (+668555us) service_pool.cc:224] Handling call
0709 12:27:48.231227 (+ 15644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.239935 (+  8708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.239965 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:48.262670 (+ 22705us) log.cc:844] Serialized 26 byte log entry
0709 12:27:48.503968 (+241298us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:48.504024 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.504974 (+   950us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.934224 (+429250us) raft_consensus.cc:1685] finished
0709 12:27:48.934248 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.934382 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3706,"raft.queue_time_us":21692,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":12383,"thread_start_us":165863,"threads_started":1,"wal-append.queue_time_us":223552}
I20260709 12:27:48.936853 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1290) took 1978 ms. Trace:
I20260709 12:27:48.937896  2975 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:48.938519  2975 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:48.938935 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:48.940450  3025 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:48.940886  3025 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:48.941285  3025 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:48.945005 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2945) took 1922 ms. Trace:
I20260709 12:27:48.945181 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:48.937269 26047 rpcz_store.cc:276] 0709 12:27:46.958395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.059291 (+1100896us) service_pool.cc:224] Handling call
0709 12:27:48.072925 (+ 13634us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:48.124768 (+ 51843us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:48.353560 (+228792us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.521863 (+168303us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:48.548327 (+ 26464us) log.cc:844] Serialized 52 byte log entry
0709 12:27:48.548578 (+   251us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:48.562833 (+ 14255us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.569007 (+  6174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.569256 (+   249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.936593 (+367337us) raft_consensus.cc:1685] finished
0709 12:27:48.936614 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.936791 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13547,"prepare.queue_time_us":245123,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":2692,"raft.queue_time_us":20592,"raft.run_cpu_time_us":439,"raft.run_wall_time_us":42092,"spinlock_wait_cycles":1082240,"thread_start_us":310722,"threads_started":2,"wal-append.queue_time_us":292321}
I20260709 12:27:48.946806 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:48.947007  3195 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:48.947575  3195 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:48.945410 26198 rpcz_store.cc:276] 0709 12:27:47.022871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.721947 (+699076us) service_pool.cc:224] Handling call
0709 12:27:47.722388 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.778150 (+ 55762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.944787 (+1166637us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b5400301954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:48.944951 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126794112}
I20260709 12:27:48.948666  3195 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.950184  3195 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:48.939484  2975 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:48.956719  3129 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:48.957289  3129 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:48.958724 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.959311 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:48.960908 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1289) took 2002 ms. Trace:
I20260709 12:27:48.961290 26053 rpcz_store.cc:276] 0709 12:27:46.958059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.967806 (+1009747us) service_pool.cc:224] Handling call
0709 12:27:48.124364 (+156558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.125071 (+   707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.960721 (+835650us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400229a50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:48.960850 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358948224}
I20260709 12:27:48.962477 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:48.963359 26053 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:48.964109 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:48.965595 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:48.966843 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:48.945799 26050 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:48.970402 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:48.971752 26053 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.969031 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:48.974846 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 79369.1us
I20260709 12:27:48.976938 26054 raft_consensus.cc:2368] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30.
W20260709 12:27:48.968873  3196 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.392s	user 0.008s	sys 0.001s
W20260709 12:27:48.985133 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49751.3us
I20260709 12:27:48.985426 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:48.986476 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:48.969100  2811 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.978080  3039 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:48.992532 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:48.980077  3122 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:48.994248 26047 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:27:48.973882  3133 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:48.993799 26049 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:48.997108 26054 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:48.998653 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4219) took 1190 ms. Trace:
I20260709 12:27:48.999053 26054 rpcz_store.cc:276] 0709 12:27:47.808636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.989417 (+1180781us) service_pool.cc:224] Handling call
0709 12:27:48.996988 (+  7571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.998422 (+  1434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.998601 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:49.000408 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:48.995265  3039 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:48.932014  3199 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.006102  3039 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:48.997992  2811 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.008347  3204 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.008913  3204 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.009479  2811 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.010099 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4222) took 1126 ms. Trace:
I20260709 12:27:49.010519 26049 rpcz_store.cc:276] 0709 12:27:47.878721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.996627 (+1117906us) service_pool.cc:224] Handling call
0709 12:27:49.004095 (+  7468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.004336 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.004363 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:49.004393 (+    30us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:49.004423 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.004460 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.004576 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:49.011153  3204 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:49.011806  3168 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.712s	user 0.003s	sys 0.000s
I20260709 12:27:49.014258 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1725) took 1682 ms. Trace:
I20260709 12:27:49.015018 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4215) took 1292 ms. Trace:
I20260709 12:27:49.016319 26049 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:27:49.015434 26046 rpcz_store.cc:276] 0709 12:27:47.722213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.787488 (+1065275us) service_pool.cc:224] Handling call
0709 12:27:48.787819 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.796399 (+  8580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.796426 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:48.804699 (+  8273us) log.cc:844] Serialized 26 byte log entry
0709 12:27:48.890895 (+ 86196us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:48.898065 (+  7170us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.898526 (+   461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.898607 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:49.014794 (+116187us) raft_consensus.cc:1685] finished
0709 12:27:49.014816 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.014940 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8180,"prepare.queue_time_us":6669,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":17526,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":41949,"thread_start_us":51263,"threads_started":1,"wal-append.queue_time_us":137456}
I20260709 12:27:49.017830 26046 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:48.995113  3122 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.024673  3122 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:49.025794 26053 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:49.027667 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.029109 26054 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:49.028899  3204 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:49.002858  3197 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:49.014689 26125 rpcz_store.cc:276] 0709 12:27:47.331836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.630351 (+298515us) service_pool.cc:224] Handling call
0709 12:27:48.119314 (+488963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.125447 (+  6133us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:48.131411 (+  5964us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.376824 (+245413us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.430107 (+ 53283us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.430487 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.431612 (+  1125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:48.431664 (+    52us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:48.431719 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.431791 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.840496 (+408705us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:27:48.943267 (+102771us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:48.952449 (+  9182us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b1800342b20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:49.014084 (+ 61635us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b5400287650. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:49.014198 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100163,"prepare.queue_time_us":1051957,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":145,"raft.queue_time_us":60256,"raft.run_cpu_time_us":3942,"raft.run_wall_time_us":418391,"spinlock_wait_cycles":1330331648,"thread_start_us":241271,"threads_started":1,"wal-append.queue_time_us":348258}
I20260709 12:27:49.038409  2830 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:49.039208 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.032814  3199 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:49.043249  3181 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.440s	user 0.003s	sys 0.007s
I20260709 12:27:49.044294  3045 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:49.048470  3183 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.523s	user 0.006s	sys 0.003s
I20260709 12:27:49.044178 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.049549  3045 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:49.029678 26047 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: [25.16-27.17]
I20260709 12:27:49.031044  3204 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:49.062336 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.060976 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1291) took 2102 ms. Trace:
I20260709 12:27:49.063429 26045 rpcz_store.cc:276] 0709 12:27:46.958660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.127498 (+1168838us) service_pool.cc:224] Handling call
0709 12:27:48.207561 (+ 80063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.235446 (+ 27885us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:48.516767 (+281321us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.678488 (+161721us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.682548 (+  4060us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.917279 (+234731us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b18003803e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:49.017144 (+ 99865us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b18003803e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:27:49.037414 (+ 20270us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.037989 (+   575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:49.038062 (+    73us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:49.038141 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.038262 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.060906 (+ 22644us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25436,"prepare.queue_time_us":106466,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":189,"raft.queue_time_us":63404,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":5895,"spinlock_wait_cycles":42192640,"thread_start_us":331579,"threads_started":3,"wal-append.queue_time_us":217945}
I20260709 12:27:49.032792 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.075870 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.068833  3045 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.058154  3116 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.077512  3045 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:49.078860 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44691.5us
I20260709 12:27:49.080520 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.080844 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.081840 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.082026 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.082279  3174 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:27:49.082800 26787 raft_consensus.cc:2368] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:49.083141 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.083739 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.082487 26049 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 17. (term mismatch)
I20260709 12:27:49.087684 26784 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:27:49.088071 26784 pending_rounds.cc:85] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 15
I20260709 12:27:49.088352 26784 pending_rounds.cc:107] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:27:49.088586 26784 raft_consensus.cc:2889] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.085112  3178 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.003973  3133 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.085656 26049 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 16
I20260709 12:27:49.101835 26049 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 29.17
W20260709 12:27:49.101859 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.102216 26049 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.094188 26053 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:49.097036 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.092746  3178 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.080094  3045 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:49.097023  3208 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:49.097360 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.097613 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: true dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.077548  3204 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.100121 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.113405 26046 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:49.100185 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:49.113993 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108544us
I20260709 12:27:49.104009 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.058095  3202 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.029649  3106 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.004033  3198 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.085s
I20260709 12:27:49.093422  3083 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:49.121026 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.122434  3178 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.123456  3133 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:49.125154  3205 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:49.119343  3106 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.127010 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4213) took 1405 ms. Trace:
I20260709 12:27:49.127380 26052 rpcz_store.cc:276] 0709 12:27:47.721528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.781244 (+1059716us) service_pool.cc:224] Handling call
0709 12:27:48.781660 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:48.795842 (+ 14182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.795890 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:48.813411 (+ 17521us) log.cc:844] Serialized 52 byte log entry
0709 12:27:48.930696 (+117285us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:48.966697 (+ 36001us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.966930 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.967010 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:49.126757 (+159747us) raft_consensus.cc:1685] finished
0709 12:27:49.126780 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.126931 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10949,"prepare.queue_time_us":9830,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":16404,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":13976,"thread_start_us":116757,"threads_started":1,"wal-append.queue_time_us":181070}
I20260709 12:27:49.069285  3025 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.133410 26019 leader_election.cc:400] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29.
I20260709 12:27:49.140625 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.110854  3149 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:49.133335  3045 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:49.134451  2499 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:27:49.155408 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55304.6us
I20260709 12:27:49.155462 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.106259  3119 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:27:49.157404 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.158385 26784 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:49.162684 26790 raft_consensus.cc:2393] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:49.134603  3025 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.164551 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.165055  2499 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:49.165331  3202 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.165716 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.166097  2499 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.167780  3039 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.168570  3156 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:49.168687 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4223) took 1222 ms. Trace:
I20260709 12:27:49.169142  3156 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.169386  3171 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:49.169592  3156 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.169850  3171 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.164413  3125 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:49.153519  3149 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.170542 26053 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:49.170950 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57495.7us
I20260709 12:27:49.169158 26047 rpcz_store.cc:276] 0709 12:27:47.946516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.997960 (+1051444us) service_pool.cc:224] Handling call
0709 12:27:49.027662 (+ 29702us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:49.074076 (+ 46414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.168468 (+ 94392us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b54003593d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:49.168626 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67360128}
I20260709 12:27:49.158301  2811 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:49.180896  3129 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.192065  3048 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:49.193140  3048 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:49.194079  3129 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.182175 26020 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.194614  3048 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:49.194756  3202 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:49.195223 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.195878 26788 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.163847  3133 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:27:49.185204  3149 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.197044  3133 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:49.197459  3133 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:49.185128  2965 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:49.198451  3133 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.177409  3171 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.186769 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:49.172761  3184 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.687s	user 0.004s	sys 0.000s
I20260709 12:27:49.186328 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.188863 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.189453  3119 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.174113  3045 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:49.200117  3133 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:49.203706 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.219842 26124 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:27:49.160104  2769 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:27:49.203043 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.202384 26123 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: [29.19-31.20]
I20260709 12:27:49.221159  3188 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:49.221529 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.238027 26788 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:49.205020 26127 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:27:49.224532 26784 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:49.224649 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.225162 26787 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:49.201668 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:49.226723 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44653.1us
I20260709 12:27:49.227274 26790 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:49.208657 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:49.230252 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.205271 26045 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:49.198493  2965 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.224086 26126 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:27:49.242084 26791 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:49.239784 26122 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
W20260709 12:27:49.235419 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:49.243631 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.243940 26129 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:49.202419 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.240931 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.247701  3093 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.248327  3093 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.249001  2965 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:49.249691  3076 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:49.250227  3076 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:49.250464 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.250670  3076 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:49.251775 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.252158 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.252334 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.252600 26195 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:49.252346 26121 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:49.250240 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.243777 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.248037 26196 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:27:49.170168  3039 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.255896 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.256404 26124 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:49.256840 26202 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:49.255812 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.257975 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.258301 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.259137 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.259871 26200 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:49.255642 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.170113  3092 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:27:49.264907 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.265609 26128 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:49.266914 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.267405 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.253551  3212 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:27:49.268805 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.237089  3188 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:49.238523  3048 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:49.270720 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:49.276604 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.278290 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.282997 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:49.279217 26125 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:49.288663 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.245333  3129 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:49.289639 26126 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:49.296895 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.299882 26203 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.261154  3093 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.226392  3119 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.305076  3129 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:49.296274  2936 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.012s
I20260709 12:27:49.270124  3188 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.325228 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.284440  3036 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:27:49.308535  3045 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.351650  3195 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.305866  3025 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:49.361995  3025 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:49.362640  3025 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.284637  2995 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:49.364459  2995 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.364943  2995 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.274025  3048 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:49.288069  2910 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.368811  3129 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.369066  3048 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.370625  3048 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.287827  3044 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:49.323652 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 66741.3us
I20260709 12:27:49.326704 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:49.376708 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69212.6us
I20260709 12:27:49.377461 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:49.378371 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.379887 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.358174  3119 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.380472  3116 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:49.379930 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.381227  3116 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:49.381922 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.382159 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.382542 26196 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:49.383221 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.338155  3188 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.380908 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.380941  3119 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.005s
I20260709 12:27:49.379827  3198 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:49.385586 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.387559 26789 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:49.309522  3122 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.389403 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.390040 26124 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:49.390372  3122 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.308305  3106 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:49.391424  3106 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:49.391835  3106 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:49.386628  3119 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:49.392655  3119 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:49.392800  3106 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.393321 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.394145  3106 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.338063 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.395169 26050 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:49.338429  2936 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:27:49.377511 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.395939  3174 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:49.396196  2936 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:49.396934  3174 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.398281  3036 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:49.397835  2936 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.352382  3195 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.393677  3119 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.412344  3119 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:49.413167  3216 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.072s	user 0.004s	sys 0.000s
I20260709 12:27:49.413336  3195 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:49.397655 26093 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.414572 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:49.415499 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.416250 26786 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:49.416504  3044 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:49.398722  3036 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.417066  3044 raft_consensus.cc:697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:49.417361  3036 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.418062  3044 consensus_queue.cc:237] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.420217  3202 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:49.395219 26198 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 17. (term mismatch)
W20260709 12:27:49.355612 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 72359.4us
I20260709 12:27:49.352648  3197 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.420717  3202 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.416635 26786 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:49.421226  3202 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:49.421808 26786 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.423013 26786 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.424548  3197 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.380266 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:49.402077 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.427275 26200 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:49.428042 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.404914  2936 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:49.409710  2910 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:49.410559 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.399554  3174 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.426371 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.430953 26046 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:49.436499 26047 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
W20260709 12:27:49.326596 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44485.7us
I20260709 12:27:49.393217 26120 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:49.334822 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.420771 26198 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 16
I20260709 12:27:49.438004 26196 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 31 because replica has last-logged OpId of term: 30 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:27:49.438395 26198 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 28.17
W20260709 12:27:49.434919 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54755.4us
I20260709 12:27:49.438752 26198 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.436497 26200 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:27:49.440580 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:49.441272 26045 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.441045  3156 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.443244  3156 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.440426  3185 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.444128  3122 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:49.445168 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 121010us
I20260709 12:27:49.445415 26786 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:49.445380  3076 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.447734  3076 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.448431  3195 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:49.449209 26203 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:49.449322 26199 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:49.449759  3093 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:49.468598 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.462594 26045 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:49.473289 26120 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:27:49.473492 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45528.2us
I20260709 12:27:49.464944 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:49.474462 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:49.464073 26194 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:49.463948  3171 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.478402 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:49.479172  3178 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:27:49.480437 26195 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:49.480367 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:49.483006 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.479322 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.483139 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.483728 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.484426 26125 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.479427  3171 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:49.438306  3198 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:49.488189 26128 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:49.489960 26127 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:27:49.490355 26054 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: [30.16-30.16]
I20260709 12:27:49.462033  2811 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:49.499050 26124 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:49.491106  2910 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:49.504585 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.508623 26120 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:49.510846 26169 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:49.514912 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.520886 26046 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:27:49.481333  3133 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
W20260709 12:27:49.527177 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.531240 26127 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:49.508013  3197 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:49.481127  3174 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:27:49.543586 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.500770  2811 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.505692  3171 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:27:49.544634 26123 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:27:49.550220 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69958.4us
I20260709 12:27:49.551505 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.552554 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.550547 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.553211 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.553336 26203 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.531706  3156 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:49.509083  3025 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.524974  3092 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:49.531946  2965 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.553444 26052 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:49.504670  3198 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.519233  3176 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.552209 26127 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
W20260709 12:27:49.557667 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.525449  3220 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:49.520668  3203 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:49.579210 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.588743 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.593930 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4221) took 1739 ms. Trace:
I20260709 12:27:49.596103 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.596441 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:49.594403 26050 rpcz_store.cc:276] 0709 12:27:47.854627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.992019 (+1137392us) service_pool.cc:224] Handling call
0709 12:27:48.998686 (+  6667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.395618 (+396932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.593700 (+198082us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54001b44d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:49.593871 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911371136}
I20260709 12:27:49.573344  3220 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:49.604374  2603 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:49.604571  3220 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:49.604979  2603 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:49.606149  2603 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.607640  2603 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.609557  2811 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.609789  3202 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.612172  3202 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.613127 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1294) took 2653 ms. Trace:
I20260709 12:27:49.613539 26048 rpcz_store.cc:276] 0709 12:27:46.959633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.328984 (+1369351us) service_pool.cc:224] Handling call
0709 12:27:48.374536 (+ 45552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.382312 (+  7776us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:48.444074 (+ 61762us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.554222 (+1110148us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b18003e40a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:27:49.590427 (+ 36205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:49.591109 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 12:27:49.591278 (+   169us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:49.591326 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.591419 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:49.612782 (+ 21363us) raft_consensus.cc:1685] finished
0709 12:27:49.612901 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.613062 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38496,"prepare.queue_time_us":60702,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":46,"raft.queue_time_us":412,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":170,"spinlock_wait_cycles":4196736,"thread_start_us":35705,"threads_started":1,"wal-append.queue_time_us":1122852}
I20260709 12:27:49.617326 26201 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:49.645136 26129 raft_consensus.cc:2393] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 31: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:49.615391  3149 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.655836 26203 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:27:49.664495 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68114.2us
I20260709 12:27:49.665512 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:49.666738 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:49.667338 26790 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:49.670851 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 72128.4us
I20260709 12:27:49.597048  3093 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:49.573956  3092 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.573853  2965 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.663515  3076 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:49.666304  3203 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:49.695219 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.597812  2995 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.708148 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:49.717419 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51997.1us
I20260709 12:27:49.718290 26094 leader_election.cc:400] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30.
I20260709 12:27:49.718564 26094 leader_election.cc:403] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:49.713250 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.720465 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.676258  3036 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:49.715293 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.723111 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.724005 26195 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:49.727033 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:49.728053 26123 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:49.730649 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.727023 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:49.730024 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.734259 26785 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:49.686467  3174 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:49.697615  2499 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.715593  2811 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:49.698946  3203 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:49.699477  3188 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.733551 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.751928 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:49.758865 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:49.741221 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.722980  3093 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.736685  2830 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:49.773345 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.748459  3202 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:49.779155 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.756561  3203 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.783080  3202 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.791025  2811 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.756479  3174 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.791671  2811 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.765071  3218 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:49.787941  2910 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:49.798909  2910 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:49.800052  2811 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.806244 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:49.810170 26120 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:49.817204 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.799863  2910 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:49.792650  3119 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.791431  3202 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.758265  2499 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:49.837675 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.768244  3219 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:27:49.765166  3188 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.865882 26125 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:49.851629 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:49.854220 26051 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:27:49.772387  3093 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.773331  3129 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:27:49.859025 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.811375  3218 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:49.865439 26194 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: []
W20260709 12:27:49.869480 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.832806  2830 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:49.879189 26195 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:49.860013 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.592s (SENT) [suppressed 5 similar messages]
I20260709 12:27:49.881094  3226 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30.
W20260709 12:27:49.882818  3233 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:27:49.865139  3239 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:49.884248  3239 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.884646  3239 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.884869 26198 raft_consensus.cc:1240] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:49.886895 26197 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 25 index: 21. (index mismatch)
I20260709 12:27:49.888919  3218 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.894026  3239 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:49.896603  3222 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:27:49.888007  3129 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:49.904606  3129 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:49.905576  3129 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.906860  3129 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.896582  2830 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:49.911739  2830 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.898393 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.912732 26194 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:49.842605  3225 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:49.914430  3225 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:49.917222  3229 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:27:49.919693  3188 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.920563 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:49.922219  3228 log.cc:927] Time spent T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:27:49.931838 26202 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:49.856787  3119 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:49.933930  3230 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 12:27:49.866101  3232 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:27:49.954231  3239 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:49.959146 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:49.959281 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 145226us
I20260709 12:27:49.964388  3239 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:49.866430  3198 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:49.976939 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.970604 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.978808 26048 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:49.979647 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:49.989321 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.990447 26195 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:49.991420 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:49.993651 26047 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:49.995491 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:50.003175 26051 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:27:49.949672  3149 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:49.949064  3025 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:49.997895 26199 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:50.020555  3236 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
W20260709 12:27:50.022276  3235 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:27:50.014115  2769 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:27:50.029757 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:49.994990  3092 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.042181 26047 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
W20260709 12:27:50.039666 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 233531us
I20260709 12:27:50.043219 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:50.044170 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 184189us
I20260709 12:27:50.011372  3119 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.047912 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.048786 26048 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:50.049755 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:50.051296 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.052291 26049 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:50.054747 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.055473 26048 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 23.
I20260709 12:27:50.035145  3225 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.035946  2965 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:50.061951 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:50.063519 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.065467 26196 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:27:50.066025 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.024173  3188 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:50.070490 26051 raft_consensus.cc:2410] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:27:50.056854  3236 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:50.052892  2769 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:50.120972 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:50.121789 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.059437  3174 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.137809  3093 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.138020 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.138340 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.139120 26051 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:50.139459 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.139981  2830 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:50.140144 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:50.140460 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.140936  3217 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:50.141846 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.142508 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:50.144505 26203 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: [26.17-28.18]
I20260709 12:27:50.146356  2995 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.152382  3039 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:50.152928  3039 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:50.153877  3039 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.155019  3039 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:50.156605  3027 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:50.157091  3027 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.157480  3027 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:50.161515  3076 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:50.162267  3076 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:50.162611 26784 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:50.162529 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:50.163188 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:50.163383 26786 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:50.164757 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:50.165671 26789 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:50.167011 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:50.172353 26786 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:50.174095  3036 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.178807  2830 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:27:50.187812 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.183775 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:50.193044 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.210239 26790 consensus_meta.cc:287] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.005s
I20260709 12:27:50.214954 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.123703  2769 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:50.216601 26120 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:50.216764 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.220000 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:50.219246 26790 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:50.220924 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.224202 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:50.226341 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:50.227473 26125 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:50.225296 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.233229 26121 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:50.231866 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.133584  2499 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:50.218518  3083 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:50.214506  3242 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.272661 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:50.280658 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:50.272862  3225 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:50.231343  3217 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:50.236290  2830 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:50.324608 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.325640 26125 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:50.219173  3092 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:50.331192 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:50.331795 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.345718  3149 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:50.346299  3149 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:50.299592  3225 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.348645  3225 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:50.321115  3217 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.349956 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:50.350255  3217 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:50.351686  3176 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:50.353196  3219 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.330973  3092 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:50.353858  3219 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.355000  3092 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:50.378963 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 158115us
W20260709 12:27:50.379333 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.231287  3036 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.099s
W20260709 12:27:50.379292 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 47572.5us
I20260709 12:27:50.379976 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.218202  3093 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:50.382665 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:50.390278 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.393878 26202 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:50.394040 26093 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.347406  3149 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.356626  2830 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:50.387043  2995 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:50.402716 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.404201  2995 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.408869  2995 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:50.409277  3174 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:50.409060  3198 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:50.409711  3174 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.410115  3174 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:50.411161  3244 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 12:27:50.386847  3119 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:50.398933  3025 raft_consensus.cc:2804] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:50.404320  3218 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.413930  3025 raft_consensus.cc:697] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:50.414268  3218 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.414927  3025 consensus_queue.cc:237] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.417688  3218 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.418318  3218 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:50.419121 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:50.419816 26202 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:50.421303 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:50.321494  3242 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:50.428614 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181145us
W20260709 12:27:50.429441 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.433878 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:50.434803 26202 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:50.440120 26120 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:27:50.431875  3036 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.404456  3176 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:50.448673 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.450410 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:50.451479  3036 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:50.455955  3203 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.009s
I20260709 12:27:50.457512 26202 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:50.424441  3149 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:50.398479  3093 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.443104  3218 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:50.490988 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.492344 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:50.479607  3129 consensus_meta.cc:287] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.569s	user 0.016s	sys 0.011s
I20260709 12:27:50.456879  3176 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.467077  2910 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:50.424598  2769 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.502934 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:50.497529 26167 connection.cc:256] RPC call timeout handler was delayed by -1.3689525119998507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 12:27:50.509608 26167 connection.cc:256] RPC call timeout handler was delayed by -1.312252511999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 12:27:50.510028 26167 connection.cc:256] RPC call timeout handler was delayed by -1.297592719000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:27:50.490044  3218 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.425002  2830 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:50.510390 26167 connection.cc:256] RPC call timeout handler was delayed by -1.2564525119996688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 12:27:50.517051 26167 connection.cc:256] RPC call timeout handler was delayed by -1.1726888289999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260709 12:27:50.466879  3203 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.479460  3226 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 12:27:50.519742 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.494581 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 316342us
I20260709 12:27:50.522536 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.517802 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:50.525074 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:50.501040  3176 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:50.498869  2499 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:50.531071 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:50.528376 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 148482us
I20260709 12:27:50.539280 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:50.518579  3218 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:50.512001  3156 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:50.541422 26124 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:50.532064  3220 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.927s	user 0.015s	sys 0.007s
I20260709 12:27:50.530411  3188 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:50.529690  3129 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:50.546881 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.547019 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.568583 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:50.563385  3125 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:27:50.499426  3044 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:27:50.563565 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:50.564255 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:50.578045 26052 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:50.554883  3202 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.674s	user 0.014s	sys 0.005s
I20260709 12:27:50.563387  3220 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:50.547555  3106 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.153s	user 0.016s	sys 0.006s
I20260709 12:27:50.521495  2910 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:50.551412  2936 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.112s	user 0.009s	sys 0.012s
I20260709 12:27:50.571616 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.589656 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:27:50.548159  3048 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.177s	user 0.013s	sys 0.013s
I20260709 12:27:50.531888  3219 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.590412 26048 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:50.554409  3156 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.566792 26053 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:50.546056 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:50.595937 26120 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:50.597123  3260 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:27:50.598557  3260 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.599086  3260 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.591053  3239 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:50.607573 26051 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:50.608774 26050 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:27:50.609226 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.611441 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:50.612300 26200 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:50.612571 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:50.613317 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 88343.3us
I20260709 12:27:50.614042 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:50.562904  3119 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:50.586201 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 75308.3us
I20260709 12:27:50.613126 26203 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:50.620083 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:50.620707 26046 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:50.620903 26201 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:27:50.542030  2499 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.565367  3188 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:50.621731 26122 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:50.623659 26167 leader_election.cc:400] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:50.624059 26167 leader_election.cc:403] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:27:50.624279 26202 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:50.625818 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:50.588955 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 186323us
I20260709 12:27:50.631639 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.627767  3188 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:50.623910  3249 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:50.635411  2603 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.027s	user 0.023s	sys 0.004s
I20260709 12:27:50.635978 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:50.636667  3251 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:27:50.608059  3156 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:50.640750  3133 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:50.619972  3106 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:50.622828  3240 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:27:50.621205  3248 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:50.620463  3239 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.700696  3239 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.594064  3202 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.701238  2995 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.702643 26129 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:50.703523  3202 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.704632  3203 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.705261  3203 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:50.706015  2499 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.656638 26051 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:50.707350 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.716799  3217 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.718078 26045 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:27:50.652698 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.720939 26050 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:27:50.722489 26053 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:27:50.659823 26127 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:27:50.725898 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2960) took 1009 ms. Trace:
I20260709 12:27:50.726514 26200 rpcz_store.cc:276] 0709 12:27:49.716645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.730399 (+ 13754us) service_pool.cc:224] Handling call
0709 12:27:50.725607 (+995208us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54002b5054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:50.725809 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025478528}
I20260709 12:27:50.727543  3176 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.662983  3242 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.655880  3048 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.668603  3093 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:50.670003  3076 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.507s	user 0.018s	sys 0.000s
W20260709 12:27:50.671733  3039 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.019s
I20260709 12:27:50.655969  3249 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:50.660910  2936 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:50.631158  3198 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 12:27:50.676514 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.680552 26121 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:27:50.681149  2830 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:50.688421 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.697104  3119 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.701356  3220 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:50.705760  3203 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.731416 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4324) took 1478 ms. Trace:
I20260709 12:27:50.779388 26122 rpcz_store.cc:276] 0709 12:27:49.253359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.271256 (+ 17897us) service_pool.cc:224] Handling call
0709 12:27:49.293394 (+ 22138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.623333 (+1329939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.731030 (+107697us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:50.731328 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3054097664}
W20260709 12:27:50.736488 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.782542 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:50.783221 26785 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:50.784226 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:50.770375  3076 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.784947 26790 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:50.748819  3149 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.650828  3263 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:50.740574  2995 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:50.645533 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:50.741225  3027 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.014s
I20260709 12:27:50.791379 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.631292 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.716063  2769 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:50.723897 26050 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:50.744341  3260 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.745935 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3300) took 1035 ms. Trace:
I20260709 12:27:50.747401  3174 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:50.806131 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 160653us
I20260709 12:27:50.806854  3174 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:50.748308  3083 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:50.807559 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:50.807916  3240 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:50.651460 26120 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:50.657513 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:50.809315 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 151868us
I20260709 12:27:50.802920 26201 rpcz_store.cc:276] 0709 12:27:49.711035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.722378 (+ 11343us) service_pool.cc:224] Handling call
0709 12:27:49.722668 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.627523 (+904855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.745655 (+118132us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:50.745868 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065312128}
I20260709 12:27:50.754387 26048 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:50.758482 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:50.760641 26052 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:27:50.761695 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 269433us
I20260709 12:27:50.763891 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.765053 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.766132 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.767557 26125 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:50.720017  3217 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:50.755165  3254 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.190s	user 0.002s	sys 0.004s
I20260709 12:27:50.777043 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:50.739392 26046 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:27:50.812716 26126 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:50.770151  3048 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.795300  3259 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.817092  3259 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.818414  3048 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.818123  3156 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.819705  3259 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.789705  3239 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:50.790258  3236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.018s	sys 0.002s
I20260709 12:27:50.741573 26054 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:50.793174 26051 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:50.771495 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.771775  3249 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:50.715626  3225 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.797237 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:50.803212 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 173788us
I20260709 12:27:50.801959  2499 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:27:50.772300  3045 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.820669 26047 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:50.822234 26123 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:50.826692 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:50.842075 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70662.5us
I20260709 12:27:50.809208  3263 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:50.865060 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:50.811558  3198 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:50.840806  3236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.020s	sys 0.007s
I20260709 12:27:50.878877 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:50.880401 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:50.880554 26124 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:50.821820  3076 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:50.795037  3149 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.826489  2603 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:27:50.888089 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 90976.1us
I20260709 12:27:50.889750 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:50.835042  3241 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:50.864648  3217 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:50.802981  3027 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.822024  3133 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:27:50.877604  3260 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.880329  3198 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.882432  3236 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:50.912864 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:50.887880  3240 raft_consensus.cc:2764] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election decision vote started in defunct term 27: won
I20260709 12:27:50.833650  3174 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.060s
I20260709 12:27:50.920670 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:50.918773  3268 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:50.898187  3242 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:50.894380  3093 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:50.873196  3218 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.864557  3048 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:27:50.894584  3265 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:50.880767  3188 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.944489 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:50.887030  3039 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
W20260709 12:27:50.930423 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 106399us
I20260709 12:27:50.881574  3083 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:27:50.952170 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:50.909204  3270 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:50.916998  3198 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:50.922657  3092 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:50.941015  3242 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.941804  3093 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.007151  2603 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:50.922322  3202 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:50.892570  3045 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:50.991933  3156 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:50.916504  3264 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 12:27:50.948840  3025 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:50.992381  3265 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:51.005788  3268 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 12:27:51.014413 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 148266us
I20260709 12:27:50.997867  3044 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:27:51.009155  3267 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:50.920691  3260 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:50.937506  3259 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.093s
I20260709 12:27:51.004662  3176 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.027032  3242 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:51.031805  3093 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:51.046284 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47907.4us
I20260709 12:27:51.046283  3202 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:51.060276 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 184989us
W20260709 12:27:51.060926 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 108863us
I20260709 12:27:51.076725 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.078228 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.078863 26200 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:51.028048  3129 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:51.081624 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:51.085592 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.085968 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.088457 26200 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:27:51.088876 26200 pending_rounds.cc:85] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 15
I20260709 12:27:51.089092 26200 pending_rounds.cc:107] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:27:51.089324 26200 raft_consensus.cc:2889] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.089612 26200 pending_rounds.cc:107] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:27:51.089847 26200 raft_consensus.cc:2889] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.052211  3265 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:51.058346  3044 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:51.062315  3025 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.106689  3271 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:51.106961  3025 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.107448  3271 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.109509  3260 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.109596  3025 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.110653 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.110868  3259 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.111351 26045 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:51.111824  3044 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:51.060209  3203 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.092607 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:51.097772 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.091374 26199 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:51.101255 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:51.030330  3270 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.105588 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.105763  3149 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:51.106297 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.052420  3263 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.127696 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.096200 26201 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:51.113332 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.113912 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.114475 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.133126 26050 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:51.115056 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.133165 26049 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:51.134102 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:51.116117 26125 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:51.119400 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.122126 26200 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:51.128400 26199 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:51.122601 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:51.138052 26785 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:51.119793 26126 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:51.124073 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.074188  3202 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.141490 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.093043  3083 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.146009 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.146881 26048 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 23: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:51.146571 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:51.095785  3225 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.110692  3188 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:51.163093 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.164324 26054 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 22: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:51.153863 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.165529 26046 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:51.151810 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.159833 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.166791 26126 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.145396  3076 consensus_queue.cc:1059] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 31.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:27:51.102413  3217 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:27:51.179606 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.193804 26202 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:27:51.194960 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:51.195479 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.159698  3248 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.153303  3156 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:51.192361 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45889.1us
I20260709 12:27:51.180384 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.198753 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:51.112385  2830 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:27:51.204846 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.205390 26125 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:51.142171  2965 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:51.111373  3259 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.117158  3129 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:51.199563 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.206190 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:51.208349 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:51.237654 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:51.150561  3202 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:51.141752  2603 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.209749 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.210009  3149 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:27:51.214110 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:51.194123  3261 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:51.068485  3236 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.224469 26054 raft_consensus.cc:1275] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:51.161782  3083 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:51.242293  3133 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:51.237780  3274 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.203539  3262 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:51.245358 26202 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:51.249063 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:51.276322 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 146867us
I20260709 12:27:51.256426 26197 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 12:27:51.243592  3276 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.243711 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:51.223781  3176 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.236032  3174 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.234073  3025 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:51.198369  3278 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.275118 26196 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:27:51.212541  3036 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.257813  3156 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:51.277289 26202 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:51.282455 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:51.255574  3218 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.237864  3188 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 12:27:51.211313  3248 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.306559 26125 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:51.234336 26046 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:51.257208  3225 consensus_queue.cc:1059] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:27:51.288630  2995 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:51.315397 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.247864  2965 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.288218  3025 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:51.320123 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.296563  3262 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:51.292048  3036 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:27:51.294541  3176 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.280392  3274 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.316740  2499 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:51.381409 26785 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:51.349994  3025 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:51.368533 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:51.295006  3174 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.325350  3282 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:51.371920  3045 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.377739  3176 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:51.392983 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62397.8us
I20260709 12:27:51.290714  3276 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.385874 26197 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:51.369745  3287 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:27:51.306679  3156 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.350549  2995 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:51.398262 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 117428us
W20260709 12:27:51.383229 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63228.7us
I20260709 12:27:51.365990  3270 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.406178 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:51.411559 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:51.303531  3278 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.395495  3282 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.353261  3283 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:27:51.326435  3284 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:27:51.366987  3171 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:27:51.403133  3203 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.402593  2499 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:51.427455  3240 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:51.430063 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.497323 26125 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:51.430835 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.432556 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.433573 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.500870 26054 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:51.434520 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:51.414613 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.441377  3027 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:51.502343 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 172999us
I20260709 12:27:51.464335  3039 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:27:51.475337 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 63892.6us
I20260709 12:27:51.506848 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:51.485761 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53771.8us
I20260709 12:27:51.426590  3285 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:27:51.488308 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56269.3us
W20260709 12:27:51.512878 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 98376.8us
I20260709 12:27:51.406245  3271 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.495407 26050 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:51.422335  3282 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:51.522943 26050 raft_consensus.cc:3055] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:51.523381 26050 raft_consensus.cc:740] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:51.524178 26050 consensus_queue.cc:260] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.525399 26050 raft_consensus.cc:3060] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:51.525908 26049 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:51.443033 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:51.505188 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.508261 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:51.405933  3176 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:51.541787 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.542728 26200 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:51.485283  3171 consensus_queue.cc:1243] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 27.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:51.496274  3240 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:51.551451 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.512089  3285 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:51.496786  3188 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:27:51.537425  3176 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.536742 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.563452  3239 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:51.559847  3129 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.560209  2965 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:51.576777  3249 consensus_meta.cc:287] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.004s
W20260709 12:27:51.570519  3286 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:51.416044  3262 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.599835 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.540956 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:51.563544  3259 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.595304 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.613090 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.596202 26126 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:27:51.596012 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:51.609115 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68251.9us
I20260709 12:27:51.614455 26050 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 31 index: 22. (term mismatch)
I20260709 12:27:51.614889 26050 pending_rounds.cc:85] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 21
I20260709 12:27:51.586205  3176 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:51.615206 26050 pending_rounds.cc:107] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 30.22
I20260709 12:27:51.615496 26050 raft_consensus.cc:2889] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.584553  3271 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:51.616642 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.619550 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.620596 26126 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:51.609753 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.613936 26128 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:51.625350 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:51.626114 26128 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:51.627050 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.588502  2936 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:51.630827 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.633474 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.618314 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:51.638716 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.592676  3239 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:51.661535 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 156417us
I20260709 12:27:51.660672 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:51.660665 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.599063  3129 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.598650  2965 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.667929 26124 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:51.598704  3283 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:51.598930  3270 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:27:51.663430  3265 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.008s
I20260709 12:27:51.643415  3045 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:51.567420  2995 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:51.688865  2830 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:51.641892  2936 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:51.737644  2936 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:51.698417  3297 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:27:51.691346 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.693320  3265 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:51.693805  3296 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.694155  3292 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:51.694628  3281 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.234s	user 0.001s	sys 0.001s
I20260709 12:27:51.694703  3293 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.182s
I20260709 12:27:51.695447  3294 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:51.696120  3257 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:51.697332  3291 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 12:27:51.706197 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 198054us
I20260709 12:27:51.599395  2499 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.714128  3045 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:27:51.681461  3093 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.004s
I20260709 12:27:51.678433  3188 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:51.699219 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:51.738559 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:51.678122  3270 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:51.643126  3027 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:27:51.661870  2811 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Stepping down as leader of term 30
W20260709 12:27:51.699329  3198 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.672s	user 0.016s	sys 0.005s
I20260709 12:27:51.795010  3156 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.610669  3249 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:51.784780 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 71237.5us
I20260709 12:27:51.842340 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:51.679790  3129 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.735435  3174 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.852948 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:51.804901 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:51.808276 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:51.680529  2965 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:51.816087 26129 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
W20260709 12:27:51.814226 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 180836us
W20260709 12:27:51.712594 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.660676  3285 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:51.836681 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 137533us
I20260709 12:27:51.692823  3295 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:51.629377  3240 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:51.841194 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 102701us
I20260709 12:27:51.800571  3274 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:51.868448 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 354475us
I20260709 12:27:51.833086  3188 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.834574  3076 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:51.805593  3276 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:51.837366  2811 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:51.885571 26126 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:51.938473 26203 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:52.034461 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.125031 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:52.040170 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.129375 26197 consensus_meta.cc:287] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.723s	user 0.018s	sys 0.002s
I20260709 12:27:52.130672 26197 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:51.801565  3296 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:51.833325  3093 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.146903 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:52.065689 26200 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
W20260709 12:27:52.143126 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 338278us
I20260709 12:27:51.843477  2910 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:52.151461 26126 pending_rounds.cc:85] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 14
I20260709 12:27:52.151948 26126 pending_rounds.cc:107] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:27:52.153262 26126 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.805944  3257 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:52.116626  3269 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:27:52.122074  3305 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.331s	user 0.000s	sys 0.000s
I20260709 12:27:52.109959  3129 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:51.836395  2830 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:52.134634  3188 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.220135  3129 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:52.220913  3129 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:52.221406  3129 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.223532  3129 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:51.888537  3291 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:27:52.121275  3303 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 12:27:51.971278  3281 log.cc:927] Time spent T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:27:52.227170 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1756) took 1123 ms. Trace:
I20260709 12:27:51.841617  3198 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.067793  3174 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:52.258569  3296 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:52.258956  3092 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.248s	user 0.011s	sys 0.002s
I20260709 12:27:51.833519  3239 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.259130  3318 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:52.259171  3296 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:51.874272  3294 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:52.227669 26123 rpcz_store.cc:276] 0709 12:27:51.103800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.103974 (+   174us) service_pool.cc:224] Handling call
0709 12:27:51.126711 (+ 22737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.143765 (+ 17054us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:51.192095 (+ 48330us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.401453 (+209358us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.408212 (+  6759us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.408377 (+   165us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:51.408414 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.408480 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.226886 (+818406us) raft_consensus.cc:1685] finished
0709 12:27:52.226916 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.227094 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38591,"prepare.queue_time_us":49072,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":20146,"raft.run_cpu_time_us":553,"raft.run_wall_time_us":43454,"thread_start_us":190474,"threads_started":1,"wal-append.queue_time_us":275317}
I20260709 12:27:52.231559 26120 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
W20260709 12:27:52.240450 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.971590  3290 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.220149  2830 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.271791 26201 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
W20260709 12:27:52.272934 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 107029us
W20260709 12:27:52.274459 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 127612us
I20260709 12:27:52.253034  3265 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:52.254148  3314 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:27:52.255117  3252 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.030s	sys 0.037s
I20260709 12:27:52.172782  3240 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:52.257761  3198 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:52.144985  3119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.003s	sys 0.005s
I20260709 12:27:52.170403  3285 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:52.261400  3273 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:52.137063  3270 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:52.290633  3252 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.030s	sys 0.037s
W20260709 12:27:52.298627  3119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.003s	sys 0.005s
W20260709 12:27:52.327395 26128 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
I20260709 12:27:52.328866 26128 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:27:52.276788 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 151826us
I20260709 12:27:52.141569  3048 consensus_queue.cc:1048] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.102s
W20260709 12:27:52.260828  2769 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.447s	user 0.009s	sys 0.005s
I20260709 12:27:52.287498 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.331481 26123 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
W20260709 12:27:52.259960  3242 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.185s	user 0.013s	sys 0.004s
I20260709 12:27:52.260028  3239 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:52.335567 26195 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 31 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 18.
I20260709 12:27:52.336273  3156 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.336980  3093 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:52.337796  3202 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.080s	user 0.011s	sys 0.000s
I20260709 12:27:52.338203  3202 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:52.338949  3044 consensus_meta.cc:287] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 1.196s	user 0.009s	sys 0.000s
I20260709 12:27:52.339243 26127 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:52.339373  3044 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.301867  3273 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:52.340901  2965 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:52.341259  3273 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.342749  3273 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:52.345623  3301 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.533s	user 0.001s	sys 0.001s
I20260709 12:27:52.347658 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1564) took 1097 ms. Trace:
I20260709 12:27:52.348053 26196 rpcz_store.cc:276] 0709 12:27:51.249957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.310187 (+ 60230us) service_pool.cc:224] Handling call
0709 12:27:51.310711 (+   524us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:51.354117 (+ 43406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.354154 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:51.364360 (+ 10206us) log.cc:844] Serialized 78 byte log entry
0709 12:27:51.600851 (+236491us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:51.671150 (+ 70299us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.671401 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.671515 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.347373 (+675858us) raft_consensus.cc:1685] finished
0709 12:27:52.347396 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.347570 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":71565,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":44,"raft.queue_time_us":33096,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":8877,"thread_start_us":182997,"threads_started":1,"wal-append.queue_time_us":299470}
I20260709 12:27:52.349468  3188 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.351630  3188 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:52.353586  3202 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.354373  3270 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.356976  3257 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:52.298383  3305 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:52.282753  3278 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.333575  3324 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:52.360949  3324 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.172948  3045 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.288851  2811 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.342927  2830 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.364331  2811 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:52.364773  3044 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:52.365170 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.334522  3306 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.368458  3306 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:52.335448  3313 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:27:52.335737  3282 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.000s
I20260709 12:27:52.336761  3156 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:52.372152 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.259845  3318 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:52.377130 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.330032 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:52.392272 26128 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:52.381443 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.280220 26018 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.592s (SENT)
I20260709 12:27:52.396499 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.363505  3324 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:52.401156 26197 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: [28.15-29.16]
I20260709 12:27:52.407028 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:52.395795 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:52.390731  3292 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.003s
I20260709 12:27:52.379364  3027 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.384325 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:52.524026 26785 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:27:52.441200 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 164983us
I20260709 12:27:52.527822 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.523327 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.554196 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4265) took 1122 ms. Trace:
W20260709 12:27:52.598052 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.013s	sys 0.008s
W20260709 12:27:52.592980 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 312760us
W20260709 12:27:52.611083 26018 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.627s (SENT)
I20260709 12:27:52.611500 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:52.611503 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.014s	sys 0.009s
I20260709 12:27:52.610558 26046 rpcz_store.cc:276] 0709 12:27:51.431457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.537807 (+106350us) service_pool.cc:224] Handling call
0709 12:27:51.538853 (+  1046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.539350 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.539395 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.568463 (+ 29068us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.820786 (+252323us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:51.820848 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.820959 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.553947 (+732988us) raft_consensus.cc:1685] finished
0709 12:27:52.553970 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.554133 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":374,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":13745,"thread_start_us":204964,"threads_started":1,"wal-append.queue_time_us":470300}
W20260709 12:27:52.613334 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.568037 26202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.026s	sys 0.018s
W20260709 12:27:52.614034 26202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.026s	sys 0.027s
I20260709 12:27:52.610275 26791 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 28 because replica has last-logged OpId of term: 26 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 20.
I20260709 12:27:52.610049 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:52.390237  3025 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.987s	user 0.014s	sys 0.005s
W20260709 12:27:52.450459 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 289350us
I20260709 12:27:52.419263 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.617501 26128 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:52.621960 26194 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:27:52.623067 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.624213 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.624191 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.624519  3333 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:52.625129 26194 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:52.380072  2910 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:27:52.595919  3176 consensus_meta.cc:287] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.928s	user 0.012s	sys 0.000s
I20260709 12:27:52.619118 26196 raft_consensus.cc:2368] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28.
I20260709 12:27:52.398401  3188 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:52.633971 26789 raft_consensus.cc:1275] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:27:52.626271 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:52.464818  3241 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.561s	user 0.014s	sys 0.000s
W20260709 12:27:52.593250  2995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.006s	sys 0.016s
W20260709 12:27:52.640260  2995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.008s	sys 0.017s
I20260709 12:27:52.639328  3188 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.629382 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.638721 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.643728 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.654960 26783 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:52.627792 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:52.519526  2936 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.781s	user 0.005s	sys 0.008s
I20260709 12:27:52.634917 26790 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:52.477324  3044 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:52.498112  2830 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:52.529413  3312 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:52.494854  3295 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:52.638991 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.639297 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:52.638724 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:52.639637 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.734897 26785 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:52.644503 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.624274 26128 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:52.379464  3242 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.653219 26788 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:52.652813 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3320) took 1014 ms. Trace:
I20260709 12:27:52.636626  3252 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.627367 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:52.476768  3294 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:52.657573 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.620870  3292 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:52.652801  3188 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:52.617488  3133 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:52.637777  3338 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:52.626528  3263 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:27:52.669615 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.767146 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:52.669211 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.663578  3314 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.637392  3176 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.772212 26050 raft_consensus.cc:1240] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: [29.16-29.16]
W20260709 12:27:52.672432 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 298) took 48147 ms (client timeout 15000 ms). Trace:
I20260709 12:27:52.666428  3306 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.669293  3241 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:52.672976 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.663398  3248 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.666174  3318 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.666719  3249 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.676219  3203 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:52.626924  3320 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.678256 26194 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:52.679893 26121 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:52.779932  3203 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.681166 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.802778 26124 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:27:52.686369 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:52.804332  3320 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:52.773339 25946 rpcz_store.cc:269] 0709 12:27:04.525181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.764972 (+2239791us) service_pool.cc:224] Handling call
0709 12:27:52.672171 (+45907199us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:38.770572 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:38.951262 (+180690us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:38.957610 (+  6348us) write_op.cc:435] Acquired schema lock
0709 12:27:38.963849 (+  6239us) tablet.cc:662] Decoding operations
0709 12:27:39.021972 (+ 58123us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:39.022088 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:39.022126 (+    38us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:27:39.023418 (+  1292us) tablet.cc:701] Row locks acquired
0709 12:27:39.023473 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:27:39.023670 (+   197us) write_op.cc:270] Start()
0709 12:27:39.023833 (+   163us) write_op.cc:276] Timestamp: P: 1783600059023587 usec, L: 0
0709 12:27:39.023856 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:27:39.119051 (+ 95195us) log.cc:844] Serialized 5193 byte log entry
0709 12:27:39.987674 (+868623us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b1800116bc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:40.543756 (+556082us) op_driver.cc:464] REPLICATION: finished
0709 12:27:42.169934 (+1626178us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b77ddbef0
0709 12:27:42.256090 (+ 86156us) write_op.cc:301] APPLY: starting
0709 12:27:42.256266 (+   176us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:42.460937 (+204671us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:42.473846 (+ 12909us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:43.456418 (+982572us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:43.456790 (+   372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:27:43.456847 (+    57us) write_op.cc:312] APPLY: finished
0709 12:27:43.457358 (+   511us) log.cc:844] Serialized 121 byte log entry
0709 12:27:43.910720 (+453362us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:43.986028 (+ 75308us) write_op.cc:454] Released schema lock
0709 12:27:44.020883 (+ 34855us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:44.074100 (+ 53217us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:44.087443 (+ 13343us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":89,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1694218,"apply.run_cpu_time_us":69815,"apply.run_wall_time_us":1802282,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58545,"cfile_init":1,"lbm_read_time_us":500,"lbm_reads_lt_1ms":4,"num_ops":13,"prepare.queue_time_us":683875,"prepare.run_cpu_time_us":18310,"prepare.run_wall_time_us":561445,"raft.queue_time_us":389257,"raft.run_cpu_time_us":55882,"raft.run_wall_time_us":1821411,"replication_time_us":1444346,"spinlock_wait_cycles":64889216,"thread_start_us":2723338,"threads_started":5,"wal-append.queue_time_us":1118707,"wal-append.run_cpu_time_us":33556,"wal-append.run_wall_time_us":2246631}]]}
I20260709 12:27:52.805503 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:52.806978 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.687795  3076 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:52.689008 26756 connection.cc:256] RPC call timeout handler was delayed by -1.2202431549999346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:27:52.691706  3156 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:52.694324 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3322) took 1054 ms. Trace:
I20260709 12:27:52.696442  3319 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:52.696312  2936 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.697292  3284 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.703339 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:52.708642  3342 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.005s
I20260709 12:27:52.397986  3282 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.735392 26196 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:52.736933 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.708078 26197 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:27:52.823153  3340 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:27:52.733462  3324 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.737061  3045 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:52.760275  3294 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.756875 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.810586  3318 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.809597  3312 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.712348 26757 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:52.763906 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 136604us
I20260709 12:27:52.621220  3027 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.774894  3314 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.737859 26202 rpcz_store.cc:276] 0709 12:27:51.638639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.638857 (+   218us) service_pool.cc:224] Handling call
0709 12:27:51.639166 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.821271 (+182105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.821320 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.839860 (+ 18540us) log.cc:844] Serialized 26 byte log entry
0709 12:27:52.614402 (+774542us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:52.614447 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.614519 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.652571 (+ 38052us) raft_consensus.cc:1685] finished
0709 12:27:52.652592 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.652751 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55284,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":829909,"threads_started":2,"wal-append.queue_time_us":675149}
I20260709 12:27:52.629225  3025 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.765515 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.765872 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:52.767643 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 314327us
I20260709 12:27:52.768483 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:52.709594  3044 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.769290 26053 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:52.769351 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:52.672115 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.767962 26789 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:52.699750 25944 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 20 to 21, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.770651 26047 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:52.756402  3260 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:52.761343  3290 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.763706  3285 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.778430 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.765185  3316 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:52.766834  2830 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:52.798380 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.772385  3263 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.797715 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:52.799643 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:52.766688  3133 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.770205  2910 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.818058  3076 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:52.827284  3335 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.873199  3335 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.875664  3335 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:52.830751 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.876935 26196 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:27:52.876497  3203 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.878019 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.878913 26196 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:52.800670 26052 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:52.802143 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4351) took 1376 ms. Trace:
I20260709 12:27:52.808240 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.832744 26787 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:52.818859  3295 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.836164  3319 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:52.881393 26052 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: [23.12-25.13]
I20260709 12:27:52.819160 26200 rpcz_store.cc:276] 0709 12:27:51.640169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.660463 (+ 20294us) service_pool.cc:224] Handling call
0709 12:27:52.694083 (+1033620us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54002b5054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:52.694238 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915396352}
I20260709 12:27:52.832782 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.883987 26791 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:52.820425 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:52.847913 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 135667us
I20260709 12:27:52.852687 26788 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:52.821266 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:52.820482 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:52.848299 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:52.760474  3252 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.865255 26049 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:52.879833 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.868696 26786 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
W20260709 12:27:52.858534 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 89438.7us
I20260709 12:27:53.473249 26196 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:52.861872 26047 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:53.417985  3039 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.861318  3285 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.022063  3314 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:53.475869  3039 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:52.929576 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.856078 26122 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:52.863065 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:53.356979 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1559) took 2237 ms. Trace:
I20260709 12:27:52.865706 26051 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:27:53.310009 26092 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.042s	sys 0.084s
I20260709 12:27:52.879572 26121 rpcz_store.cc:276] 0709 12:27:51.426200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.679263 (+1253063us) service_pool.cc:224] Handling call
0709 12:27:52.679666 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.801836 (+122170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.802070 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.480984 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4358) took 1743 ms. Trace:
I20260709 12:27:53.482007 26045 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:52.858806 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:52.852224 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:53.485477 26202 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
W20260709 12:27:52.864701 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.894861 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.488509 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.855767 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:52.859115 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:53.491168 26789 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:53.483878 26053 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:53.477202 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:53.498400 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:53.524930 26125 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.858269 26783 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:52.831638 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:52.884119 26785 raft_consensus.cc:2368] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25.
I20260709 12:27:52.810388  3292 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:27:53.481576 26092 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.059s	sys 0.099s
I20260709 12:27:52.857573  3044 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:52.882100  3331 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.583640  3331 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.586511 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3343) took 2510 ms. Trace:
I20260709 12:27:53.586913 26120 rpcz_store.cc:276] 0709 12:27:51.079949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.097126 (+ 17177us) service_pool.cc:224] Handling call
0709 12:27:51.097405 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:52.261198 (+1163793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.586226 (+1325028us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.586424 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607552384}
I20260709 12:27:52.865556  3263 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:53.183271 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:52.835745 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3317) took 1299 ms. Trace:
I20260709 12:27:52.855892  3176 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:53.189440 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 304067us
W20260709 12:27:53.471123  3076 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.001s
I20260709 12:27:53.482627 26126 rpcz_store.cc:276] 0709 12:27:51.737129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.150847 (+413718us) service_pool.cc:224] Handling call
0709 12:27:52.151197 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:52.445132 (+293935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:52.445177 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:52.445900 (+   723us) log.cc:844] Serialized 52 byte log entry
0709 12:27:52.798775 (+352875us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:52.798815 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.798887 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:53.442130 (+643243us) raft_consensus.cc:1685] finished
0709 12:27:53.442165 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.480911 (+ 38746us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133132,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5627,"thread_start_us":553077,"threads_started":2,"wal-append.queue_time_us":870158}
I20260709 12:27:53.617300 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:53.515653 26123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.010s	sys 0.022s
W20260709 12:27:53.618310 26123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.010s	sys 0.023s
I20260709 12:27:53.481010 26790 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.481019 26203 rpcz_store.cc:276] 0709 12:27:51.075346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.096552 (+ 21206us) service_pool.cc:224] Handling call
0709 12:27:51.096835 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.124069 (+1027234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.312215 (+1188146us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400163354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.312480 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1935094528}
I20260709 12:27:53.620628 26203 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 17. (term mismatch)
I20260709 12:27:53.620975 26203 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 16
I20260709 12:27:53.621209 26203 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 30.17
I20260709 12:27:53.621435 26203 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.188627 25944 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 14 to 23. New cstate: current_term: 23 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:52.857892 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:52.851274  3027 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.858982  3156 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:53.539326 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:53.622332 26046 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:53.622926 26784 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:52.853705  3242 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:52.822600  3323 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:53.500854 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 680757us
I20260709 12:27:53.543184 26054 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: [30.16-30.16]
I20260709 12:27:52.836591  3045 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.543329  3292 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.545179 26789 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:53.648495 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:52.861059  3290 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.846473  3318 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.549695  3039 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.550868 26202 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:27:53.551630 26228 heartbeater.cc:646] Failed to heartbeat to 127.24.71.190:42395 (7 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.24.71.190:42395 timed out after 15.000s (SENT)
I20260709 12:27:52.862313  3316 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:52.998291  3284 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.551652  3252 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.557171  3319 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.568948  3285 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.513092  3275 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:52.836542  3324 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.570909  3217 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.573643  3133 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:53.577364  3083 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.581072  3226 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:53.519536  3314 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:53.582641  3218 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.860771  3260 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.589388 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3326) took 1183 ms. Trace:
I20260709 12:27:53.725615 26194 rpcz_store.cc:276] 0709 12:27:52.405770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.625814 (+220044us) service_pool.cc:224] Handling call
0709 12:27:53.589225 (+963411us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400217fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:53.589332 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51037440}
I20260709 12:27:53.588137 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.311602 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1760) took 2101 ms. Trace:
I20260709 12:27:53.728420 26127 rpcz_store.cc:276] 0709 12:27:51.210443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.241541 (+ 31098us) service_pool.cc:224] Handling call
0709 12:27:51.241823 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.339674 (+1097851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.311227 (+971553us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540010f4d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.311518 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523402624}
I20260709 12:27:53.729977 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.730878 26127 raft_consensus.cc:2368] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:53.222931  3174 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:53.217872  3257 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.848512  3198 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:52.854394  3025 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.210587  3338 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.882277  3203 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:53.503973 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:53.745855 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1570) took 1064 ms. Trace:
I20260709 12:27:52.831610  3282 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:52.870018  2936 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.610726 26783 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:27:53.601617 26200 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:53.609481 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:53.748977 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 139620us
I20260709 12:27:53.602787 26199 rpcz_store.cc:276] 0709 12:27:51.536102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.558255 (+ 22153us) service_pool.cc:224] Handling call
0709 12:27:51.558687 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:51.939019 (+380332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.939054 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:52.126270 (+187216us) log.cc:844] Serialized 52 byte log entry
0709 12:27:52.379266 (+252996us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:52.379307 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.379383 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.835488 (+456105us) raft_consensus.cc:1685] finished
0709 12:27:52.835513 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.835680 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31762,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":2815,"thread_start_us":415925,"threads_started":2,"wal-append.queue_time_us":558555}
I20260709 12:27:53.642257 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1924) took 1008 ms. Trace:
I20260709 12:27:53.662132  3323 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:53.600684  3264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.017s	sys 0.043s
I20260709 12:27:53.640113 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:53.749822 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:53.654397  3045 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.657189 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.758083 26049 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:53.758383 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.666028 26196 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:53.666929 26790 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:53.759498 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1925) took 1125 ms. Trace:
I20260709 12:27:53.759997 26790 rpcz_store.cc:276] 0709 12:27:52.634092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.862713 (+228621us) service_pool.cc:224] Handling call
0709 12:27:53.759418 (+896705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.663947 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:52.862442  2830 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.671355 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.672979 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.683084 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1927) took 1048 ms. Trace:
I20260709 12:27:53.764364 26052 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:27:53.669435  3149 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.659327  3265 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:53.685497 26124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.010s	sys 0.036s
I20260709 12:27:53.736987  3226 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.723868 26782 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:53.733141 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:53.667850  3236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.007s	sys 0.008s
I20260709 12:27:53.670552  3176 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.018s
I20260709 12:27:53.727154 26120 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:53.750583 26784 rpcz_store.cc:276] 0709 12:27:52.633483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.507427 (+873944us) service_pool.cc:224] Handling call
0709 12:27:53.642160 (+134733us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.646454 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:53.698274  3217 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.746284 26788 rpcz_store.cc:276] 0709 12:27:52.681552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.180320 (+498768us) service_pool.cc:224] Handling call
0709 12:27:53.745779 (+565459us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.639310  3076 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.778354  3226 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.786124  3076 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.658434 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1922) took 1028 ms. Trace:
I20260709 12:27:53.787082  3226 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:53.788944 26784 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:27:53.789518 26784 pending_rounds.cc:85] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 15
I20260709 12:27:53.789826 26784 pending_rounds.cc:107] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:27:53.790166 26784 raft_consensus.cc:2889] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.791195 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1560) took 1154 ms. Trace:
I20260709 12:27:53.793421 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2910) took 1024 ms. Trace:
I20260709 12:27:53.793889 26054 rpcz_store.cc:276] 0709 12:27:52.772471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.776883 (+  4412us) service_pool.cc:224] Handling call
0709 12:27:52.777204 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:53.643829 (+866625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.793143 (+149314us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b540008e8d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.793354 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760737664}
I20260709 12:27:53.791671 26784 rpcz_store.cc:276] 0709 12:27:52.636330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.779048 (+1142718us) service_pool.cc:224] Handling call
0709 12:27:53.788457 (+  9409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.790813 (+  2356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.791113 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.795543 26054 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
W20260709 12:27:53.796972 26201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2989) took 2695 ms (client timeout 1563 ms). Trace:
W20260709 12:27:53.797423 26201 rpcz_store.cc:269] 0709 12:27:51.103001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.141229 (+ 38228us) service_pool.cc:224] Handling call
0709 12:27:53.796703 (+2655474us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540039a154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:53.796861 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597585152}
I20260709 12:27:53.798113 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1767) took 1535 ms. Trace:
I20260709 12:27:53.762130 26126 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.798681 26123 rpcz_store.cc:276] 0709 12:27:52.262550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.845217 (+582667us) service_pool.cc:224] Handling call
0709 12:27:52.845752 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:52.846222 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:52.846259 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:52.846894 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:27:53.618847 (+771953us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:53.618909 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.619012 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:53.797828 (+178816us) raft_consensus.cc:1685] finished
0709 12:27:53.797861 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.798036 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1905,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":771425,"threads_started":1,"wal-append.queue_time_us":888517}
I20260709 12:27:53.762519  3290 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.764402 26791 rpcz_store.cc:276] 0709 12:27:52.634866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.666658 (+1031792us) service_pool.cc:224] Handling call
0709 12:27:53.682378 (+ 15720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.682608 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.682653 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.682715 (+    62us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:53.682771 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.682848 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.682995 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.737212  3218 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.768103 26790 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:53.769294 26124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.010s	sys 0.036s
I20260709 12:27:53.680114 25944 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:53.771136 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 107252us
I20260709 12:27:53.807319 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1559) took 1171 ms. Trace:
I20260709 12:27:53.807896 26790 rpcz_store.cc:276] 0709 12:27:52.636076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.767584 (+1131508us) service_pool.cc:224] Handling call
0709 12:27:53.767929 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.806968 (+ 39039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.807250 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.767637  3149 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.686672  3319 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
W20260709 12:27:53.757391  3264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.017s	sys 0.043s
I20260709 12:27:53.772065 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.824779 26049 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 28 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 12.
I20260709 12:27:53.737584  3174 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:53.736131  3314 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:53.777238 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.779435 26053 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.755088 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1926) took 1120 ms. Trace:
I20260709 12:27:53.838042 26052 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
W20260709 12:27:53.782225 25939 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.068s	sys 0.146s
W20260709 12:27:53.839421 25939 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.068s	sys 0.147s
I20260709 12:27:53.782167  3242 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:27:53.776811  3265 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.763383 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.801023 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.842501 26127 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:53.838356 26783 rpcz_store.cc:276] 0709 12:27:52.634490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.526149 (+891659us) service_pool.cc:224] Handling call
0709 12:27:53.610327 (+ 84178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:53.754734 (+144407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.755018 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:53.799439 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2262) took 1964 ms (client timeout 1514 ms). Trace:
W20260709 12:27:53.843861 26786 rpcz_store.cc:269] 0709 12:27:51.835349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.835543 (+   194us) service_pool.cc:224] Handling call
0709 12:27:53.799276 (+1963733us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400639f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:53.799380 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333891456}
I20260709 12:27:53.765995 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:53.786866 26789 rpcz_store.cc:276] 0709 12:27:52.630335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.512729 (+882394us) service_pool.cc:224] Handling call
0709 12:27:53.544830 (+ 32101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.648160 (+103330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.658350 (+ 10190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.805228 26788 raft_consensus.cc:1240] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:53.805104 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.845881 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1562) took 1167 ms. Trace:
I20260709 12:27:53.846079 26123 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:53.805428 26791 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:27:53.819680  3294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.976s	user 0.019s	sys 0.070s
I20260709 12:27:53.846303 26788 rpcz_store.cc:276] 0709 12:27:52.678917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.781205 (+1102288us) service_pool.cc:224] Handling call
0709 12:27:53.804993 (+ 23788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.845623 (+ 40630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.845828 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.846899 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1567) took 1166 ms. Trace:
I20260709 12:27:53.847381 26791 rpcz_store.cc:276] 0709 12:27:52.680705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.804121 (+1123416us) service_pool.cc:224] Handling call
0709 12:27:53.805098 (+   977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.846623 (+ 41525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.846823 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4992}
I20260709 12:27:53.802981  3324 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:53.766162  3198 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:53.854754 26127 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:27:53.807300 26784 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:53.718834  3275 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.773797  2936 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:53.814749 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3342) took 2738 ms. Trace:
I20260709 12:27:53.814989 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1568) took 1134 ms. Trace:
I20260709 12:27:53.865046 26790 rpcz_store.cc:276] 0709 12:27:52.680985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.808949 (+1127964us) service_pool.cc:224] Handling call
0709 12:27:53.814460 (+  5511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.814663 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.814690 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.814735 (+    45us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:53.814767 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.814834 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.814936 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.753990  3257 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:53.828753 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 980506us
I20260709 12:27:53.872166  3316 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:53.872702  3265 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.760196  3338 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:53.776115  3236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.008s	sys 0.008s
I20260709 12:27:53.770443  3044 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:53.858457 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1565) took 1178 ms. Trace:
I20260709 12:27:53.923149 26784 rpcz_store.cc:276] 0709 12:27:52.680129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.795298 (+1115169us) service_pool.cc:224] Handling call
0709 12:27:53.795692 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.857991 (+ 62299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.858386 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.781816  3025 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.059s
I20260709 12:27:53.864167  2936 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.836899  3226 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.778272  3203 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.864248 26129 rpcz_store.cc:276] 0709 12:27:51.075922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.096996 (+ 21074us) service_pool.cc:224] Handling call
0709 12:27:51.123507 (+ 26511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.858524 (+735017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.814460 (+1955936us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54001c4b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.814682 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588107008}
I20260709 12:27:53.817826 25944 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 19 to 23, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 23 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:53.843329 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
W20260709 12:27:53.874569 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.968643 26120 raft_consensus.cc:606] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:53.969164 26120 raft_consensus.cc:614] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:53.876837 26126 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:53.877913 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:53.871272 26017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.020s
W20260709 12:27:53.972968 26017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.013s	sys 0.020s
I20260709 12:27:53.850446  3312 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:53.850349  3294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.019s	sys 0.070s
I20260709 12:27:53.886102  3311 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:27:53.880882 26125 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:53.908723 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:53.989454 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80841us
I20260709 12:27:53.992359 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4361) took 1324 ms. Trace:
I20260709 12:27:53.992926 26125 rpcz_store.cc:276] 0709 12:27:52.667424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.886320 (+218896us) service_pool.cc:224] Handling call
0709 12:27:53.992268 (+1105948us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:53.909070 26053 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:53.894430 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:53.893605 26046 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:53.915304  3354 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.917826 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.997457 26123 raft_consensus.cc:2410] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 18.
I20260709 12:27:53.839763  3174 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.929792  3198 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:53.930668 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2976) took 3650 ms (client timeout 2102 ms). Trace:
I20260709 12:27:53.931485 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 2996) took 1530 ms. Trace:
I20260709 12:27:53.937572 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3327) took 1434 ms. Trace:
I20260709 12:27:53.938951  3310 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:54.008988 26198 rpcz_store.cc:276] 0709 12:27:52.402447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.402610 (+   163us) service_pool.cc:224] Handling call
0709 12:27:52.402958 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.654009 (+251051us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:52.661559 (+  7550us) log.cc:844] Serialized 14 byte log entry
0709 12:27:52.770865 (+109306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:52.770902 (+    37us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:52.770939 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.931192 (+1160253us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400336b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.931255 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.931415 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":537,"prepare.queue_time_us":7384,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":316,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4258,"spinlock_wait_cycles":551043712,"thread_start_us":108863,"threads_started":1,"wal-append.queue_time_us":170080,"wal-append.run_cpu_time_us":1571,"wal-append.run_wall_time_us":1005069}
I20260709 12:27:54.011264  3311 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:54.011698  3311 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:53.944007  3331 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:53.946727 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1317) took 1148 ms. Trace:
I20260709 12:27:53.947870 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1919) took 1318 ms. Trace:
I20260709 12:27:53.953373 26047 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:53.995363 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46972 (request call id 1324) took 1194 ms. Trace:
I20260709 12:27:54.016955 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4346) took 2592 ms. Trace:
W20260709 12:27:54.015204 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.017572 26122 rpcz_store.cc:276] 0709 12:27:51.424545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.427922 (+  3377us) service_pool.cc:224] Handling call
0709 12:27:51.428362 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.492803 (+2064441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.016634 (+523831us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540037c154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:54.016875 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282688768}
I20260709 12:27:54.018802 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4365) took 1341 ms. Trace:
I20260709 12:27:54.016130 26050 rpcz_store.cc:276] 0709 12:27:52.798604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.798729 (+   125us) service_pool.cc:224] Handling call
0709 12:27:52.799018 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.775848 (+976830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.775892 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.775942 (+    50us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:53.775983 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.946441 (+170458us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b5400330754. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.946506 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.946659 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2245931392}
I20260709 12:27:54.019441 26121 rpcz_store.cc:276] 0709 12:27:52.677828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.520476 (+842648us) service_pool.cc:224] Handling call
0709 12:27:53.544122 (+ 23646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.883402 (+339280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.883451 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.883513 (+    62us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:53.883560 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.018429 (+134869us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b540013e554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:54.018494 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:54.018730 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779610496}
I20260709 12:27:54.016332 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:53.965075  3324 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.991391 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:53.996559  3076 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.009s
I20260709 12:27:54.006920  3260 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.007644  3284 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.008728 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:54.063508 26129 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:54.008548 26195 rpcz_store.cc:269] 0709 12:27:50.280072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.280314 (+   242us) service_pool.cc:224] Handling call
0709 12:27:53.930450 (+3650136us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:53.930587 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4717575040}
I20260709 12:27:54.003374  3236 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:54.013825  3276 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.005s	sys 0.015s
I20260709 12:27:53.966574  2936 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.009223 26196 rpcz_store.cc:276] 0709 12:27:52.503425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.474596 (+971171us) service_pool.cc:224] Handling call
0709 12:27:53.474979 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.762829 (+287850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.937332 (+174503us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002b4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:53.937501 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423032576}
I20260709 12:27:54.015383 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:54.016908 26046 rpcz_store.cc:276] 0709 12:27:52.800602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.857547 (+ 56945us) service_pool.cc:224] Handling call
0709 12:27:53.995284 (+1137737us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:54.023195 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.026s
W20260709 12:27:54.082301 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.012s	sys 0.026s
I20260709 12:27:54.021552 26123 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:54.016299 26782 rpcz_store.cc:276] 0709 12:27:52.629341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.629626 (+   285us) service_pool.cc:224] Handling call
0709 12:27:53.947640 (+1318014us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b54000e2c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:53.947802 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2193130240}
I20260709 12:27:53.980367  3290 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.965813  3292 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:53.982358  3226 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.995801 26020 leader_election.cc:400] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:54.118855 26020 leader_election.cc:403] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:54.119591 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 123965us
I20260709 12:27:54.054507  2995 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:54.066686 26168 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:54.122700 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.010633  3319 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:54.065552 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:54.123406 26121 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
W20260709 12:27:54.071171  3276 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.005s	sys 0.016s
I20260709 12:27:54.070626 25944 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 24 to 25, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 25 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.021488  3310 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:53.939306  3156 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:54.013010  3242 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.067802 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:54.127946 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1769) took 1449 ms. Trace:
I20260709 12:27:54.054278  3316 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 12:27:54.068744  3236 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.128422 26122 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
W20260709 12:27:54.129524 26785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1564) took 1449 ms (client timeout 1627 ms). Trace:
I20260709 12:27:54.055568  3324 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:53.997136  3203 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:54.127461 26787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 1917) took 1498 ms (client timeout 1943 ms). Trace:
I20260709 12:27:53.997610  3312 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.008919  3354 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.028718  3294 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:27:54.125075  3319 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:54.155795 26755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.073s	sys 0.139s
W20260709 12:27:54.156275 26755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.074s	sys 0.139s
I20260709 12:27:54.125356  3045 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26.
I20260709 12:27:54.125032  2995 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:54.135807 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:54.117998  3320 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:54.128355 26128 rpcz_store.cc:276] 0709 12:27:52.678202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.526964 (+848762us) service_pool.cc:224] Handling call
0709 12:27:53.527337 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.677406 (+150069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.677450 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.677507 (+    57us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:53.677553 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.127748 (+450195us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:54.127787 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:54.127897 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344593280}
I20260709 12:27:54.139322  3323 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.138669 26125 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 30 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
W20260709 12:27:54.129596 26019 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.019s	sys 0.027s
W20260709 12:27:54.190165 26019 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.019s	sys 0.027s
I20260709 12:27:54.121838  3290 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:54.144542 26787 rpcz_store.cc:269] 0709 12:27:52.628681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.628810 (+   129us) service_pool.cc:224] Handling call
0709 12:27:54.127207 (+1498397us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b54005c35d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:54.127393 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223619328}
I20260709 12:27:54.148918 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:54.152456 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 402697us
I20260709 12:27:54.194033 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:54.194645 26784 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.170117 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:54.008375  3314 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.138459  3236 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.174610 26120 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:27:54.174854 25944 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:54.197710 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 416057us
W20260709 12:27:54.134377 26785 rpcz_store.cc:269] 0709 12:27:52.679818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.739209 (+ 59391us) service_pool.cc:224] Handling call
0709 12:27:54.129306 (+1390097us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003101d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:54.129454 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267137280}
I20260709 12:27:54.169497 26127 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:54.193463 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:54.190647 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 703854us
W20260709 12:27:54.199975 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:54.199479 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 63736.1us
I20260709 12:27:54.164712  3319 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:54.205044 26167 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.016s	sys 0.015s
I20260709 12:27:54.176453  3133 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:54.139490  3242 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.172688  2995 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.205762 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:54.216117 26052 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:54.219544 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:54.217679 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 409096us
W20260709 12:27:54.205747 26167 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.016s	sys 0.015s
I20260709 12:27:54.137571  3156 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.197014  3320 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.222908 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 571825us
I20260709 12:27:54.228322  3362 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.230341 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:54.207847 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 335072us
I20260709 12:27:54.133715  3311 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:54.155958  3076 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:27:54.207589 25944 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 10 to 20, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:54.252774 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:54.255734 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:54.205909  3252 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:54.263861 26785 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:54.210461  3133 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.267724 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:54.276782 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:54.287092 26786 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:54.275348 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:54.290967 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:54.299475 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.299901 26790 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:54.262741  3044 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.300836 26129 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:54.301491 26784 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
W20260709 12:27:54.302389 26167 connection.cc:256] RPC call timeout handler was delayed by -1.2309528109994972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
I20260709 12:27:54.270617  3276 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:54.267277  3257 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:54.308048 26167 connection.cc:256] RPC call timeout handler was delayed by -1.217470979999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
I20260709 12:27:54.308555 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:54.269811  3338 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.285681  3316 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.214498  3310 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.343583 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.262439  3320 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.348227 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:54.305138  3133 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:54.297746  3252 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:54.348165 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 281553us
I20260709 12:27:54.302731  3092 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.316367  3276 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:54.338477  3356 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:54.292088  3025 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:27:54.356947 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:54.374949 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.364657  3292 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:54.379782  3292 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:54.380324  3292 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:54.381953  3324 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.349344  3044 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:54.388067 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 188595us
I20260709 12:27:54.346467  3275 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.366336  3252 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:54.373324  3356 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:54.352663  3311 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:54.288126  3362 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.364862  3338 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:54.456084 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99212.2us
I20260709 12:27:54.456990 26786 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:54.462707 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:54.463943 26050 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 28 because replica has last-logged OpId of term: 26 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 20.
I20260709 12:27:54.351066  3357 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:54.469780 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:54.474437 26050 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:54.472499 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:54.338886  3361 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.464790 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:54.388801  3312 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.444573  3044 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.354419  3372 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:54.376950  3226 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.354910  3316 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:54.367894  3359 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.163s
W20260709 12:27:54.491503 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53097.9us
I20260709 12:27:54.531404 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.378247  3092 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:54.521637 26167 connection.cc:256] RPC call timeout handler was delayed by -1.0414565769997353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2565s for a response
I20260709 12:27:54.531893 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.532572 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:54.501643 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:54.493644 26786 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
W20260709 12:27:54.484962 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 209666us
I20260709 12:27:54.546752 25944 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.582345 26791 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
W20260709 12:27:54.485327 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 130158us
I20260709 12:27:54.529172 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.604851 26050 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:54.469017  3356 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:54.636410 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:54.469363  3369 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.650046 26049 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:54.492074  3264 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:54.492031  3376 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:27:54.492661  3364 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.226s	user 0.001s	sys 0.001s
W20260709 12:27:54.691782  3350 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.822s	user 0.002s	sys 0.002s
I20260709 12:27:54.493120  3357 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.651770 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:54.693910  3357 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:54.603873 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:54.500110  3203 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.635838  3312 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.578893  3260 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.685392 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:54.585691 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:54.711464 26050 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
W20260709 12:27:54.711716 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 126142us
W20260709 12:27:54.637436  3365 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 12:27:54.578926  3275 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:54.623134  3276 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.688917  3320 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.635612  3156 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.689460  3376 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.699141 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:54.582291  3044 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:54.706516 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 348344us
I20260709 12:27:54.525882  3354 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.649765  3385 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.708141 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 104348us
I20260709 12:27:54.725694  3203 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:54.571419 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 263816us
I20260709 12:27:54.675105  3198 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.056s
I20260709 12:27:54.518059  3361 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:54.719702  3260 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.712466 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:54.726004  3312 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.695570  3290 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:54.674460  3242 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.623718  3226 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.097s
I20260709 12:27:54.738948 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:54.522392  3378 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.737216  3320 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.722923  3383 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:54.751201 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:54.738412  3374 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:27:54.760941 26019 leader_election.cc:400] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28.
W20260709 12:27:54.761106 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 109432us
I20260709 12:27:54.803787 26019 leader_election.cc:403] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:54.804258 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:54.763159 26093 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:54.742331 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:54.750296  3389 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.702190  3369 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:54.811381 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48287.1us
I20260709 12:27:54.739892  2910 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:54.775525  3361 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:54.777311 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:54.753358  3203 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.805917 26050 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:54.775249 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:54.757745  3156 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.793224 26201 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:54.819621 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:54.752400  3025 consensus_queue.cc:1048] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.054s
I20260709 12:27:54.820974 26049 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:54.780409 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:54.809401 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.810185 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:54.812697 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 64785us
I20260709 12:27:54.773569  3276 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:54.814229 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.776700  3242 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:54.757171  3386 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:54.760632  3359 consensus_queue.cc:1059] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:27:54.824551 26198 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:54.823158 26200 raft_consensus.cc:2410] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 29 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:27:54.738760  3275 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.823974 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:54.800778  3383 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:54.829665 26049 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:54.832571 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.818223  3203 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.776506  3307 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 12:27:54.832474 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.822471  3397 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:54.819103  3156 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:54.818017  3381 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28.
I20260709 12:27:54.800251  3378 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:54.803246  3320 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:54.867633 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.869867 26203 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:54.820611  2910 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.871048 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.872486 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.873704 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:54.874933 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.802501  3382 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.079s
I20260709 12:27:54.830175  3314 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.852130  3368 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 12:27:54.881479 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:54.880832 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76662.3us
I20260709 12:27:54.869326  3383 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.865810 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:54.760886  3385 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.885623  3400 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:27:54.887005  3380 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:27:54.890026  2830 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:54.856828  3392 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:27:54.858139  3362 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.931372  3400 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.869153  3076 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:54.931897  3400 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:54.932050  3076 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.932538  3076 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.934924  3312 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:54.814585  3389 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:54.937944  3389 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:54.922302 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:54.929068  3314 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:54.942479  3394 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:27:54.909368  3092 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:54.882746  3359 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:54.929705 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63994.1us
I20260709 12:27:54.930066 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:54.955600 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:54.956498 26785 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
W20260709 12:27:54.956816 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 205703us
I20260709 12:27:54.949156 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.882112  3354 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:27:54.894801  3397 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:54.876960 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:54.920264  3316 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:54.931298 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.957733 26203 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:54.962165 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:55.015703 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 53621.3us
I20260709 12:27:54.951782  3361 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:55.029673 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.002293  3092 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:54.942956  3381 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:55.034977 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.035791 26203 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:55.035961  3092 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:55.036834 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:55.037453 26203 raft_consensus.cc:2376] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30. Re-sending same reply.
I20260709 12:27:55.037348  3092 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.020632 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:55.021219 26128 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:55.000334  3242 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:54.930742  3220 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:27:54.952634  3025 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:55.010663 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44607.2us
I20260709 12:27:55.022589  3045 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:55.023001  3311 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:27:55.035101  3381 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:55.018074 26167 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.514s (SENT)
I20260709 12:27:55.038595  3092 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:55.251921 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.175105 26126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4366) took 1627 ms (client timeout 1775 ms). Trace:
I20260709 12:27:55.041440 26789 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:55.190871 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4372) took 1064 ms. Trace:
I20260709 12:27:55.253734 26129 raft_consensus.cc:2368] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30.
W20260709 12:27:55.123515 26169 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.130:40611 timed out after 1.775s (SENT) [suppressed 20 similar messages]
I20260709 12:27:55.200583 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.252624 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:55.255021 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.191732 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3359) took 1230 ms. Trace:
I20260709 12:27:55.038676 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:54.966502  2830 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:55.252964 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.325996 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4267) took 1713 ms. Trace:
W20260709 12:27:55.216387  3133 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.004s
W20260709 12:27:55.205574  3292 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.824s	user 0.008s	sys 0.004s
I20260709 12:27:55.327631 26048 rpcz_store.cc:276] 0709 12:27:53.612329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.640660 (+ 28331us) service_pool.cc:224] Handling call
0709 12:27:53.660539 (+ 19879us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:53.660998 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.661039 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:53.661797 (+   758us) log.cc:844] Serialized 52 byte log entry
0709 12:27:54.150015 (+488218us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:54.150090 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.150187 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.258266 (+1108079us) raft_consensus.cc:1685] finished
0709 12:27:55.258300 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.325900 (+ 67600us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1041,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4644,"thread_start_us":474859,"threads_started":1,"wal-append.queue_time_us":708969}
I20260709 12:27:55.363627  3292 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.366250  3292 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:55.358067 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:55.329200 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 311440us
I20260709 12:27:55.370137  3385 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:55.327349 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 203881us
I20260709 12:27:55.323693 26120 rpcz_store.cc:276] 0709 12:27:53.960909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.033288 (+ 72379us) service_pool.cc:224] Handling call
0709 12:27:54.033685 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.195689 (+162004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.191401 (+995712us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54002ca1d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:55.191638 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323081600}
W20260709 12:27:55.372689 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.367384  3076 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.324636 26128 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:55.360340 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:55.257814 26126 rpcz_store.cc:269] 0709 12:27:53.547625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.648188 (+100563us) service_pool.cc:224] Handling call
0709 12:27:55.174842 (+1526654us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400287654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:55.175009 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259756928}
I20260709 12:27:55.355777  3295 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:55.202389  3156 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:55.361788  3413 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.327312  3025 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.362490 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.254396  3044 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.040637  3405 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.362353  3242 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.257872  3045 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.356477  2830 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.182164  3257 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:55.373835  3369 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:55.258632  3203 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.086498  3361 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:55.193230  3358 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.064s	user 0.003s	sys 0.000s
I20260709 12:27:55.377048  3382 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:55.377193 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 500270us
I20260709 12:27:55.153576  3400 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.378172  3318 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.323890 26127 rpcz_store.cc:276] 0709 12:27:54.126720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.126871 (+   151us) service_pool.cc:224] Handling call
0709 12:27:54.127207 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.207830 (+ 80623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.190432 (+982602us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b5400185bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:55.190771 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96749952}
I20260709 12:27:55.379421 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.498418 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:55.342259 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.499320 26048 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
W20260709 12:27:55.384222  3252 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.882s	user 0.010s	sys 0.002s
I20260709 12:27:55.384858  3379 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:55.503584 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:55.386600  3383 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.371937 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:55.227363  3174 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.222s	user 0.010s	sys 0.003s
W20260709 12:27:55.237107  3264 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.001s
W20260709 12:27:55.173622  3398 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 12:27:55.014426  3338 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.341s
W20260709 12:27:55.240839  2936 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.120s	user 0.010s	sys 0.003s
W20260709 12:27:55.039222  3236 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.828s	user 0.013s	sys 0.004s
I20260709 12:27:55.009760  3312 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.317s
I20260709 12:27:55.401724 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.451126  3405 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.174454  3310 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:55.170943  3331 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:55.555008  3331 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:55.555323  3149 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:55.042399  3396 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 12:27:55.199894  3319 consensus_meta.cc:287] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.976s	user 0.009s	sys 0.002s
W20260709 12:27:55.557077  3391 log.cc:927] Time spent T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.675s	user 0.000s	sys 0.002s
I20260709 12:27:55.556957 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.557317  3025 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:55.557811 26199 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:55.557521  3319 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.558867 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.559314 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2276) took 1206 ms. Trace:
I20260709 12:27:55.184020  3373 raft_consensus.cc:3055] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:55.039538  3359 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.259850  3381 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:55.412935 26051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.013s	sys 0.056s
W20260709 12:27:55.303457  2995 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.043s	user 0.010s	sys 0.003s
I20260709 12:27:55.407130 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.202418  3275 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.515933  3408 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:55.211889  2910 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.022650  3316 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.403411 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:55.241964  3378 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:55.419876 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:27:55.415369 26194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.004s	sys 0.016s
W20260709 12:27:55.442538 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 51821us
W20260709 12:27:55.451810 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.033s	sys 0.021s
I20260709 12:27:55.327740  3376 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.406560  3311 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:55.487093 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.450142  3045 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:55.453647  2830 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:55.613803 26194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.005s	sys 0.024s
I20260709 12:27:55.560089 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.499971 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.584272 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:55.617417 26129 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:27:55.618535 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.619153 26129 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:55.500897 26200 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:55.500882 26048 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:55.454293  3257 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:55.386287 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.625684 26201 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 30 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:27:55.505102 26049 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:27:55.506486 26054 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:55.508441 26053 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:27:55.392592  3133 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.633400 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:55.633855 26052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4284) took 1736 ms (client timeout 1552 ms). Trace:
I20260709 12:27:55.634052 26200 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
W20260709 12:27:55.634321 26052 rpcz_store.cc:269] 0709 12:27:53.889309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.894052 (+  4743us) service_pool.cc:224] Handling call
0709 12:27:55.624970 (+1730918us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540022c254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:55.625135 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509430528}
I20260709 12:27:55.635053  3133 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:55.458254  3354 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:55.635789 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:55.635905  3354 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.636485 26048 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
W20260709 12:27:55.555220  3276 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.712s	user 0.012s	sys 0.000s
I20260709 12:27:55.446499  3220 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:27:55.493171  3382 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.042313  3397 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.516942 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:55.520531 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.516402 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:55.398257  3356 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.740s	user 0.006s	sys 0.005s
I20260709 12:27:55.526310 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.503949  3379 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:55.537405 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.706933 26122 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:55.620797  3373 raft_consensus.cc:740] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:55.519304  3174 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.527324  3236 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.544495  3217 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.521327  3264 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:55.546418  3076 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:55.521926  3263 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 12:27:55.550053  3203 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.550185 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.550053  3044 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:55.507510  3252 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:55.552867  3387 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.822s	user 0.000s	sys 0.003s
I20260709 12:27:55.553375 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3351) took 2922 ms. Trace:
I20260709 12:27:55.527503  2936 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.455231  3413 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.561031 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:55.615779 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.037s	sys 0.021s
I20260709 12:27:55.560544 26120 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 22.
W20260709 12:27:55.621194 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 217873us
I20260709 12:27:55.559741 26784 rpcz_store.cc:276] 0709 12:27:54.352967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.353183 (+   216us) service_pool.cc:224] Handling call
0709 12:27:54.373406 (+ 20223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.373757 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.373785 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:54.393242 (+ 19457us) log.cc:844] Serialized 26 byte log entry
0709 12:27:54.705989 (+312747us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:54.706039 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.706110 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.556652 (+850542us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b24003235b8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:27:55.558987 (+  2335us) raft_consensus.cc:1685] finished
0709 12:27:55.559012 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.559249 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16464,"raft.queue_time_us":178097,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":21539,"spinlock_wait_cycles":15291264,"thread_start_us":205853,"threads_started":1,"wal-append.queue_time_us":360228}
I20260709 12:27:55.583000 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:55.620168 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:55.613720 26051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.016s	sys 0.059s
I20260709 12:27:55.461341 26123 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:55.628404 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:55.634996 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:55.638069 26202 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:55.498510 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:55.510551 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:55.630932 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:55.663401 26199 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:55.569571  3357 consensus_meta.cc:287] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.857s	user 0.008s	sys 0.003s
W20260709 12:27:55.560995  3417 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:27:55.566954  3403 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.527s	user 0.002s	sys 0.002s
I20260709 12:27:55.578423  3292 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:55.560024  3319 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:55.695868 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.567386  3320 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:55.565097  3383 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:55.704473 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 188239us
I20260709 12:27:55.622615  3310 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:55.563012  3400 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:55.540345  3218 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:55.637954  3316 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.708748 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:55.677443  3376 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.119s
I20260709 12:27:55.637686  3257 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.645473  2995 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.639178  3359 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.709254  3379 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 25.21, Last appended by leader: 21, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.652599  3408 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.556058  3331 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:55.573868  3361 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:55.565124  3405 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.161756 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4269) took 2548 ms. Trace:
W20260709 12:27:56.165071 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 534148us
I20260709 12:27:56.164086 26196 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 23.
I20260709 12:27:56.170239 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3334) took 1349 ms. Trace:
I20260709 12:27:56.163723 26197 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.166067 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:56.167896 26129 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:27:55.798354 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.171854 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.760820 26047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4279) took 2091 ms (client timeout 2061 ms). Trace:
I20260709 12:27:56.173635 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:56.174371 26047 rpcz_store.cc:269] 0709 12:27:53.669183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.758023 (+ 88840us) service_pool.cc:224] Handling call
0709 12:27:55.760594 (+2002571us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:55.760746 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446249728}
I20260709 12:27:56.171011 26196 rpcz_store.cc:276] 0709 12:27:54.821070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.522697 (+701627us) service_pool.cc:224] Handling call
0709 12:27:56.170151 (+647454us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:55.757056 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 232883us
W20260709 12:27:56.175792 26018 connection.cc:256] RPC call timeout handler was delayed by -1.3705014950000987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 12:27:56.176324 26018 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.591s (SENT)
I20260709 12:27:55.637873  3312 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:56.176744 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:55.709434 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.176967 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.177851 26788 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:56.163767 26045 rpcz_store.cc:276] 0709 12:27:53.613052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.640708 (+ 27656us) service_pool.cc:224] Handling call
0709 12:27:53.658098 (+ 17390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.774676 (+116578us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:54.195568 (+420892us) log.cc:844] Serialized 14 byte log entry
0709 12:27:54.446523 (+250955us) log.cc:844] Serialized 14 byte log entry
0709 12:27:54.451714 (+  5191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:54.451774 (+    60us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:54.451828 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.327203 (+875375us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b180032e8a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:55.367968 (+ 40765us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b180032e8a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:55.376507 (+  8539us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b180032e8a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419
0709 12:27:56.161397 (+784890us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400228b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:56.161471 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.161673 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4523,"prepare.queue_time_us":566860,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4802,"raft.queue_time_us":10876,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":48590,"spinlock_wait_cycles":379923712,"thread_start_us":644219,"threads_started":2,"wal-append.queue_time_us":454203,"wal-append.run_cpu_time_us":2717,"wal-append.run_wall_time_us":1266241}
I20260709 12:27:56.179721 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1773) took 1159 ms. Trace:
I20260709 12:27:56.163391 26782 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:56.180112 26121 rpcz_store.cc:276] 0709 12:27:55.020136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.020279 (+   143us) service_pool.cc:224] Handling call
0709 12:27:55.550047 (+529768us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280eee 000000000036eb39 00007f5b792f41e0 00007f5b793bd0c4 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:27:56.179672 (+629625us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328704}
I20260709 12:27:56.165103 26124 rpcz_store.cc:276] 0709 12:27:52.631196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.880354 (+249158us) service_pool.cc:224] Handling call
0709 12:27:52.880703 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:52.881069 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:52.881101 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:52.881627 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:27:53.807189 (+925562us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:53.807228 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.807291 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.192340 (+1385049us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:55.553033 (+360693us) raft_consensus.cc:1685] finished
0709 12:27:55.553060 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.553308 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32231,"raft.queue_time_us":1993,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":214542,"spinlock_wait_cycles":13730048,"thread_start_us":925167,"threads_started":1,"wal-append.queue_time_us":1122670}
W20260709 12:27:56.156783 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 476522us
I20260709 12:27:56.179659 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3340) took 1159 ms. Trace:
I20260709 12:27:56.183068 26199 rpcz_store.cc:276] 0709 12:27:55.020659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.559832 (+539173us) service_pool.cc:224] Handling call
0709 12:27:56.179602 (+619770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.164070 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.181051 26201 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:27:56.186980 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.187899 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.188002 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:56.167047 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.189811  3149 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:56.190336  3149 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.189922  3310 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.190744  3354 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:56.184873  3133 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:56.191831  3133 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.192207  3133 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:56.192605  3363 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.193142  3363 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.193733  3252 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.194876 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.195215  3363 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.195485 26201 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:55.696909  3382 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:56.198313 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.189270 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:56.201924 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.205366  3045 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.528s	user 0.004s	sys 0.007s
I20260709 12:27:56.205756  3316 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:56.205825  3045 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.206436  3044 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:56.182029  3312 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.190790  3149 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:56.209411 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.209653  3408 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:56.035928  3423 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:56.210186 26122 raft_consensus.cc:2393] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:56.203748 26203 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:27:56.212692 26046 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:56.214926 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.208187  3312 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:56.215193 26195 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.203083 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:56.202328 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1869276470001751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 12:27:56.217509 26018 connection.cc:256] RPC call timeout handler was delayed by -1.1276726410001174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:27:56.218633 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 51664.7us
I20260709 12:27:56.211175 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:56.220518 26018 connection.cc:256] RPC call timeout handler was delayed by -1.0129236520001541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 12:27:56.220796 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:27:56.220870 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:56.222100 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.219347 26783 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:55.711490  3276 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.219982 26128 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:27:55.712996  3220 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.222440 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.219213 26126 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:56.224175  3076 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:56.224359 26094 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:56.178894  3323 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.225082 26122 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.226223 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.230234 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:55.796586  3271 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:56.225721 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:56.201045  3331 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:56.210860  3381 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.534s	user 0.002s	sys 0.008s
I20260709 12:27:56.202452  3379 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:56.229274 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.229890 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:55.710246  3406 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.541s
I20260709 12:27:56.201498  3363 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.230893 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:56.166666  3292 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:56.216413  3383 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.211441  3176 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:56.218444  3354 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:56.231875 26197 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:56.231861 26129 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:27:56.233619 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.232476  3408 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:56.222862  3257 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:56.218768  3045 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:56.243072 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.269781 26790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2278) took 1822 ms. Trace:
I20260709 12:27:56.270484 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:56.246747 26199 raft_consensus.cc:2368] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 31. Current term is 32.
I20260709 12:27:56.224668  3044 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:56.225409  3217 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.706561  3356 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.247998 26201 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:56.250159 26200 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
W20260709 12:27:56.249594 26018 leader_election.cc:336] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.558s (SENT)
I20260709 12:27:56.276635 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:56.228219  3276 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:56.185493  3357 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.228767  3316 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.228621  3378 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:56.232769  3252 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:56.260390  3203 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:56.288798 26018 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.823s (SENT)
I20260709 12:27:56.239473  3405 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.240181  3264 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.266439 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.293337 26129 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:56.278684 26197 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:27:56.231490  3220 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.232298  3218 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:27:56.263785 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.422719 26122 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:56.424187 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.232582  3076 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.241114  3323 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.262917  3025 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.029s
W20260709 12:27:56.303155  3351 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 2.177s	user 0.000s	sys 0.005s
W20260709 12:27:56.399050  3425 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.132s	user 0.003s	sys 0.000s
I20260709 12:27:56.306953  2995 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:56.301239 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.247664  3381 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.266919  3292 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.251678  3271 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.274473  3044 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:56.279585  3421 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:56.270313 26790 rpcz_store.cc:276] 0709 12:27:54.446924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.447216 (+   292us) service_pool.cc:224] Handling call
0709 12:27:54.447553 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:54.500739 (+ 53186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.500805 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:54.546199 (+ 45394us) log.cc:844] Serialized 52 byte log entry
0709 12:27:54.896912 (+350713us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:54.929707 (+ 32795us) log.cc:844] Serialized 14 byte log entry
0709 12:27:55.373851 (+444144us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b18003da020. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b7827958f 00007f5b712b8571 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd
0709 12:27:55.415002 (+ 41151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.415122 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.463926 (+ 48804us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b18003da020. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:27:55.566282 (+102356us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b24003235b8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b7113e9c4 00007f5b7113cacb 00007f5b712f7dc9 00007f5b712f7a55 00007f5b712f709d 00007f5b73f2910e 00007f5b73f4088c 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221
0709 12:27:56.269505 (+703223us) raft_consensus.cc:1685] finished
0709 12:27:56.269526 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.269688 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32736,"prepare.queue_time_us":24676,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":252991,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":23249,"spinlock_wait_cycles":409365888,"thread_start_us":763675,"threads_started":3,"wal-append.queue_time_us":437063}
I20260709 12:27:56.506836 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:56.308990  3045 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:56.285791  3378 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.285455  3276 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:56.292078  3316 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.285975  3252 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.327556  3357 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.319346 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:56.527915 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:56.528676 26129 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:56.308370 26125 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:56.294866 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:56.264434  3383 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.389968  3236 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:56.390525  3359 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.252349  3355 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:27:56.307109  3149 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.268330  3363 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:56.433450 26198 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.020s	sys 0.043s
I20260709 12:27:56.284373  3217 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:56.497792 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 59161.6us
W20260709 12:27:56.541415 26198 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.020s	sys 0.044s
I20260709 12:27:56.507053 26203 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.308023  3312 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.507437 26790 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.513000  2995 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:56.318775  3264 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.413340  3133 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:56.545708 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 1.03519e+06us
I20260709 12:27:56.507970  3044 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:56.506572  3174 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.511476  3292 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:56.513686  3378 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:56.232328  3373 consensus_queue.cc:260] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.543898 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.526511  3252 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:56.549281 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.550926 26129 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
W20260709 12:27:56.552304 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 378761us
I20260709 12:27:56.514815  3421 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:56.562181 26790 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
W20260709 12:27:56.563457 26194 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.003s	sys 0.016s
I20260709 12:27:56.581930 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:56.582443 26194 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.004s	sys 0.017s
I20260709 12:27:56.582749 26197 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.583478 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.584683 26757 leader_election.cc:400] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30.
W20260709 12:27:56.554240  2830 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.917s	user 0.008s	sys 0.006s
I20260709 12:27:56.573637 26784 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:56.541414  3413 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.592429 26199 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.585144 26757 leader_election.cc:403] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Cancelling election due to peer responding with higher term
I20260709 12:27:56.546957  2995 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.612093 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.613222 26790 raft_consensus.cc:2368] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28.
I20260709 12:27:56.614087 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:56.617148 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.564320  3356 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:56.563946 26127 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:56.588711 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:56.599185 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 369072us
I20260709 12:27:56.610116 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:56.575107  3397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.016s	sys 0.008s
I20260709 12:27:56.628392 26129 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:56.617174 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81116us
W20260709 12:27:56.636270 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 51774.1us
I20260709 12:27:56.636588 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.637241 26199 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:56.635455 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3342) took 1614 ms. Trace:
W20260709 12:27:56.639295 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.622336  2995 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:56.653654 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.638460 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.646463  3316 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:56.641399  3381 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.636114  3413 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.646662  3295 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:56.651155 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3330) took 1831 ms. Trace:
I20260709 12:27:56.639534 26198 rpcz_store.cc:276] 0709 12:27:55.021178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.575289 (+554111us) service_pool.cc:224] Handling call
0709 12:27:55.575716 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.576016 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:56.541926 (+965910us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.632765 (+ 90839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.632821 (+    56us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:56.632888 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.632968 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.635396 (+  2428us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1033784,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":993710,"threads_started":2}
I20260709 12:27:56.678766 26202 rpcz_store.cc:276] 0709 12:27:54.819924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.820107 (+   183us) service_pool.cc:224] Handling call
0709 12:27:56.650898 (+1830791us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:56.651083 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774573440}
I20260709 12:27:56.679507 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.680300 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.680325 26198 raft_consensus.cc:2368] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:56.656402 26784 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:56.659803 26129 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
W20260709 12:27:56.660980 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 221913us
I20260709 12:27:56.611002  3307 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:56.684051  3381 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:56.683952  3307 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.684523  3381 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.684916  3381 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:56.685355  3149 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:56.686092  3307 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:56.642772 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.687429 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.688254 26129 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:27:56.635171  3397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.008s
I20260709 12:27:56.625711  3174 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:27:56.674412 26093 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.682086 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.690336 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.691103 26199 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.610509  2830 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:56.681644 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.700407 26202 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:56.700678 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4383) took 1203 ms. Trace:
W20260709 12:27:56.700882 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 406085us
W20260709 12:27:56.701249 26019 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.590s (SENT)
I20260709 12:27:56.702598 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:56.703709 26126 rpcz_store.cc:276] 0709 12:27:55.497683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.526010 (+ 28327us) service_pool.cc:224] Handling call
0709 12:27:56.700508 (+1174498us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:56.700621 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1184580352}
I20260709 12:27:56.704185 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.704859 26129 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:56.704959 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.705330 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3370) took 1594 ms. Trace:
I20260709 12:27:56.705650 26126 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:56.705775 26129 rpcz_store.cc:276] 0709 12:27:55.110942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.703821 (+1592879us) service_pool.cc:224] Handling call
0709 12:27:56.705273 (+  1452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.650408  3433 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:56.707588 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2917) took 2241 ms. Trace:
I20260709 12:27:56.675102  3405 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.722289 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3367) took 1612 ms. Trace:
I20260709 12:27:56.722893 26126 rpcz_store.cc:276] 0709 12:27:55.110105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.704748 (+1594643us) service_pool.cc:224] Handling call
0709 12:27:56.722206 (+ 17458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.724261 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.725325 26126 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 33: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:56.712899 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.719369 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3351) took 1346 ms. Trace:
I20260709 12:27:56.726527 26196 rpcz_store.cc:276] 0709 12:27:55.372958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.245990 (+873032us) service_pool.cc:224] Handling call
0709 12:27:56.248222 (+  2232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.248500 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:56.549090 (+300590us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.718785 (+169695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.719093 (+   308us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:56.719142 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.719205 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.719318 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":394003,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"thread_start_us":450217,"threads_started":2}
I20260709 12:27:56.721239 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.728770 26199 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:56.730715 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.731725 26198 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:56.703719 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.709506 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.733072 26202 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:56.733440 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3371) took 1622 ms. Trace:
I20260709 12:27:56.733985 26129 rpcz_store.cc:276] 0709 12:27:55.111248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.708957 (+1597709us) service_pool.cc:224] Handling call
0709 12:27:56.733359 (+ 24402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.734222 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.735180 26202 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:56.735456 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:56.737004 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:56.739676 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.638814  3218 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:27:56.684054  3323 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:56.740945 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.741789 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.742559 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.744804 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:56.760057 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 78085.7us
I20260709 12:27:56.747828 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:56.756453 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.747148 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:56.764433 26126 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:56.778258 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:56.779222 26129 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:56.761704 26782 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:27:56.781412 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.783759 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4385) took 1266 ms. Trace:
I20260709 12:27:56.784320 26126 rpcz_store.cc:276] 0709 12:27:55.517428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.742267 (+1224839us) service_pool.cc:224] Handling call
0709 12:27:56.783674 (+ 41407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.719475  3376 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:56.743290  3356 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:56.726426  3405 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.720013  3260 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:56.719296  3435 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.693313  3413 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.749049  3295 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:56.744705  3220 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:56.792852 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.797418 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.811630 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.815938 26129 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
W20260709 12:27:56.818229 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 147856us
I20260709 12:27:56.778493  3355 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:56.822412 26787 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:56.797711 26126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "689b9b7393b349afbf302f0f99710830"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
W20260709 12:27:56.823194 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3363) took 1789 ms (client timeout 2130 ms). Trace:
I20260709 12:27:56.825160 26126 raft_consensus.cc:606] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:56.825775 26126 raft_consensus.cc:614] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:56.825254 26128 rpcz_store.cc:269] 0709 12:27:55.108756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.406792 (+298036us) service_pool.cc:224] Handling call
0709 12:27:56.823032 (+1416240us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54001d15d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:56.823133 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388322560}
I20260709 12:27:56.821110 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:56.821712 26196 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:56.829526 26129 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:56.830206 26128 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:27:56.836404 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3369) took 1725 ms. Trace:
I20260709 12:27:56.837239 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:56.837482 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58316 (request call id 47) took 1792 ms. Trace:
I20260709 12:27:56.840418 26126 rpcz_store.cc:276] 0709 12:27:55.033815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.797556 (+1763741us) service_pool.cc:224] Handling call
0709 12:27:56.826218 (+ 28662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.836925 26128 rpcz_store.cc:276] 0709 12:27:55.110700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.826572 (+1715872us) service_pool.cc:224] Handling call
0709 12:27:56.827418 (+   846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.836094 (+  8676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.836345 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979968}
W20260709 12:27:56.844091 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.791352 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2914) took 2329 ms. Trace:
I20260709 12:27:56.839594 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:56.825058 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.846810 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:56.848412 26196 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:56.849716 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.796399  3388 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:56.707463  3421 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.850025 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:56.851214 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1776) took 1808 ms. Trace:
I20260709 12:27:56.863778 26129 rpcz_store.cc:276] 0709 12:27:55.022199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.826540 (+1804341us) service_pool.cc:224] Handling call
0709 12:27:56.827027 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.829956 (+  2929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.830934 (+   978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22912}
I20260709 12:27:56.845144 26202 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:56.820668  3264 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.820755  3447 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.809405  3405 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:56.824792  3331 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.003s
I20260709 12:27:56.811111  3435 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.854521 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.829346  3217 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:56.847570 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3009) took 1154 ms. Trace:
W20260709 12:27:56.830801  3428 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.293s	user 0.002s	sys 0.000s
I20260709 12:27:56.798110  3314 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:56.845696  3307 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:56.845748  3430 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:56.817366  3356 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:56.854151 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:56.794101  3362 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:56.844882 26051 rpcz_store.cc:276] 0709 12:27:54.461438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.461594 (+   156us) service_pool.cc:224] Handling call
0709 12:27:54.462024 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.462441 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.462483 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:54.473089 (+ 10606us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.169361 (+1696272us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:56.554857 (+385496us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.790895 (+236038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.791007 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.791075 (+    68us) raft_consensus.cc:1685] finished
0709 12:27:56.791094 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.791270 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22008,"prepare.queue_time_us":97693,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":44379,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":39860,"thread_start_us":2157753,"threads_started":4,"wal-append.queue_time_us":1002988,"wal-append.run_cpu_time_us":2407,"wal-append.run_wall_time_us":1021873}
I20260709 12:27:56.830084  3383 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:56.829811  3439 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.879104 26129 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:27:56.873178 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:56.864167 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.879614 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1580) took 2066 ms. Trace:
I20260709 12:27:56.847470  3432 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.135s
I20260709 12:27:56.975798 26198 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
W20260709 12:27:57.053174 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 199074us
I20260709 12:27:57.030113 26196 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:27:56.882488 26201 rpcz_store.cc:276] 0709 12:27:55.692960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.275633 (+582673us) service_pool.cc:224] Handling call
0709 12:27:56.276187 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.276920 (+   733us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:56.661834 (+384914us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.847029 (+185195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.847247 (+   218us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:56.847303 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.847374 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.847506 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":365815,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"raft.queue_time_us":107,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":12085,"thread_start_us":552422,"threads_started":2}
I20260709 12:27:56.986967 26199 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:57.055030 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.054153 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:57.055725 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:56.965488 26128 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:56.952200 26789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2285) took 1369 ms. Trace:
I20260709 12:27:56.866603  3076 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:57.084144 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:56.998281 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.254005 26053 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:27:57.059568 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:56.865576  3437 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.356s
I20260709 12:27:57.256199 26051 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:57.256640 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4378) took 1844 ms. Trace:
I20260709 12:27:57.056365 26049 raft_consensus.cc:1275] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:57.257254 26054 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:56.972796 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87425.4us
I20260709 12:27:57.258308 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.122432 26052 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:27:56.862424  3355 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:57.148572 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:57.059381 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 312337us
I20260709 12:27:57.229029 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:57.115877 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.230981 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.260061 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4375) took 1926 ms. Trace:
I20260709 12:27:57.257207 26129 rpcz_store.cc:276] 0709 12:27:55.412074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.877796 (+1465722us) service_pool.cc:224] Handling call
0709 12:27:56.878513 (+   717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.256368 (+377855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.256580 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.056646 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:56.861547  3263 raft_consensus.cc:2764] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:27:57.258453 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:57.052095 26194 rpcz_store.cc:276] 0709 12:27:54.813107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.813324 (+   217us) service_pool.cc:224] Handling call
0709 12:27:54.813642 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.813925 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:27:54.821190 (+  7265us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.587032 (+1765842us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.878621 (+291589us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.878773 (+   152us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:56.879040 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.879422 (+   382us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.879558 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58666,"prepare.queue_time_us":1037952,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":88,"raft.queue_time_us":80524,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":48645,"thread_start_us":1999403,"threads_started":4,"wal-append.queue_time_us":1063754,"wal-append.run_cpu_time_us":1366,"wal-append.run_wall_time_us":1022059}
W20260709 12:27:57.264629 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.278860 26788 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:57.264441 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.279728 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:57.279947 26196 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:57.280673 26053 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:57.280802 26129 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:57.255117 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.122139 26094 leader_election.cc:400] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 31. Current term is 32.
I20260709 12:27:57.264801 26782 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:57.281773 26094 leader_election.cc:403] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
W20260709 12:27:57.279768 26786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2287) took 1696 ms (client timeout 1701 ms). Trace:
I20260709 12:27:57.278362 26128 rpcz_store.cc:276] 0709 12:27:55.333989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.877096 (+1543107us) service_pool.cc:224] Handling call
0709 12:27:56.965309 (+ 88213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.259830 (+294521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.260007 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.282582 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:57.279922 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:57.285156 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 163145us
I20260709 12:27:57.286800 26129 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:57.260531 26789 rpcz_store.cc:276] 0709 12:27:55.582846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.583002 (+   156us) service_pool.cc:224] Handling call
0709 12:27:55.583448 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:55.583993 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.584051 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:55.584943 (+   892us) log.cc:844] Serialized 52 byte log entry
0709 12:27:55.768992 (+184049us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:55.769053 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.769165 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.951977 (+1182812us) raft_consensus.cc:1685] finished
0709 12:27:56.951996 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.952141 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44753,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":175101,"threads_started":1,"wal-append.queue_time_us":909238}
I20260709 12:27:57.288576 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.293527 26047 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:57.278712 26048 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:27:57.289852 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.285461 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:27:57.282550 26786 rpcz_store.cc:269] 0709 12:27:55.583517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.585509 (+  1992us) service_pool.cc:224] Handling call
0709 12:27:57.230758 (+1645249us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:27:57.279707 (+ 48949us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16995712}
I20260709 12:27:56.943420  2830 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:56.956954  3257 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.003s
I20260709 12:27:57.297880 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.302546 26052 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:56.881022  3149 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:57.299242 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:56.876397  3312 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:57.015931  3379 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.003s
I20260709 12:27:56.965660  3045 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:56.882514  3388 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:56.884140  3435 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:57.221570  3378 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.001s
I20260709 12:27:57.306617 26053 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:57.163563  2936 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:57.308832 26196 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:27:56.922290  3133 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:57.332333 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:57.333163 26788 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:27:57.052471  3439 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.296952  3314 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:57.089087  3323 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:57.365108 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.365696 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1775) took 2343 ms. Trace:
I20260709 12:27:57.125341  3025 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.306605 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:56.966420  3413 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.366444 26125 rpcz_store.cc:276] 0709 12:27:55.021909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.034398 (+ 12489us) service_pool.cc:224] Handling call
0709 12:27:55.034781 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.531885 (+1497104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.365373 (+833488us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540010e0d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:57.365620 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2928437248}
I20260709 12:27:57.014866  3316 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.299s
W20260709 12:27:57.291640  3382 consensus_meta.cc:287] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.063s	user 0.013s	sys 0.000s
I20260709 12:27:57.319031 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.278182  3355 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.257035  3383 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:57.391808 26787 raft_consensus.cc:2368] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:57.016057  3376 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.282s
I20260709 12:27:57.330042 26054 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:57.331616 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:57.374259 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:56.947198  3363 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:57.343768  2995 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.666s	user 0.007s	sys 0.000s
I20260709 12:27:57.347817 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:56.942757  3295 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.161705  3356 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:57.350425  3433 consensus_meta.cc:287] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.604s	user 0.008s	sys 0.001s
I20260709 12:27:57.023901  3044 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.397531  3413 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.351552  3133 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:27:57.353474  3292 consensus_meta.cc:287] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.000s
I20260709 12:27:57.355046  3405 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.290993  3448 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 31. Current term is 32.
I20260709 12:27:57.358364 26051 raft_consensus.cc:1240] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:27:57.358806 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:57.490147 26202 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: [31.17-31.17]
I20260709 12:27:57.490289 26122 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:57.359849 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:57.313717 26201 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:56.887156  3220 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.430s
I20260709 12:27:57.368583 26053 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:56.945552  3252 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.310204  3076 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.366108 26052 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:57.518549 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.031332  3276 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.393445 26128 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:57.330744  3378 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.394296 26125 raft_consensus.cc:2410] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:27:56.956172  3381 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.116432  3362 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.535319 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:57.536303 26049 raft_consensus.cc:2368] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:57.397221 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.404474 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.537050 26045 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:57.538007 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.538902  3276 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.539191 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.539976 26788 raft_consensus.cc:2410] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 21.
W20260709 12:27:57.407227 26791 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2291) took 1820 ms (client timeout 2156 ms). Trace:
W20260709 12:27:57.540436 26791 rpcz_store.cc:269] 0709 12:27:55.586869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.638661 (+ 51792us) service_pool.cc:224] Handling call
0709 12:27:57.347656 (+1708995us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b5800003c98. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7936b0a0 00007f5b7936a984 00007f5b728e56d4 00007f5b728e5662
0709 12:27:57.407134 (+ 59478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19660032}
I20260709 12:27:57.541514 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:57.542531 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.543032 26791 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:27:57.543785 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:57.308398  3264 consensus_queue.cc:1059] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.091s
I20260709 12:27:57.464349  3413 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.524933  3388 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:57.545034 26046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4291) took 2049 ms (client timeout 2179 ms). Trace:
W20260709 12:27:57.545469 26046 rpcz_store.cc:269] 0709 12:27:55.495538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.495674 (+   136us) service_pool.cc:224] Handling call
0709 12:27:57.544847 (+2049173us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400172d54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.544976 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628553984}
I20260709 12:27:57.546559  3413 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:57.546649 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.406540  2995 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:56.885764  3447 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.548455  3044 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.549576  2995 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:57.550016  3447 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:57.550927 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1569) took 4869 ms. Trace:
I20260709 12:27:57.551323 26783 rpcz_store.cc:276] 0709 12:27:52.681264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.843573 (+1162309us) service_pool.cc:224] Handling call
0709 12:27:53.843932 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.223630 (+2379698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.550644 (+1327014us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b54004f4554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:57.550847 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5462415616}
I20260709 12:27:57.552502 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.553085  3435 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.553673 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.554766 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:57.555450 26783 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:57.555315  3435 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:57.364306  3314 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:57.414562 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 543844us
W20260709 12:27:57.416982 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.392531  3355 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:57.369189  3323 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.364073  3439 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:57.505471  3133 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:57.391289  3025 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.523415 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.566820 26051 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:57.413184  3433 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30.
W20260709 12:27:57.565734 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 245855us
I20260709 12:27:57.533310  3252 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.534225  3378 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:57.535230  3381 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.536695 26197 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:57.558056 26199 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:57.391846  3382 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.548022 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.559279 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.572991 26783 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:57.574260 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.574478 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.574869 26783 raft_consensus.cc:2376] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28. Re-sending same reply.
I20260709 12:27:57.575812 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.576201 26046 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:27:57.565078  3323 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.560460 26053 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:27:57.536118 26125 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:57.575116 26788 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:57.401503  3383 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.562341 26791 raft_consensus.cc:2393] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:57.322006  3149 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.567487 26128 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:27:57.614527 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:57.615346 26128 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:57.615752 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.526968 26202 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:57.616766 26125 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:27:57.568812 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.561653  3314 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:57.561365  3076 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.623996 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:57.620775 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.578275  3044 raft_consensus.cc:3055] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:57.630752 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:57.623587 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:57.595609 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1579) took 1010 ms. Trace:
I20260709 12:27:57.619097 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:57.584612 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:57.631738 26783 rpcz_store.cc:276] 0709 12:27:56.585050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.575510 (+990460us) service_pool.cc:224] Handling call
0709 12:27:57.595490 (+ 19980us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:57.596987 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 306464us
I20260709 12:27:57.632277 26125 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.634102 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.635046 26202 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:57.633251 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:57.640941 26783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce"
dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:46996
I20260709 12:27:57.641525  3263 consensus_queue.cc:1048] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:57.641633 26169 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:57.643190  3363 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:57.643607  3363 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.643976  3363 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:57.644629  3435 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:57.645177  3435 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:57.646000  3435 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.648406  3376 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:57.641896  3378 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:57.652774 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:57.641120  3410 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:57.641610 26783 raft_consensus.cc:606] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:57.656948 26783 raft_consensus.cc:614] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:57.654798 26198 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.656108 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:57.633934  3260 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:57.726233  3260 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.629963  3044 raft_consensus.cc:740] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:57.726958  3260 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:57.727698  3044 consensus_queue.cc:260] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.729182  3044 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:57.657235  3198 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.010s
I20260709 12:27:57.633008  3133 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:57.672098  3319 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:27:57.677193  3464 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:57.654479  3378 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.732542  3319 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:57.732920  3378 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:57.656760  3410 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:57.734661  3410 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.735882  3410 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.686319  3382 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.686352 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38286 (request call id 2292) took 2055 ms. Trace:
I20260709 12:27:57.737726 26784 rpcz_store.cc:276] 0709 12:27:55.631255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.219281 (+588026us) service_pool.cc:224] Handling call
0709 12:27:57.686117 (+1466836us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54000587d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.686267 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753256704}
I20260709 12:27:57.739766 26784 raft_consensus.cc:1240] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:27:57.740197  3405 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:57.664829  3263 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.031s
I20260709 12:27:57.694500  3025 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:57.699083 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1588) took 1029 ms. Trace:
I20260709 12:27:57.741910 26782 rpcz_store.cc:276] 0709 12:27:56.669523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.850667 (+181144us) service_pool.cc:224] Handling call
0709 12:27:57.698834 (+848167us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b5400821254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.698990 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880307968}
I20260709 12:27:57.746280  3319 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:57.706468  3427 log.cc:927] Time spent T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.439s	user 0.010s	sys 0.008s
W20260709 12:27:57.710603 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.716800 26789 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:57.758080  3263 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:57.764009  3263 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:57.766920  3263 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:57.802593  3263 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:57.793830  3319 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:57.799938  3373 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:57.806504  3310 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.831269  3439 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:57.702917 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3345) took 2617 ms (client timeout 1850 ms). Trace:
I20260709 12:27:57.703683 26196 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:57.676492 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3353) took 1094 ms. Trace:
I20260709 12:27:57.675094 26199 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:57.691664 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:27:57.716952 26203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 2999) took 2087 ms (client timeout 2273 ms). Trace:
I20260709 12:27:57.654767 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:57.701903 26786 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:57.873992 26048 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:27:57.922508 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.884958 26052 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
W20260709 12:27:57.824900  3406 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.557s	user 0.010s	sys 0.005s
W20260709 12:27:57.921391 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.878553 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.868345 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:57.994349 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 1949) took 1135 ms. Trace:
I20260709 12:27:57.995241 26788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2294) took 1457 ms. Trace:
W20260709 12:27:57.976869 26127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4382) took 2516 ms (client timeout 1505 ms). Trace:
I20260709 12:27:57.994933 26786 rpcz_store.cc:276] 0709 12:27:56.858935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.700949 (+842014us) service_pool.cc:224] Handling call
0709 12:27:57.701442 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.985548 (+284106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.993866 (+  8318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.001086 26045 raft_consensus.cc:2368] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30.
I20260709 12:27:58.002488 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.003539 26045 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33.
I20260709 12:27:58.004510  3257 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.004863  3356 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:58.005744  3257 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.006136  3356 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.007961 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1584) took 1266 ms. Trace:
W20260709 12:27:57.998142 26127 rpcz_store.cc:269] 0709 12:27:55.460740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.498297 (+ 37557us) service_pool.cc:224] Handling call
0709 12:27:57.975935 (+2477638us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54001f54d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.976479 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906600704}
I20260709 12:27:58.008519 26199 rpcz_store.cc:276] 0709 12:27:56.741133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.674426 (+933293us) service_pool.cc:224] Handling call
0709 12:27:57.674838 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.986824 (+311986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.007645 (+ 20821us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.975914 26197 rpcz_store.cc:276] 0709 12:27:56.582163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.609774 (+ 27611us) service_pool.cc:224] Handling call
0709 12:27:57.676169 (+1066395us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.676399 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088150912}
W20260709 12:27:58.012008 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.015586 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:58.016397 26195 rpcz_store.cc:269] 0709 12:27:55.085693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.200135 (+114442us) service_pool.cc:224] Handling call
0709 12:27:57.702590 (+2502455us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b5400336b54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.702821 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049484160}
I20260709 12:27:57.871619 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:58.010097 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:27:57.985514 26203 rpcz_store.cc:269] 0709 12:27:55.629392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.214665 (+585273us) service_pool.cc:224] Handling call
0709 12:27:57.716664 (+1501999us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54002b4154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:57.716867 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665215744}
I20260709 12:27:58.019088 26045 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:27:58.025588 26202 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:58.025600 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:57.999359  3149 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:27:57.632761 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48316.4us
W20260709 12:27:58.047010  3406 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.724s	user 0.010s	sys 0.006s
I20260709 12:27:58.048751 26045 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 27 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
W20260709 12:27:58.049151 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 394323us
I20260709 12:27:56.717157 26050 rpcz_store.cc:276] 0709 12:27:54.466210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.828804 (+362594us) service_pool.cc:224] Handling call
0709 12:27:54.855361 (+ 26557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.855653 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:54.927318 (+ 71665us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.707049 (+1779731us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800351dc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:27:56.707252 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.707299 (+    47us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:56.707352 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.707415 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.707535 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":164108,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":6651,"spinlock_wait_cycles":23405952,"thread_start_us":232484,"threads_started":2,"wal-append.queue_time_us":519449}
I20260709 12:27:58.050431 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.052104 26050 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:58.053305 26047 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.054618 26050 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 18. (term mismatch)
I20260709 12:27:58.055136 26050 pending_rounds.cc:85] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 17
I20260709 12:27:58.055457 26050 pending_rounds.cc:107] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 27.18
I20260709 12:27:58.055840 26050 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:57.995947 26788 rpcz_store.cc:276] 0709 12:27:56.538027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.644344 (+1106317us) service_pool.cc:224] Handling call
0709 12:27:57.644729 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:57.646699 (+  1970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.646752 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:57.647574 (+   822us) log.cc:844] Serialized 52 byte log entry
0709 12:27:57.994317 (+346743us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:57.994437 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.994643 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.994753 (+   110us) raft_consensus.cc:1685] finished
0709 12:27:57.994769 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.994968 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1324,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":338662,"threads_started":1,"wal-append.queue_time_us":154213}
I20260709 12:27:58.056193  3439 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:58.058751 26197 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:58.075294 26046 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:27:58.067919 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.069660  3382 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:27:58.056469 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 410146us
I20260709 12:27:58.058547 26050 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:27:58.067137 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:58.083082 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 212166us
I20260709 12:27:58.074771 26197 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:58.065910 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.081018  3382 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:58.086206  3382 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:58.086517 26785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1589) took 1399 ms. Trace:
I20260709 12:27:58.087008 26785 rpcz_store.cc:276] 0709 12:27:56.677725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.650775 (+973050us) service_pool.cc:224] Handling call
0709 12:27:57.651164 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.655605 (+  4441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.655633 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.656127 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 12:27:58.024932 (+368805us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:58.025612 (+   680us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.030248 (+  4636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.030370 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.035257 (+  4887us) raft_consensus.cc:1685] finished
0709 12:27:58.035281 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.076714 (+ 41433us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4044,"prepare.queue_time_us":9213,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":91,"raft.queue_time_us":9323,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":9076,"thread_start_us":305052,"threads_started":1,"wal-append.queue_time_us":161920}
W20260709 12:27:58.088790 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3364) took 2979 ms (client timeout 1815 ms). Trace:
I20260709 12:27:58.089062 26195 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
W20260709 12:27:58.089416 26123 rpcz_store.cc:269] 0709 12:27:55.109237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.355960 (+246723us) service_pool.cc:224] Handling call
0709 12:27:58.088278 (+2732318us) spinlock_profiling.cc:243] Waited 94.3 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.088566 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216860928}
I20260709 12:27:58.090204 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1587) took 1331 ms. Trace:
I20260709 12:27:58.090785 26195 rpcz_store.cc:276] 0709 12:27:56.758661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.021190 (+1262529us) service_pool.cc:224] Handling call
0709 12:27:58.088452 (+ 67262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.089747 (+  1295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.090100 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.091581 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.102732  2830 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:58.103618 26121 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:58.101233 26197 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.062057  3271 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:58.096455 26120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3368) took 2972 ms (client timeout 1865 ms). Trace:
I20260709 12:27:58.078397  3307 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.030s
W20260709 12:27:58.047335  3479 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.180s	user 0.002s	sys 0.000s
I20260709 12:27:58.150468 26047 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:27:58.147163 26120 rpcz_store.cc:269] 0709 12:27:55.110401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.371682 (+261281us) service_pool.cc:224] Handling call
0709 12:27:58.072490 (+2700808us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.082832 (+ 10342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109054208}
W20260709 12:27:58.170701 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103953us
I20260709 12:27:58.176007 26127 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:58.147040  3356 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:58.183730  3363 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.539s	user 0.000s	sys 0.018s
W20260709 12:27:58.195259 26200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 3003) took 2564 ms (client timeout 2024 ms). Trace:
I20260709 12:27:58.196390 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3016) took 1634 ms. Trace:
I20260709 12:27:58.170231 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:58.211853 26019 connection.cc:256] RPC call timeout handler was delayed by -1.4061161149998043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 12:27:58.185351  3388 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:58.184675 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.191900 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.186661 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3018) took 1621 ms. Trace:
I20260709 12:27:58.186484 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:58.199026 26125 consensus_meta.cc:287] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.003s
I20260709 12:27:58.208348 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2298) took 1492 ms. Trace:
I20260709 12:27:58.193228  3437 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:58.204919 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:58.206122 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.206416 26122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3372) took 3037 ms (client timeout 2954 ms). Trace:
W20260709 12:27:58.203277 26128 consensus_meta.cc:287] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.587s	user 0.015s	sys 0.009s
I20260709 12:27:58.226137 26196 rpcz_store.cc:276] 0709 12:27:56.565500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.321314 (+755814us) service_pool.cc:224] Handling call
0709 12:27:57.703428 (+382114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.984713 (+281285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.186199 (+201486us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400336b50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.186437 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852028288}
W20260709 12:27:58.231998 26122 rpcz_store.cc:269] 0709 12:27:55.254854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.423850 (+1168996us) service_pool.cc:224] Handling call
0709 12:27:58.206005 (+1782155us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.206185 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2449888000}
I20260709 12:27:58.248248 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.178283  2830 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:58.269989 26197 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:58.272398  3263 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:58.260847 26125 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:58.276863  3379 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:58.280351 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.297753 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1582) took 1581 ms. Trace:
W20260709 12:27:58.298197 26167 connection.cc:256] RPC call timeout handler was delayed by -1.4800689590001639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 12:27:58.298797 26167 connection.cc:256] RPC call timeout handler was delayed by -1.3968340070000522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.42400000000000004s for a response
W20260709 12:27:58.299813 26167 connection.cc:256] RPC call timeout handler was delayed by -1.1020689590000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
I20260709 12:27:58.300460  3252 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:58.300731  3260 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.011s
I20260709 12:27:58.301471  3252 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.301585  3260 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.304813 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:58.305512 26019 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.850s (SENT)
I20260709 12:27:58.304962 26121 raft_consensus.cc:2376] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28. Re-sending same reply.
I20260709 12:27:58.306322 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:58.307050 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.308182  3433 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.312292  3433 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.313124 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:58.304873 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61973.5us
I20260709 12:27:58.314296 26125 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:58.314823 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:58.316725 26790 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
W20260709 12:27:58.318701  3314 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.685s	user 0.011s	sys 0.007s
I20260709 12:27:58.315969  3356 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:58.231355  3437 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:58.308444 26121 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 31 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
W20260709 12:27:58.320493 26167 connection.cc:256] RPC call timeout handler was delayed by -1.101168959000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
W20260709 12:27:58.309810 26018 connection.cc:256] RPC call timeout handler was delayed by -1.2235437539998202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 12:27:58.326098 26018 connection.cc:256] RPC call timeout handler was delayed by -1.096643753999615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
I20260709 12:27:58.315804 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.305747 26788 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.327877 26125 raft_consensus.cc:2393] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:58.219182 26194 rpcz_store.cc:276] 0709 12:27:56.561694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.279055 (+717361us) service_pool.cc:224] Handling call
0709 12:27:57.301521 (+ 22466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.611647 (+310126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.611706 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.627628 (+ 15922us) log.cc:844] Serialized 26 byte log entry
0709 12:27:58.076097 (+448469us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:58.104762 (+ 28665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.104892 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.195920 (+ 91028us) raft_consensus.cc:1685] finished
0709 12:27:58.195948 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.196141 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55363,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":80,"thread_start_us":343140,"threads_started":2,"wal-append.queue_time_us":516259}
I20260709 12:27:58.322767 26127 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:58.329965  3323 consensus_meta.cc:287] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.712s	user 0.009s	sys 0.011s
I20260709 12:27:58.330502  3437 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.330657  3323 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.333212 26054 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:58.305409 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2300) took 1338 ms. Trace:
I20260709 12:27:58.336613 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.338546 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.298287 26201 rpcz_store.cc:276] 0709 12:27:56.716111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.494312 (+778201us) service_pool.cc:224] Handling call
0709 12:27:57.494659 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.514443 (+ 19784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.514495 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.527844 (+ 13349us) log.cc:844] Serialized 26 byte log entry
0709 12:27:58.052892 (+525048us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:58.164708 (+111816us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.288567 (+123859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.288720 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.288763 (+    43us) raft_consensus.cc:1685] finished
0709 12:27:58.288773 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.297528 (+  8755us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100729,"raft.queue_time_us":121538,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":21204,"thread_start_us":521732,"threads_started":2,"wal-append.queue_time_us":156363}
I20260709 12:27:58.342185  3252 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:58.346410 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:58.347643  3406 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:58.349393 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.350469 26129 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:58.351284 26129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:58.308866  3260 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:58.346493 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2930) took 1040 ms. Trace:
I20260709 12:27:58.331800  3437 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:58.219915 26200 rpcz_store.cc:269] 0709 12:27:55.630253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.634744 (+  4491us) service_pool.cc:224] Handling call
0709 12:27:58.194827 (+2560083us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b54002b5f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.194986 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107581824}
I20260709 12:27:58.319191  3314 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.238309  3363 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.624836 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:58.709863 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.710040 26052 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:27:58.711133 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:58.711851 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.293700 26128 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
W20260709 12:27:58.712405 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.712774 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 542610us
I20260709 12:27:58.421001 26123 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.411134 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3381) took 1145 ms. Trace:
I20260709 12:27:58.629326 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.617461 26127 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:27:58.551481 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.230127 26787 rpcz_store.cc:276] 0709 12:27:56.714676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.677972 (+963296us) service_pool.cc:224] Handling call
0709 12:27:57.678349 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.678668 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:58.040457 (+361789us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.186647 (+146190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.187496 (+   849us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:58.188754 (+  1258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.204942 (+ 16188us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.206599 (+  1657us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7855,"prepare.queue_time_us":133115,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":87,"raft.queue_time_us":6963,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":167,"thread_start_us":408526,"threads_started":2}
W20260709 12:27:58.343900 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 827697us
I20260709 12:27:58.355903 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:58.713261 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 366963us
I20260709 12:27:58.631014 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44004 (request call id 0) took 3012 ms. Trace:
I20260709 12:27:58.628750 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.719056 26120 rpcz_store.cc:276] 0709 12:27:55.618538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.629643 (+3011105us) service_pool.cc:224] Handling call
0709 12:27:58.630925 (+  1282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:58.441319 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.375907 26788 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:27:58.715668 26125 rpcz_store.cc:276] 0709 12:27:57.220868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.315320 (+1094452us) service_pool.cc:224] Handling call
0709 12:27:58.365694 (+ 50374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.708994 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:58.440227 26198 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.780s	user 0.003s	sys 0.016s
I20260709 12:27:58.727391 26048 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 33: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:58.721143 26049 rpcz_store.cc:276] 0709 12:27:57.306109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.306290 (+   181us) service_pool.cc:224] Handling call
0709 12:27:58.345970 (+1039680us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540022bd54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.346287 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386923776}
I20260709 12:27:58.729082 26047 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:58.473233 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:58.731341 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3387) took 1169 ms. Trace:
I20260709 12:27:58.708946 26129 raft_consensus.cc:606] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:58.735474 26129 raft_consensus.cc:614] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:58.319757  3356 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:58.706084  3044 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.976s	user 0.009s	sys 0.013s
I20260709 12:27:58.440351  3252 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:58.311548  3355 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.007s
I20260709 12:27:58.438660  3381 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.732610 26128 rpcz_store.cc:276] 0709 12:27:57.561950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.614139 (+ 52189us) service_pool.cc:224] Handling call
0709 12:27:58.723663 (+1109524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.740945 26052 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:58.736562 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 582874us
I20260709 12:27:58.325380  3257 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:58.740803 26122 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
W20260709 12:27:58.744328 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.472726  3390 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.479652  3323 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:58.747047 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 441544us
I20260709 12:27:58.748215 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:58.748831 26791 raft_consensus.cc:2368] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31.
I20260709 12:27:58.749020 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.750164 26757 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:58.751693  3373 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:58.755638  3378 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.022s	user 0.024s	sys 0.014s
W20260709 12:27:58.755647 25939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 26 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:27:58.756915 26167 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:58.758491 26125 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:58.626477  3383 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:58.323432  3388 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.742079  3356 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.785485 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:58.786648 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:58.787774  3076 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:58.789588  3363 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:58.803727  3487 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 12:27:58.648077  3025 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:58.792512 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 319461us
I20260709 12:27:58.336439 26782 rpcz_store.cc:276] 0709 12:27:56.967242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.742927 (+775685us) service_pool.cc:224] Handling call
0709 12:27:57.743245 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.743518 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:27:58.024785 (+281267us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.169449 (+144664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.172900 (+  3451us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:27:58.219764 (+ 46864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.242780 (+ 23016us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.304996 (+ 62216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5899,"prepare.queue_time_us":68635,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":18381,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":13362,"thread_start_us":347510,"threads_started":2}
I20260709 12:27:58.805891 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.565900  3382 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.795403 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.798256 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3382) took 1542 ms. Trace:
W20260709 12:27:58.803812 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53813.3us
I20260709 12:27:58.807807 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4390) took 3182 ms. Trace:
I20260709 12:27:58.807529 26122 rpcz_store.cc:276] 0709 12:27:57.253730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.339855 (+1086125us) service_pool.cc:224] Handling call
0709 12:27:58.796327 (+456472us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:58.808302 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.742578 26198 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
W20260709 12:27:58.795712 26197 consensus_meta.cc:287] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
W20260709 12:27:58.803853 25939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48498.7us
I20260709 12:27:58.808221 26125 rpcz_store.cc:276] 0709 12:27:55.618845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.738280 (+3119435us) service_pool.cc:224] Handling call
0709 12:27:58.758170 (+ 19890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.800948 (+ 42778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.801160 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.809527 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:58.810518 26051 consensus_meta.cc:287] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.087s	user 0.009s	sys 0.009s
I20260709 12:27:58.811090 26051 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:27:58.803876  3406 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:58.811678  3406 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:58.811818 26051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2943) took 1390 ms (client timeout 1530 ms). Trace:
I20260709 12:27:58.803727  3378 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:58.812492 26051 rpcz_store.cc:269] 0709 12:27:57.422001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.489351 (+ 67350us) service_pool.cc:224] Handling call
0709 12:27:58.811694 (+1322343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.812619  3044 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:58.813344 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.813705 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.814090 26122 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.814565 26051 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:58.814733  3378 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:58.812119  3406 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:58.815622 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.816987  3133 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:58.805166 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.818883 26052 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:27:58.819291 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.807289 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4389) took 3188 ms. Trace:
I20260709 12:27:58.799381  3257 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.819886 26121 rpcz_store.cc:276] 0709 12:27:55.615959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.424924 (+2808965us) service_pool.cc:224] Handling call
0709 12:27:58.734618 (+309694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.801232 (+ 66614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:58.801259 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.804181 (+  2922us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:58.804225 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.804313 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.804428 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.798368  3252 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:58.822741 26197 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:27:58.823002 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.823184 26197 pending_rounds.cc:85] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 18
I20260709 12:27:58.826205 26197 pending_rounds.cc:107] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:27:58.826557 26197 raft_consensus.cc:2889] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:58.755589  3355 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.828851 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48540 (request call id 3023) took 2256 ms. Trace:
I20260709 12:27:58.832782 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1586) took 2084 ms. Trace:
I20260709 12:27:58.441850  2830 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.825129 26048 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:58.833346 26197 rpcz_store.cc:276] 0709 12:27:56.748890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.016180 (+1267290us) service_pool.cc:224] Handling call
0709 12:27:58.017025 (+   845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.058612 (+ 41587us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b540031bda0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:27:58.832482 (+773870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.832735 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89681152}
I20260709 12:27:58.810551  3373 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.830705 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.829229 26047 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 24: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:58.835095  3236 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:27:58.832991 26198 rpcz_store.cc:276] 0709 12:27:56.568549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.652513 (+1083964us) service_pool.cc:224] Handling call
0709 12:27:58.820985 (+1168472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.823434  3383 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:58.835813 26049 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.835656 26127 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:58.824474  3381 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:58.834648 26052 raft_consensus.cc:2376] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28. Re-sending same reply.
I20260709 12:27:58.801753  3433 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:58.829922  3379 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.840416 26051 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:58.842113 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:58.842883 26128 raft_consensus.cc:2393] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:58.843830 26045 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:27:58.836172  3263 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.006s
I20260709 12:27:58.839489 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:58.845995  3260 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:58.849443 26122 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:58.851542  3310 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:58.830506  3133 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:58.853140 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.852689 26129 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:27:58.839809  3381 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.860236 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:58.854001 26046 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:58.868919 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3376) took 2674 ms. Trace:
I20260709 12:27:58.870438 26121 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 25. Current term is 26. Ops: [25.21-25.21]
W20260709 12:27:58.873037 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48363.3us
I20260709 12:27:58.874178 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.834412  3257 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.876195 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1779) took 2335 ms. Trace:
I20260709 12:27:58.869470 26129 rpcz_store.cc:276] 0709 12:27:56.186218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.821394 (+2635176us) service_pool.cc:224] Handling call
0709 12:27:58.846607 (+ 25213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.858207 (+ 11600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.860657 (+  2450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.872728 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2942) took 1460 ms. Trace:
I20260709 12:27:58.824178  3390 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.879514 26121 rpcz_store.cc:276] 0709 12:27:56.540619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.829442 (+2288823us) service_pool.cc:224] Handling call
0709 12:27:58.862051 (+ 32609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.875802 (+ 13751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.876115 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.880703 26045 rpcz_store.cc:276] 0709 12:27:57.412071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.049814 (+637743us) service_pool.cc:224] Handling call
0709 12:27:58.872533 (+822719us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.872670 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813613824}
I20260709 12:27:58.888976 26053 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 31 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:27:58.890462  3263 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:58.891059  3263 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:27:58.892886 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 144836us
W20260709 12:27:58.896425 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 139707us
I20260709 12:27:58.896484 26047 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:58.868163  3381 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:58.902365 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1782) took 2355 ms. Trace:
I20260709 12:27:58.902828 26127 rpcz_store.cc:276] 0709 12:27:56.547245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.855374 (+2308129us) service_pool.cc:224] Handling call
0709 12:27:58.885659 (+ 30285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.901984 (+ 16325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:58.902017 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.902075 (+    58us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:58.902121 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.902183 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.902291 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:58.901621  3133 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.905771  3388 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:58.907334  3133 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:58.902395 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.911646 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:27:58.914458 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 129122us
I20260709 12:27:58.897507  3373 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:58.917455 26127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:27:58.918362 26127 raft_consensus.cc:606] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:58.918658 26127 raft_consensus.cc:614] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:58.922910  3025 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:58.923660  3025 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.924998 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1581) took 2339 ms. Trace:
I20260709 12:27:58.925637  3362 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:58.926218 26783 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:58.909641  3314 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:58.933171 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 146684us
I20260709 12:27:58.925429 26791 rpcz_store.cc:276] 0709 12:27:56.585566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.613312 (+1027746us) service_pool.cc:224] Handling call
0709 12:27:58.924721 (+1311409us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54004ffe54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:58.924914 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492486912}
I20260709 12:27:58.935537 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:58.941584 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.943005 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:58.940688 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:58.951093 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:58.939149  3382 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:58.959318 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:58.956426  3133 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:27:58.973224 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 163126us
I20260709 12:27:58.972184 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.915920  3252 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:58.984825 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:58.936967  3263 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:58.971248  3378 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:27:58.992808 26019 connection.cc:256] RPC call timeout handler was delayed by -1.2231211069997698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3093s for a response
W20260709 12:27:58.993301 26019 connection.cc:256] RPC call timeout handler was delayed by -1.0930080349999116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 12:27:58.975970  3362 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:59.002066 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:59.004762  3492 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.533s	user 0.000s	sys 0.007s
I20260709 12:27:59.005204  3257 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:58.988658  3378 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.010442 26094 leader_election.cc:400] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:59.013034 26094 leader_election.cc:403] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:59.011368 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:58.990413  3355 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.019868  3406 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:59.021102 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1585) took 2426 ms (client timeout 1510 ms). Trace:
W20260709 12:27:59.021561 26790 rpcz_store.cc:269] 0709 12:27:56.594775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.613816 (+ 19041us) service_pool.cc:224] Handling call
0709 12:27:59.003932 (+2390116us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400680454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.021002 (+ 17070us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3913604352}
I20260709 12:27:59.022743  3406 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:59.024853 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.025921 26127 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
W20260709 12:27:59.026166 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.026829 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.027139 26127 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:59.027820 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1785) took 2455 ms. Trace:
I20260709 12:27:59.003232 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:27:59.031400  3497 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:27:59.013815 26051 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:27:59.033041  3382 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.033713  3382 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:59.035431 26120 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:59.035180  3382 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.036156 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4394) took 2333 ms. Trace:
I20260709 12:27:59.036577 26120 rpcz_store.cc:276] 0709 12:27:56.703192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.034808 (+2331616us) service_pool.cc:224] Handling call
0709 12:27:59.035197 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.035876 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.036098 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.038637 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4398) took 2297 ms. Trace:
I20260709 12:27:59.039001 26120 rpcz_store.cc:276] 0709 12:27:56.741544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.037568 (+2296024us) service_pool.cc:224] Handling call
0709 12:27:59.037937 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.038209 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:59.038234 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:59.038313 (+    79us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:59.038355 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.038422 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.038572 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.040716 26120 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: [31.17-31.17]
W20260709 12:27:59.041174  3319 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 1.173s	user 0.013s	sys 0.006s
I20260709 12:27:59.041370 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3384) took 1787 ms. Trace:
I20260709 12:27:59.041801 26120 rpcz_store.cc:276] 0709 12:27:57.254429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.040047 (+1785618us) service_pool.cc:224] Handling call
0709 12:27:59.040458 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.041097 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.041315 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.046756 26121 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:59.047746 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1786) took 1156 ms. Trace:
W20260709 12:27:59.016925 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 89317us
I20260709 12:27:59.054921 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:59.054864  3236 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:59.055378 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:59.055931 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4388) took 3526 ms. Trace:
I20260709 12:27:59.057521  3406 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.057855 26783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2297) took 2351 ms. Trace:
I20260709 12:27:59.058123  3406 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:59.056172 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1592) took 1482 ms. Trace:
I20260709 12:27:59.054208 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.028138 26127 rpcz_store.cc:276] 0709 12:27:56.572940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.026579 (+2453639us) service_pool.cc:224] Handling call
0709 12:27:59.026966 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.027544 (+   578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.027739 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.056588 26126 rpcz_store.cc:276] 0709 12:27:55.529290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.123993 (+1594703us) service_pool.cc:224] Handling call
0709 12:27:58.092547 (+968554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.093740 (+  1193us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:58.093779 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:58.094570 (+   791us) log.cc:844] Serialized 26 byte log entry
0709 12:27:58.166245 (+ 71675us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:58.166275 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.166328 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.898787 (+732459us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:59.005883 (+107096us) raft_consensus.cc:1685] finished
0709 12:27:59.005909 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.055586 (+ 49677us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b5400065ed0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.055707 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2332,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":371,"spinlock_wait_cycles":2330113664,"thread_start_us":71154,"threads_started":1,"wal-append.queue_time_us":346853}
I20260709 12:27:59.058262 26783 rpcz_store.cc:276] 0709 12:27:56.706942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.671684 (+964742us) service_pool.cc:224] Handling call
0709 12:27:58.925827 (+1254143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.926743 (+   916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.057622 (+130879us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54004ffe50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.057771 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2883258112}
W20260709 12:27:59.065524 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78106.5us
I20260709 12:27:59.064527 26121 rpcz_store.cc:276] 0709 12:27:57.891536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.045708 (+1154172us) service_pool.cc:224] Handling call
0709 12:27:59.046210 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.047362 (+  1152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.047665 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.056854 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:59.068876 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:59.069200 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:59.014256  3378 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:59.076018  3439 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:59.077275  3439 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:59.077709  3439 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:59.078773  3439 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.081200  3390 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.080879  3406 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.081805  3133 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:59.082007 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 142418us
W20260709 12:27:59.073381 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 43807.5us
I20260709 12:27:59.076119 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.081419 26167 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:59.085376 26127 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
W20260709 12:27:59.086863 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76678.7us
I20260709 12:27:59.087327 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:59.084636 26054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2939) took 1678 ms (client timeout 1654 ms). Trace:
I20260709 12:27:59.089824 26123 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:27:59.062119 26203 rpcz_store.cc:276] 0709 12:27:57.557704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.108296 (+550592us) service_pool.cc:224] Handling call
0709 12:27:58.108956 (+   660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.109588 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:58.236523 (+126935us) log.cc:844] Serialized 14 byte log entry
0709 12:27:59.039556 (+803033us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800340c60. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:27:59.039836 (+   280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:59.039903 (+    67us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:59.039969 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.040069 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.040206 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":541187,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":13331,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":131,"spinlock_wait_cycles":2796416,"thread_start_us":393717,"threads_started":2,"wal-append.queue_time_us":501738}
I20260709 12:27:59.060715  3236 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:59.095309 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4407) took 1194 ms. Trace:
I20260709 12:27:59.097930 26094 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:27:59.098506  3502 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:27:59.085395  3439 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.094524  3310 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
I20260709 12:27:59.106360  3310 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:27:59.090126 26054 rpcz_store.cc:269] 0709 12:27:57.392754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.396943 (+  4189us) service_pool.cc:224] Handling call
0709 12:27:59.059536 (+1662593us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54000f8a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.071557 (+ 12021us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829739392}
I20260709 12:27:59.097980 26782 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 31. Current term is 32. Ops: []
I20260709 12:27:59.096328 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:27:59.086939  3498 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.194s	user 0.003s	sys 0.000s
I20260709 12:27:59.096424 26123 rpcz_store.cc:276] 0709 12:27:57.897480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.177502 (+280022us) service_pool.cc:224] Handling call
0709 12:27:59.091361 (+913859us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.112713 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.118259  3133 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.119084 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.119616 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2941) took 1715 ms. Trace:
I20260709 12:27:59.119768 26127 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:59.091941  3355 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:59.124505  3355 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.124914  3355 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:59.125061 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:59.092308  3373 raft_consensus.cc:2764] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:27:59.119998 26050 rpcz_store.cc:276] 0709 12:27:57.404224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.123562 (+719338us) service_pool.cc:224] Handling call
0709 12:27:58.219609 (+ 96047us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.272993 (+ 53384us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:58.273037 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:58.273726 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 12:27:58.560999 (+287273us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:58.561059 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.561153 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:59.066172 (+505019us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:27:59.119330 (+ 53158us) raft_consensus.cc:1685] finished
0709 12:27:59.119350 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.119559 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31756,"raft.queue_time_us":8072,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":55274,"spinlock_wait_cycles":4013440,"thread_start_us":286686,"threads_started":1,"wal-append.queue_time_us":612863}
W20260709 12:27:59.137007 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.141948 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.160343 26783 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:27:59.108395  3310 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:59.162181 26049 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:59.144598 26123 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:59.109706  3236 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:59.165434 26048 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:27:59.172818 26122 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:59.158640 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4380) took 3617 ms. Trace:
I20260709 12:27:59.174633 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.175448 26782 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:27:59.171710 26127 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:59.108054  3362 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:27:59.183092 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:59.187350 26054 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 29 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:27:59.144543  3257 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.124074  3356 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.192193 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:27:59.192507 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 93873.1us
I20260709 12:27:59.165187  3310 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:27:59.203926 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1780) took 2660 ms. Trace:
I20260709 12:27:59.207422 26128 rpcz_store.cc:276] 0709 12:27:56.543450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.853484 (+2310034us) service_pool.cc:224] Handling call
0709 12:27:58.871899 (+ 18415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.909019 (+ 37120us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:27:58.936299 (+ 27280us) log.cc:844] Serialized 14 byte log entry
0709 12:27:59.203637 (+267338us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:59.203682 (+    45us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:59.203722 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.203775 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.203873 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5614,"prepare.queue_time_us":20573,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":41,"thread_start_us":258382,"threads_started":2,"wal-append.queue_time_us":198647}
I20260709 12:27:59.209378 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:59.212409 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:59.220644 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.221359 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:59.173921 26124 rpcz_store.cc:276] 0709 12:27:55.451673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.943134 (+1491461us) service_pool.cc:224] Handling call
0709 12:27:58.104384 (+1161250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.104915 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:58.105364 (+   449us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.604530 (+499166us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.644304 (+ 39774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.644377 (+    73us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:58.644445 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.645196 (+   751us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.062252 (+417056us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540010e5d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.068712 (+  6460us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57890,"prepare.queue_time_us":58331,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":39576,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":7975,"spinlock_wait_cycles":2669427968,"thread_start_us":437657,"threads_started":2,"wal-append.queue_time_us":543240}
I20260709 12:27:59.224498 26123 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:59.226784 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.228345 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:59.228926 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:59.229779 26051 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 33: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:27:59.210273  3260 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:59.173826  3437 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:27:59.211958  3500 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.234601 26054 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.236579 26052 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:59.236658 26050 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: [28.15-30.16]
I20260709 12:27:59.237622 26128 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:27:59.240091 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.240806 26124 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:59.241681 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:59.242372 26045 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.244707  3236 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:59.246858 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61723us
I20260709 12:27:59.249922 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.240376  3362 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:59.251763  3362 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:59.250947  3406 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:59.244776 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 163439us
I20260709 12:27:59.196748  3025 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:59.254478 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 145539us
I20260709 12:27:59.255522 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:59.259313 26167 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.259570  3310 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.261021 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.261279  3310 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.231561  3312 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24.
I20260709 12:27:59.264820  3408 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.264932  3312 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.265617  3408 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:59.266748  3408 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.268183 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.248051  3381 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.270064 26129 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:59.270804 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1788) took 1378 ms. Trace:
I20260709 12:27:59.271136  3381 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.271854  3356 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.272986  3356 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.273588  3356 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.274041  3314 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:59.274812  3314 raft_consensus.cc:2764] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
I20260709 12:27:59.276856  3362 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.206804  3257 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.278456 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.271107 26129 rpcz_store.cc:276] 0709 12:27:57.892451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.269411 (+1376960us) service_pool.cc:224] Handling call
0709 12:27:59.269837 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.270535 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.270745 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.276997  3378 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.284977 26124 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:27:59.241253 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3385) took 1978 ms. Trace:
I20260709 12:27:59.285530 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.284439 26169 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:59.289605 26124 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:59.288173 26120 rpcz_store.cc:276] 0709 12:27:57.256061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.042594 (+1786533us) service_pool.cc:224] Handling call
0709 12:27:59.042982 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.043246 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:59.043733 (+   487us) log.cc:844] Serialized 14 byte log entry
0709 12:27:59.234024 (+190291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:59.234093 (+    69us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:59.234134 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.234196 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.234310 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":388,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":83657,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"spinlock_wait_cycles":140800,"thread_start_us":188811,"threads_started":2,"wal-append.queue_time_us":160862}
I20260709 12:27:59.290611 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1791) took 1397 ms. Trace:
I20260709 12:27:59.291484 26124 rpcz_store.cc:276] 0709 12:27:57.893666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.288975 (+1395309us) service_pool.cc:224] Handling call
0709 12:27:59.289370 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.290290 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.290542 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.291749 26120 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:27:59.298014 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:27:59.304425 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47295.9us
I20260709 12:27:59.304432 26128 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:27:59.306387 26051 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:27:59.313067 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1793) took 1150 ms. Trace:
I20260709 12:27:59.313901 26120 rpcz_store.cc:276] 0709 12:27:58.154361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.290810 (+1136449us) service_pool.cc:224] Handling call
0709 12:27:59.291206 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.298446 (+  7240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.304517 (+  6071us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:59.318111 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 69730.6us
W20260709 12:27:59.321702 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.325426 26045 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
W20260709 12:27:59.329136 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 204156us
I20260709 12:27:59.335793 26128 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:59.257414  3500 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.265635  3312 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.314254  3356 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.334200 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:59.325913  3381 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:27:59.340787 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91064.1us
I20260709 12:27:59.337601 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:59.345587  3257 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.345963  3378 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:27:59.347079  3518 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:27:59.353175  3510 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:27:59.371506  3025 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:27:59.372705 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74782.1us
I20260709 12:27:59.374429 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:27:59.354698 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 93799.3us
I20260709 12:27:59.376603 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.379099  3260 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:27:59.330390  3501 raft_consensus.cc:2764] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:59.382076  3252 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:59.339375  3514 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.382598  3252 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:59.383008  3514 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:27:59.368705 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 134696us
I20260709 12:27:59.384038  3514 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.384886 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.385563 26128 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:59.369067 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:27:59.369676 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50268us
I20260709 12:27:59.354148  3500 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.370436  3381 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:59.374547  3312 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.379709  3260 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:59.369781  3355 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.365324 26054 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:27:59.390623 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1593) took 1830 ms. Trace:
I20260709 12:27:59.389842  2830 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:59.400979 26094 leader_election.cc:400] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33.
I20260709 12:27:59.421800 26094 leader_election.cc:403] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:27:59.399850  3518 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:59.401959 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:59.402833  3381 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:59.404764 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:27:59.410403 26198 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:27:59.411094 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:59.412062 26018 leader_election.cc:400] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31.
I20260709 12:27:59.450488 26018 leader_election.cc:403] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:59.392453 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 53913.7us
I20260709 12:27:59.412545  3513 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:59.414038 26123 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
W20260709 12:27:59.416033 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47079.5us
I20260709 12:27:59.454536 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.455261  3381 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:27:59.419253  3511 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:27:59.383566  3252 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.355804 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:59.421026  3133 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.472188 26197 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:27:59.419023 26202 rpcz_store.cc:276] 0709 12:27:57.557997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.086301 (+528304us) service_pool.cc:224] Handling call
0709 12:27:58.086841 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.112791 (+ 25950us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:58.555534 (+442743us) log.cc:844] Serialized 14 byte log entry
0709 12:27:59.084765 (+529231us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.111179 (+ 26414us) log.cc:844] Serialized 26 byte log entry
0709 12:27:59.111328 (+   149us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:59.111372 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.111446 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:59.387900 (+276454us) raft_consensus.cc:1685] finished
0709 12:27:59.387922 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.388072 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48438,"prepare.queue_time_us":596544,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":59945,"raft.run_cpu_time_us":531,"raft.run_wall_time_us":61708,"thread_start_us":806574,"threads_started":2,"wal-append.queue_time_us":537747}
I20260709 12:27:59.474845 26123 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
W20260709 12:27:59.472026 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71198.2us
I20260709 12:27:59.473217 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:59.458432  3513 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:59.463689  3312 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.509697  3312 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.510917  3373 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:27:59.511957  3312 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.520421  2830 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:59.525861 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76381.3us
I20260709 12:27:59.527091 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:59.468263  3439 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:59.533257 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:59.536509  3378 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:59.537181  3378 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:59.538313  3378 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.540436  3314 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:59.542593  3520 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:59.543702  3312 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:27:59.544111  3312 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:59.545408  3312 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.547147  3312 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:59.547325  3501 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:59.496960  3310 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.548861  3310 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.551339  3310 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.553544  3355 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.554329  3355 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:59.554979  3355 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:59.555869  3133 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:59.558219  3133 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:27:59.496218 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 94386.3us
I20260709 12:27:59.495589  3408 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33.
I20260709 12:27:59.500685  3406 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.501324 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:27:59.502426 26123 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:59.508342  3525 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:27:59.508545 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.547997  3501 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.567184 26198 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:59.566234 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:59.567402 26128 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:27:59.571604 26126 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
W20260709 12:27:59.572520 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.560961  3513 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.576507 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.577327 26790 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:27:59.563064  3382 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:59.565654  3388 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:59.568624  3501 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:59.567139  3310 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:27:59.581143  3390 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.588963  3390 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:59.588963 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 177122us
I20260709 12:27:59.575778  3263 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:59.595016 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 59714.2us
I20260709 12:27:59.594563 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:27:59.584039  3439 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:59.585706  3525 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:59.599859 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:59.601449 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93014.6us
I20260709 12:27:59.574209  3260 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:59.610193 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.630163  3534 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:59.633813  3379 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:27:59.645772 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60461us
I20260709 12:27:59.646514  3310 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.591754  3390 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.656371 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:27:59.602221  3525 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:27:59.658538 26788 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: [22.12-24.13]
I20260709 12:27:59.659402  3379 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.665287  3260 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.587281  3382 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.597916  3263 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.666559  3319 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:27:59.641083  3408 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:59.597551  3518 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 16, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.592437  3388 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:59.669148 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 142205us
I20260709 12:27:59.669900  3500 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.669973 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:59.673887 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.674031  3406 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:59.680583  3525 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:59.691557 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.695024  3535 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:59.696039  3535 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:27:59.686754  3408 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.696573  3535 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:59.696919 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.697188  3379 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.697600  3535 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.17, Last appended by leader: 17, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.698827 26168 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.699131  3310 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.698915  3236 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:59.700706  3263 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.696408  3260 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.701555  3382 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.701699  3260 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.704650  3524 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:59.705534  3524 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:59.706197  3524 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:27:59.707804  3536 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:59.709180  3524 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:59.709570  3524 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:27:59.710024  3382 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.710520  3524 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.710652  3382 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.691794 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:27:59.713953 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 57702.5us
I20260709 12:27:59.713290 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:27:59.703991  3260 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.717165 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:59.696817  3408 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:59.723050 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.724113  3524 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:59.700115  3532 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:59.727283 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1790) took 1834 ms. Trace:
I20260709 12:27:59.688721  3520 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:59.732434  3252 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.729988 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:27:59.727705 26129 rpcz_store.cc:276] 0709 12:27:57.893288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.284670 (+1391382us) service_pool.cc:224] Handling call
0709 12:27:59.308565 (+ 23895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.385656 (+ 77091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:59.385687 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.386309 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:27:59.493640 (+107331us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:59.493705 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.493797 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:59.727021 (+233224us) raft_consensus.cc:1685] finished
0709 12:27:59.727043 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.727217 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21425,"raft.queue_time_us":52907,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":11144,"thread_start_us":106855,"threads_started":1,"wal-append.queue_time_us":197220}
I20260709 12:27:59.698928  3535 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:59.762248 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.720053  2830 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31.
I20260709 12:27:59.713299  3382 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.765017 26020 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:27:59.768064 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54850.1us
I20260709 12:27:59.768745 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:59.772894 26787 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.688558  3319 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.738095  3388 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:27:59.782888 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 403057us
I20260709 12:27:59.784655 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.785578  3373 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:59.829137  3373 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:27:59.830165  3373 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.831391  3373 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:59.785653 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.833242 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.834460 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:59.786688 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.841352 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:59.842260 26784 raft_consensus.cc:2410] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:27:59.843112 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.787560 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.845253 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.845860 26785 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:27:59.846710 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.847541 26785 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:27:59.788012  3533 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:59.848029  3520 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.848578  3533 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.848963  3520 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.739614  3536 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:27:59.841372  3513 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.788728 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.854406 26783 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:27:59.855366 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.855937 26783 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:27:59.856983 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.849017  3533 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.858162 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.859135 26783 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:59.694038  3500 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.798553  3355 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:59.860826  3355 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:59.862051  3500 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.862222  3355 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:27:59.799973 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.865083 26788 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:27:59.865197  3408 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:27:59.866179 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.800274 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4306) took 1172 ms. Trace:
I20260709 12:27:59.866987 26049 rpcz_store.cc:276] 0709 12:27:58.627762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.743195 (+115433us) service_pool.cc:224] Handling call
0709 12:27:59.800093 (+1056898us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54000ac3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.800193 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23610752}
I20260709 12:27:59.867519  3236 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.804628 26790 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:27:59.868027  3236 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.813850 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.769204  3534 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.869378  3534 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:27:59.870486  3513 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.761090  3362 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:59.823725  3260 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:59.825112 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.826861 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:27:59.828398 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:59.821086 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1781) took 3276 ms. Trace:
I20260709 12:27:59.870343  3534 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.872303  3236 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:27:59.822976  3319 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.876703 26169 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:27:59.876627 26167 leader_election.cc:400] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:27:59.918514 26167 leader_election.cc:403] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:59.878314  3520 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:27:59.880415 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:59.878425 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.844139 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:27:59.932474 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:59.906962 26783 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:27:59.910259 26122 rpcz_store.cc:276] 0709 12:27:56.545159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.851512 (+2306353us) service_pool.cc:224] Handling call
0709 12:27:59.091048 (+239536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.173552 (+ 82504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.735754 (+562202us) spinlock_profiling.cc:243] Waited 81.6 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:27:59.820832 (+ 85078us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540013ef50. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:27:59.820983 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737549952}
I20260709 12:27:59.916105 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.919031 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:27:59.939378 26785 raft_consensus.cc:2368] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29.
I20260709 12:27:59.942839 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.889509 26787 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:27:59.947448 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.948613 26126 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 31 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:27:59.952494 26129 raft_consensus.cc:2410] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 31 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:27:59.953188 26789 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:27:59.955116 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:27:59.890497  3362 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.955878 26782 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:27:59.957264 26784 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:27:59.960850 26018 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.857s (SENT)
I20260709 12:27:59.878425  3501 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:27:59.962952 26756 leader_election.cc:400] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30.
I20260709 12:27:59.968734 26756 leader_election.cc:403] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Cancelling election due to peer responding with higher term
I20260709 12:27:59.951823  3513 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:59.945506  3500 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:59.962777  3236 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
W20260709 12:27:59.973658 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 240294us
W20260709 12:27:59.976648 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 149874us
I20260709 12:27:59.967621 26053 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:27:59.959350  3362 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:59.989665 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:27:59.988798 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:27:59.991958 26054 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:59.995775 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:27:59.996167 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.996743 26094 leader_election.cc:336] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.598s (SENT)
I20260709 12:27:59.999257 26126 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:27:59.986833  3319 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30.
I20260709 12:28:00.008038 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:28:00.011993 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.015789 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:00.016400 26127 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:59.993502  3513 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:00.018334 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.019925 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:27:59.996255  3500 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:00.024426 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:00.031562 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 155112us
I20260709 12:28:00.033114 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.034001 26054 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:00.043200 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.042909 25946 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 26, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 26 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:00.052455 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.058713  3533 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.018860  3535 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 12:28:00.062840 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 112688us
I20260709 12:28:00.042229  3319 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.021s
I20260709 12:28:00.041540 26053 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:00.038820 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.068394 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.088685 26127 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:00.089322 26054 raft_consensus.cc:2393] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 32: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:00.098176  3500 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.098265 25946 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 21 to 22, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 22 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:00.103576 25946 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 12 to 26. New cstate: current_term: 26 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:00.105528 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68926.6us
I20260709 12:28:00.105808 25946 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:00.108629  3319 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:00.109054  3319 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:00.108268 25946 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:00.109511  3501 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:00.109892  3319 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.111208  3319 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:00.095176 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.120169 26047 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:00.122377  3548 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:00.123991  3149 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:00.124418 26052 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:28:00.124822 26053 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:00.124577  3149 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:00.125478 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.126183 26202 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:28:00.126241  3149 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.127660 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.128315 26203 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:28:00.128430 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 136709us
W20260709 12:28:00.129133 26094 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.688s (SENT)
W20260709 12:28:00.130523 26094 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.701s (SENT)
W20260709 12:28:00.131335 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 113755us
W20260709 12:28:00.131721 26094 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.754s (SENT)
I20260709 12:28:00.133445 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.133684 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.134578 26201 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:28:00.134716 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.133379 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.135538 26196 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.135694 26194 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:00.136143 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.136835 26199 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:00.137710 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.138331 26199 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:00.138747 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.090637  3388 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:00.139803 26196 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 29: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.139842 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.136010 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.144657 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.145469 26200 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.145295 26195 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 24.
I20260709 12:28:00.146757  3521 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.147301  3521 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.111088 25946 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:00.132144 26094 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.161412 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.172013 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.172845 26053 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:00.164367  3312 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:00.163559  3525 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.175781 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.176484 26196 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.177656  3525 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.178570 26123 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:00.176512 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.179487 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.087591  3439 consensus_queue.cc:1059] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 12:28:00.183633 26201 raft_consensus.cc:2410] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:28:00.186285 26194 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:00.188027 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.188651 26202 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:28:00.189488 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.186579 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.190433 26195 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:00.191151 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.191836 26194 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.191607 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.192785 26198 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 31 because replica has last-logged OpId of term: 30 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:28:00.195327 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.195327 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.195814  3362 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.196420 26199 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.196416 26052 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:00.203506 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:00.205410 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:00.205087 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 129456us
W20260709 12:28:00.208256 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76541.4us
I20260709 12:28:00.203894 26200 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
W20260709 12:28:00.213547 26019 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.510s (SENT)
I20260709 12:28:00.214385 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.224711 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:00.224995 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:00.233863 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.235879 26203 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.236068  3252 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:00.207657 26018 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.946s (SENT) [suppressed 51 similar messages]
I20260709 12:28:00.239323 26196 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:28:00.240926 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.244068 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:00.244997 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.245505 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.244004 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.247200 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:00.247503 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.248338 26128 raft_consensus.cc:1240] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:00.249821 26195 raft_consensus.cc:2393] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:28:00.253635 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46287us
I20260709 12:28:00.254063 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:00.256213 26046 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:00.257205 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.259346 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.259366 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.260461 26127 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:28:00.260031 26053 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:28:00.263690 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.242195  3537 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:28:00.269280 26094 leader_election.cc:336] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.873s (SENT)
I20260709 12:28:00.269986 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.272672 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.272994 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.273379 26195 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:00.273919 26199 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:28:00.273905 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.274959 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.276021 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.180279  3513 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.205359  3382 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:28:00.279320 26121 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
I20260709 12:28:00.281741 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.287431 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.293223 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.294415 26203 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.283600  3520 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:00.298959 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 43560.9us
I20260709 12:28:00.300371  3525 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.300560 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.301365 26052 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.300927  3525 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.302332 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.303020 26201 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.303505  3525 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.301221 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.305310 26198 raft_consensus.cc:2410] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 31 because replica has last-logged OpId of term: 30 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:28:00.309368 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.310570 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.311218 26198 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:28:00.312598 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.313330  3382 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:00.313881  3382 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.314013 26782 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:00.314299  3382 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.313468 26197 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33.
I20260709 12:28:00.317893 26195 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:00.318876 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.319582 26197 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:28:00.312108 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.321185 26198 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:00.322170 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.323335 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.323871  3378 raft_consensus.cc:3055] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:00.324102 26198 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.324469  3378 raft_consensus.cc:740] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:00.325148 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.325991 26197 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:00.314059 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.325471  3378 consensus_queue.cc:260] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.326754 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.327802  3378 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.329947 26201 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 26 because replica has last-logged OpId of term: 25 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 20.
I20260709 12:28:00.312744 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.318877 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.269166  3537 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.345016 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:00.320411  3563 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.003s
I20260709 12:28:00.345818 26194 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
W20260709 12:28:00.346680 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 90859.3us
I20260709 12:28:00.346800 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:00.347739 26202 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:00.358341 26199 raft_consensus.cc:2376] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30. Re-sending same reply.
I20260709 12:28:00.359982 26195 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:28:00.283558  3501 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:00.364080 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 119165us
I20260709 12:28:00.373045 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:00.347517  3532 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:28:00.374749 26201 raft_consensus.cc:2410] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:28:00.347783  3537 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.376909 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.377632 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:00.384811 26784 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:00.359171  3525 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.382381 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.345129  3521 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:00.382758 26019 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.130s (SENT)
W20260709 12:28:00.376374 26094 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.024s (SENT)
I20260709 12:28:00.388661 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.389521 26202 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:00.285993  3534 consensus_queue.cc:1059] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:28:00.377867 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.387842 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.388994 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.399274 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.361328  3373 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:28:00.408155 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.408463 26124 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:00.412668 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.413451 26047 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:00.413606 26195 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:00.413991 26199 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:00.415797  3362 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.415912 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.416996  3383 raft_consensus.cc:3055] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:00.417814  3383 raft_consensus.cc:740] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:00.422561 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.430037 26127 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 33 index: 16. (index mismatch)
I20260709 12:28:00.431137 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.372629  3382 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.434863  3521 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:00.435840 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4423) took 1047 ms. Trace:
I20260709 12:28:00.436292 26123 rpcz_store.cc:276] 0709 12:27:59.388881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.582070 (+193189us) service_pool.cc:224] Handling call
0709 12:27:59.582375 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.178965 (+596590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.435549 (+256584us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400065ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:00.435768 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370632064}
I20260709 12:28:00.437592 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:00.438515  3535 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:00.438647 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:00.439004  3535 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:00.439517  3535 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.440436  3501 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:00.440897  3501 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:00.441303  3501 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.394563  3525 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.444494  3355 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:00.446184 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.446679  3355 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.446928 26049 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 21.
I20260709 12:28:00.447153  3355 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.447414  3525 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.399057  3533 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:00.448995 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76815.6us
I20260709 12:28:00.449458 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:00.450017 26094 leader_election.cc:336] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.115s (SENT)
I20260709 12:28:00.450237 26121 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:00.450618 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.452899  3382 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.453101  3513 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:00.453945  3382 consensus_queue.cc:1059] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:00.419947 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.455063  3382 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:00.455368 26197 raft_consensus.cc:2393] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.457106 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.436419  3500 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:00.458113 26200 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:00.458977 26202 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:28:00.461184 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:00.461436 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.461655  3356 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:28:00.463544 26094 leader_election.cc:336] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.140s (SENT)
I20260709 12:28:00.464162 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.465128  3373 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:00.466782 26019 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.235s (SENT)
I20260709 12:28:00.473354 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.439826 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:00.476542 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 98742.9us
I20260709 12:28:00.480072 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.481887 26203 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30. Re-sending same reply.
I20260709 12:28:00.486377  3513 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:00.487267 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:00.437595  3562 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.201s	user 0.002s	sys 0.000s
I20260709 12:28:00.512828 26203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4"
dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:54828
I20260709 12:28:00.513053 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:00.513660 26203 raft_consensus.cc:606] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Received request to transfer leadership
I20260709 12:28:00.513799 26129 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.514151 26203 raft_consensus.cc:614] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:00.515228 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.494705 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54972.5us
I20260709 12:28:00.508947 26194 raft_consensus.cc:1240] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:00.487931 26201 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.485885 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:00.519822 26784 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:00.520192 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.478857  3500 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.522359 26093 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:00.521921  3362 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.527132 26197 raft_consensus.cc:1275] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 33 index: 16. (index mismatch)
I20260709 12:28:00.527143 26126 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:28:00.488277  3536 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:28:00.531776 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.532181 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:00.542290 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:00.550998 26202 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:00.522390  3236 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.531093  3564 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:28:00.576583 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 203655us
I20260709 12:28:00.531517  3568 raft_consensus.cc:3055] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:00.608114 26201 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:00.583071  3534 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:00.618832 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.594851  3568 raft_consensus.cc:740] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:00.598727  3564 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:00.584125  3236 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.632855 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44004 (request call id 1) took 1770 ms. Trace:
I20260709 12:28:00.593178  3378 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:28:00.615864  3566 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:00.637980  3383 consensus_queue.cc:260] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.670127 26045 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:28:00.623502  3513 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.640861  3568 consensus_queue.cc:260] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.641696  3314 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:00.642267  3537 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.648319 26120 rpcz_store.cc:276] 0709 12:27:58.867557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.315557 (+448000us) service_pool.cc:224] Handling call
0709 12:28:00.632347 (+1316790us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54002ca6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743ab735 00007f5b743a39fa 00007f5b7936b4c9 00007f5b728e7c54 00007f5b728e7be2 00007f5b728e7b11 00007f5b728e7a8c 00007f5b728e6d42 00007f5b726e048c 00007f5b726df8c6
0709 12:28:00.632683 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984228224}
W20260709 12:28:00.634073 26094 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.273s (SENT)
I20260709 12:28:00.663103  3382 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:00.622165  3252 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:00.673674 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.640686  3501 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.676740 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.677539 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.679141 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.705346 26046 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:00.687696 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.688247 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.655447 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.693670 26047 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 32 index: 17. (term mismatch)
I20260709 12:28:00.707049 26047 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 16
I20260709 12:28:00.707376 26047 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 31.17
I20260709 12:28:00.707168 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:00.707746 26047 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:00.662787  3566 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:00.697954 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 212147us
W20260709 12:28:00.702338 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 97065us
I20260709 12:28:00.709164  3566 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.709707 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.710636  3514 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:00.712226 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:00.713340  3514 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:00.668989  3408 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:00.715972 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.716190 26049 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
W20260709 12:28:00.719964 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.714471 25946 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 23, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 23 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:00.729769 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 219333us
I20260709 12:28:00.731119 26049 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:00.681667  3314 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:00.735152 26046 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
W20260709 12:28:00.735615 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.695463  3382 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:00.739323 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 256257us
I20260709 12:28:00.744072 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.722111  3568 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.742859 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.748402 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.740582  3383 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.744138  3382 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.757166 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.757498 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.758388 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.738286  3356 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:00.765084 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.768090 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.768366 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:00.769552 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 135533us
I20260709 12:28:00.749811  3437 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:00.772176  3500 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:00.778268  3533 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:00.778291  3378 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:00.778609  3314 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.779075 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.779798  3314 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:00.748649  3514 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:28:00.759953  3570 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.782076  3533 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:00.781885  3570 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.781893 26019 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:00.765939  3567 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.234s	user 0.002s	sys 0.000s
I20260709 12:28:00.784901  3236 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:00.785311 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.786180  3362 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:00.787019  3500 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:00.787194  3533 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:00.787602  3533 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.761911  3578 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.787994  3533 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:00.788949  3578 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:00.789426  3578 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:00.789875  3578 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.790370  3260 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:00.791522  3260 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:28:00.776186 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 64047.9us
I20260709 12:28:00.777513  3437 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:00.792416  3437 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.792482 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.794060  3388 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:00.794545  3388 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.795002  3388 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:00.795723  3575 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:28:00.791857  3260 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:00.787743  3500 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:00.789996 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.799903 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.800935  3525 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:00.801327  3525 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:00.801826  3525 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.803332  3356 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:00.803905  3356 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.804318  3356 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:00.805806 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.812911  3565 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:00.813556  3574 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:28:00.817912  3537 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.769541  3579 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:28:00.819957  3579 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:00.820456  3579 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:00.821504  3579 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.776106 26020 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:00.787032 26019 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.833076 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:00.833102 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.835166 26167 leader_election.cc:400] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29.
W20260709 12:28:00.836205 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93441us
I20260709 12:28:00.837431 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.838084  3312 raft_consensus.cc:2764] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:28:00.839941  3378 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:00.840348  3362 consensus_queue.cc:1175] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a981c9a7972c4e99a0903924c64fbb5c" responder_term: 32 status { last_received { term: 32 index: 18 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:00.841702  3520 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.843384 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.843756  3579 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:00.844321  3520 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:00.747982  3564 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.852200  3520 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:00.858507 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:00.864552  3520 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:00.865244  3501 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:00.866580 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 98546.3us
W20260709 12:28:00.866791 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 90803.9us
I20260709 12:28:00.868058 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.869503 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:00.835045  3581 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:28:00.870110 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.881013 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:00.900185 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:00.900727 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:00.900900 26789 raft_consensus.cc:2376] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30. Re-sending same reply.
I20260709 12:28:00.901302 26787 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:00.872700  3548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.011s	sys 0.008s
W20260709 12:28:00.904886 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 117932us
I20260709 12:28:00.866169  3520 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:00.918717 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.919482  3520 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.876926  3570 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.922567 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.922778  3382 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.879321  3501 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:00.902865 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:00.910928 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:00.911142 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:00.915337 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.870774  3236 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:00.917097  3534 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:00.898975  3025 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.917735 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.918298 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.927208 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:00.872936  3579 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:28:00.927232 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 94765.4us
I20260709 12:28:00.928697  3570 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.920190  3533 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:00.934248  3584 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:00.928414 26786 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:00.929237 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:00.890102  3378 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:00.930523 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:00.930703 26050 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.931320 26787 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:00.934738  3546 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 12:28:00.934947 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:00.928136 26046 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:00.949000 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:00.934978  3570 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.935825  3382 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.938236  3533 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:00.933537  3580 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
W20260709 12:28:00.910244  3548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.013s	sys 0.011s
I20260709 12:28:00.952266 26053 raft_consensus.cc:2410] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:28:00.868643  3355 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.951499  3579 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:00.931951  3534 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.873888  3535 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.904520  3537 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
W20260709 12:28:00.955150 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54701.8us
I20260709 12:28:00.949373  3586 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:00.951370  3584 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:00.976193 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49904.2us
I20260709 12:28:00.976692 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:00.985224  3584 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.988394  3586 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:00.988955  3589 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:28:00.989494  3586 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.979645  3534 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.990268  3546 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:00.991142  3546 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:00.991498  3546 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:00.994190  3439 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.994846  3439 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.996574 26052 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:00.998265  3535 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:00.998512  3546 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.999836  3546 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:01.004356  3591 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:00.946257  3590 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:28:00.961802  3378 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:01.024921 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:01.015949 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:00.962497  3501 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:01.021059 26787 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:28:01.012729 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:01.027477 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:00.978415  3578 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:00.958848  3388 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.030776  3501 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:01.055636 25939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:01.006071  3579 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.023729  3355 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.067154 26782 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:00.983459  3383 consensus_queue.cc:1048] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 12:28:00.981058  3437 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.065508  3501 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:01.072196 26786 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:01.072952  3566 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.073513  3437 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.046051  3537 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:28:01.101717 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76864.1us
I20260709 12:28:01.110870 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:01.115754  3564 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:01.117520  3564 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.118605  3564 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:01.061189  3513 consensus_queue.cc:1059] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:28:01.009244 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:01.149199 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 172571us
I20260709 12:28:01.150259 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:01.155030 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 170545us
I20260709 12:28:01.155810 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:01.078166  3525 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.121807  3592 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.140419  3543 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:28:01.167526 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 138638us
I20260709 12:28:01.154446  3537 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.154805  3373 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.157271  3566 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.170598  3319 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:01.205507 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:01.215958 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:01.170763  3355 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.220957 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 50485.2us
I20260709 12:28:01.177644  3025 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:01.224576 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:01.226274 26756 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:01.219745  3437 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:01.185096  3439 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.203033  3570 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.181619  3373 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.166427  3592 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.221596  3566 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:01.240545 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 90410.8us
I20260709 12:28:01.181972  3537 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.231029  3382 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:01.230362  3548 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:01.230091  3543 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:01.235072  3236 raft_consensus.cc:2764] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:28:01.220885  3356 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.237432  3501 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:01.295099 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46916.9us
I20260709 12:28:01.243481  3319 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:01.308516 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:01.319249 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:01.320207 26784 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:01.309370 26050 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:01.308185 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:01.311396 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:01.323475 26049 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:01.248093  3578 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:01.330513 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 90815.7us
I20260709 12:28:01.333889 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:01.334834 26129 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:01.333148 26049 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:01.335372 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:01.249481  3382 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:01.338245 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:01.338506 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:01.340135 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:01.252204  3388 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.343348 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:01.305299  3373 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.232185  3355 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.355813 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:01.356611 26051 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:01.356773 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:01.357398 26129 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:01.305097  3597 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:28:01.333705 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:01.369462 26120 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:01.300098  3598 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:01.246393  3025 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.082s
I20260709 12:28:01.352029 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:01.364872 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:01.365671 26125 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:01.367424 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:01.301450  3605 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.392649  3605 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:01.332751  3565 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.001s
I20260709 12:28:01.372144 26123 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:28:01.395124 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:01.395946 26123 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:28:01.396898 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:01.397162 26123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:28:01.349499 26756 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:01.369488 26050 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:01.349740 26046 raft_consensus.cc:2376] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32. Re-sending same reply.
I20260709 12:28:01.301571  3601 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:01.380224 26128 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:01.350759  3362 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.382584 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:01.410578 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61152.5us
I20260709 12:28:01.387353 26126 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:01.337481  3319 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:01.389432  3524 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:01.411131 26047 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:01.337512  3500 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:01.370029  3252 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:01.449771 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:01.450676 26120 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:01.452651 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:01.420172 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:01.410455 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:01.350118  3548 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:01.460654  3604 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:01.461108  3604 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.461251 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:01.461499  3604 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:01.462491 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:01.463024 26785 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:01.346778  3382 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:01.456619  3500 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.457155 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:01.456980  3319 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.470614  3500 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.426101  3524 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:01.457633 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:01.457858 26129 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:28:01.473786 26790 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:28:01.438153  3381 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:01.437187  3388 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:01.459022 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:01.477010 26125 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:01.459229 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48864.2us
I20260709 12:28:01.458536 26784 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:28:01.473361  3524 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.482558 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:28:01.497957 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.034s	sys 0.059s
W20260709 12:28:01.498507 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.034s	sys 0.059s
W20260709 12:28:01.498965 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 549993us
I20260709 12:28:01.476254  3388 raft_consensus.cc:697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:01.516096 26757 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:01.375548  3566 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.109s
I20260709 12:28:01.412624  3601 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:01.514637 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:01.530501 26046 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:01.427707  3565 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:01.513983 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:01.531337 26128 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:01.355881  3534 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.541877 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:01.513124  3373 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:01.500851  3525 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.437753  3362 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.586339  3612 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.587201  3605 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:01.587720  3614 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.588073  3609 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.445408  3533 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 12:28:01.580804 26782 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:01.569451 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:01.536141  3381 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:01.598439 26045 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:01.576167  3578 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:01.537227  3601 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:01.684378 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:01.689002 26127 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:01.584077  3373 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.403250  3598 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.527971  3500 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:28:01.580063  3524 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:01.684561 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140898us
W20260709 12:28:01.701972  3546 consensus_meta.cc:287] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.000s
I20260709 12:28:01.702174  3362 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.702646  3356 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.703409  3388 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:01.705752  3613 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:28:01.706319  3595 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.709045 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:01.710418 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:01.808943 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 98640.7us
I20260709 12:28:01.536469  3610 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.174s
W20260709 12:28:01.715209  3616 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:28:01.568226  3543 consensus_queue.cc:237] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.725322  3606 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.730772  3601 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.744761  3501 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.776988  3578 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:01.753899  3534 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.760861  3592 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:01.691984  3609 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.774804 26046 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:01.776821 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:01.776832  3373 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:01.776415  3566 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.682771  3612 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:01.801529 26166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.006s	sys 0.005s
I20260709 12:28:01.691742  3614 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:01.806421 26125 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:01.765755  3548 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:01.805934 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 472267us
W20260709 12:28:01.818970 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.009s	sys 0.003s
I20260709 12:28:01.747064  3355 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:01.821233 26123 raft_consensus.cc:606] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:01.800927 26785 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:28:02.038759 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.010s	sys 0.004s
I20260709 12:28:02.023007 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:02.039028 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 262346us
I20260709 12:28:01.993526 26120 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:02.040731 26123 raft_consensus.cc:614] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:01.877167 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.960947 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:02.048848 26166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.007s	sys 0.006s
I20260709 12:28:01.824414 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:02.027033 25946 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:01.938261  3616 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 12:28:02.095005 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.012081  3604 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.003s
I20260709 12:28:01.818127  3546 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.821753  3437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.005s	sys 0.008s
I20260709 12:28:01.817579  3605 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:28:01.954957  3596 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.767s	user 0.003s	sys 0.000s
I20260709 12:28:01.957989  3598 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:02.092612 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52382.4us
I20260709 12:28:01.817663  3362 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:02.008287  3382 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.001s
W20260709 12:28:02.119268 26051 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.004s	sys 0.017s
W20260709 12:28:01.999585  3619 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:28:02.006208  3564 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.847s	user 0.014s	sys 0.000s
W20260709 12:28:02.050536  3319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.054s	sys 0.106s
W20260709 12:28:02.000583  3611 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:28:01.988090  3520 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.008078  3601 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.080549  3548 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:02.113725 26052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.034s
I20260709 12:28:01.948632  3617 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.885516  3584 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.900s	user 0.014s	sys 0.000s
I20260709 12:28:02.094287  3568 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:01.815974  3595 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:02.119570 26050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.011s	sys 0.010s
I20260709 12:28:02.122732 26126 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:28:02.123468 26785 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 20. (term mismatch)
W20260709 12:28:02.127095 26051 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.004s	sys 0.018s
I20260709 12:28:02.006721  3578 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:02.134413 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.026s	sys 0.090s
W20260709 12:28:02.158595 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.026s	sys 0.090s
W20260709 12:28:02.159409 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 739292us
I20260709 12:28:02.114382  3605 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.094971  3566 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:02.141377 26052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.012s	sys 0.034s
W20260709 12:28:02.143687 26128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.077s	sys 0.173s
W20260709 12:28:02.133610  3319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.054s	sys 0.106s
I20260709 12:28:02.112998  3604 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:01.822973  3606 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:02.110225  3437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.006s	sys 0.009s
W20260709 12:28:02.154726 26050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.012s	sys 0.010s
I20260709 12:28:02.155223 26785 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 19
I20260709 12:28:02.177142 26785 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 29.20
I20260709 12:28:02.177533 26785 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:02.113929  3546 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.126246  3362 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.235206  3362 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:02.168602 26128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.077s	sys 0.173s
I20260709 12:28:02.143236  3548 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:02.170295 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:28:02.154803  3602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.001s	sys 0.010s
I20260709 12:28:02.186154  3604 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:02.186342  3566 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.156370  3617 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.164422  3534 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:02.147600  3633 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.155292  3584 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.133178  3382 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.188792  3260 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:02.192973  3626 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:28:02.197264  3614 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.201216  3373 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.206218 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2335) took 1036 ms. Trace:
I20260709 12:28:02.142503  3520 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.361929  3260 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:02.169016  3533 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.381461  3533 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.383755 26787 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:02.213600  3595 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.347438  3534 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.216037  3612 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:02.386332  3612 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.387454  3534 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.387110  3612 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.219046 26122 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:02.221048 26786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2331) took 1214 ms. Trace:
I20260709 12:28:02.388944 26791 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:02.389070 26786 rpcz_store.cc:276] 0709 12:28:01.007064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.008201 (+  1137us) service_pool.cc:224] Handling call
0709 12:28:01.020519 (+ 12318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.072617 (+ 52098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.220498 (+1147881us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b540069e6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:02.220953 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118388608}
I20260709 12:28:02.389882  3594 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.225191  3609 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:02.390657  3594 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.169421  3605 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:02.169844  3568 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:02.392444  3603 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.098s
I20260709 12:28:02.393489  3568 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 25.21, Last appended by leader: 21, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.231846  3149 raft_consensus.cc:3055] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:02.396889  3149 raft_consensus.cc:740] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:02.398186  3149 consensus_queue.cc:260] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.296823 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4341) took 1234 ms. Trace:
I20260709 12:28:02.176276  3629 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.409898 26127 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:02.308337 26123 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
W20260709 12:28:02.318507 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 199320us
W20260709 12:28:02.298097  3602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.002s	sys 0.011s
W20260709 12:28:02.323163 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 152980us
I20260709 12:28:02.437790 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:02.326609 26202 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
W20260709 12:28:02.328537 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 217675us
I20260709 12:28:02.347738 26196 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:02.350230 26203 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:28:02.351289 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.240794  3610 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:02.352124 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.354530 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.299645  3524 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.360047 26782 rpcz_store.cc:276] 0709 12:28:01.169360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.169525 (+   165us) service_pool.cc:224] Handling call
0709 12:28:01.175446 (+  5921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.687865 (+512419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.205818 (+517953us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54001225d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:02.206131 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931508480}
I20260709 12:28:02.136140  3564 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.363999  3373 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.371927  3548 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.380319  3520 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.210243 26125 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:02.490097 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4446) took 1104 ms. Trace:
I20260709 12:28:02.490751 26125 rpcz_store.cc:276] 0709 12:28:01.386119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.209554 (+823435us) service_pool.cc:224] Handling call
0709 12:28:02.209984 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.489745 (+279761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.490012 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.349881  3604 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:02.469117 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:02.469617 25939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:02.359012  2830 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:02.474952 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:02.418108  3537 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.499363 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:02.418639 26048 rpcz_store.cc:276] 0709 12:28:01.012550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.012722 (+   172us) service_pool.cc:224] Handling call
0709 12:28:01.022338 (+  9616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.022980 (+   642us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:01.023377 (+   397us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.064727 (+ 41350us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.077067 (+ 12340us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.101731 (+ 24664us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:01.112772 (+ 11041us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.113012 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.113084 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.212830 (+1099746us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b180045df00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:02.220466 (+  7636us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b180045df00. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1
0709 12:28:02.246197 (+ 25731us) raft_consensus.cc:1685] finished
0709 12:28:02.246232 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.246452 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":394,"prepare.queue_time_us":14863,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":903,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":10510,"spinlock_wait_cycles":18938112,"thread_start_us":40702,"threads_started":1,"wal-append.queue_time_us":96473}
I20260709 12:28:02.493029 26201 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:28:02.481338 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:02.528564 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:02.483721  3626 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:28:02.529865 26054 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:02.539610 26047 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:28:02.552140 26121 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:02.404632  3629 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:02.603680 26166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.023s	sys 0.038s
W20260709 12:28:02.652838 26166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.023s	sys 0.038s
W20260709 12:28:02.629083 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.482088  3362 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.482667  3592 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.356606  3633 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.646777 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:02.646086 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.645476 26048 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 18. (term mismatch)
I20260709 12:28:02.671204 26048 pending_rounds.cc:85] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 17
I20260709 12:28:02.671120 26125 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:02.671612 26048 pending_rounds.cc:107] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 31.18
I20260709 12:28:02.658111 26054 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:02.465235  3637 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.114s
W20260709 12:28:02.649291 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 300750us
I20260709 12:28:02.472409  3524 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.485066  3595 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.164s
I20260709 12:28:02.671926 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1367) took 1192 ms. Trace:
I20260709 12:28:02.499140  3260 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.163s
I20260709 12:28:02.682783 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:02.671936 26048 raft_consensus.cc:2889] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:02.684499 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:02.688441 26201 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:02.689690 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:02.667964 26194 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:02.472615  3610 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.667062  2830 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:02.526558  3604 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.668263  3536 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.651619  3381 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:02.696434 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.703158 26194 raft_consensus.cc:2393] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:02.660804  3533 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.700716 26199 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:02.701011 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:02.700314  3408 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:02.716955  3408 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:02.717373  3635 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:28:02.717486  2830 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.717878  3408 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.719065 26047 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:28:02.719148  3408 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:02.722682  3629 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:02.723863  3524 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:02.724484  3524 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.724983  3524 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.720994 26125 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:28:02.725975 26125 raft_consensus.cc:606] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:02.726333 26125 raft_consensus.cc:614] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:02.724659  3629 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:02.727856  3629 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.728814  3629 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.680871  3592 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.730571 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1813) took 1390 ms. Trace:
I20260709 12:28:02.730996 26124 rpcz_store.cc:276] 0709 12:28:01.339967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.340088 (+   121us) service_pool.cc:224] Handling call
0709 12:28:01.366663 (+ 26575us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:01.476842 (+110179us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:01.923331 (+446489us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.481927 (+558596us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:02.495273 (+ 13346us) log.cc:844] Serialized 52 byte log entry
0709 12:28:02.495470 (+   197us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:02.495504 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.495574 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.730304 (+234730us) raft_consensus.cc:1685] finished
0709 12:28:02.730325 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.730514 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93144,"prepare.queue_time_us":234525,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":142948,"raft.run_cpu_time_us":478,"raft.run_wall_time_us":54591,"thread_start_us":726724,"threads_started":2,"wal-append.queue_time_us":736894}
I20260709 12:28:02.743806  3149 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:02.744182  3633 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.674670  3362 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.753275 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2973) took 1441 ms. Trace:
I20260709 12:28:02.753712 26052 rpcz_store.cc:276] 0709 12:28:01.312160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.344755 (+ 32595us) service_pool.cc:224] Handling call
0709 12:28:01.345129 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.472515 (+127386us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:01.570375 (+ 97860us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.172641 (+602266us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.172918 (+   277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.172986 (+    68us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:02.173058 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.753046 (+579988us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400173254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:02.753093 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.753217 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25754,"prepare.queue_time_us":427974,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":101,"raft.queue_time_us":124,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":66709,"spinlock_wait_cycles":292301184,"thread_start_us":694470,"threads_started":2,"wal-append.queue_time_us":754534}
I20260709 12:28:02.754981 26049 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:02.720399  3639 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:02.756810 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1368) took 1168 ms. Trace:
I20260709 12:28:02.757100 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.757210 26050 rpcz_store.cc:276] 0709 12:28:01.588157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.588332 (+   175us) service_pool.cc:224] Handling call
0709 12:28:01.588724 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.176434 (+587710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.176496 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.249328 (+ 72832us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.651030 (+401702us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:02.651073 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.651149 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.756474 (+105325us) raft_consensus.cc:1685] finished
0709 12:28:02.756498 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.756651 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141101,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":879833,"threads_started":2,"wal-append.queue_time_us":433240}
I20260709 12:28:02.758379 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.760454  3637 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:02.760591  3580 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:02.761070  3637 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.761394 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.761372  3580 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.763301 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.763903 26200 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:02.769739  3536 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.773428 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3415) took 1070 ms. Trace:
I20260709 12:28:02.773725 26129 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:02.773835 26127 rpcz_store.cc:276] 0709 12:28:01.702581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.708707 (+  6126us) service_pool.cc:224] Handling call
0709 12:28:02.773231 (+1064524us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:02.773369 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383224704}
I20260709 12:28:02.777076  3381 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.762305 26197 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:02.782924 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.784181 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.785252 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.785799 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:02.786046  3590 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:02.786514  3590 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:02.786587 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.787413  3590 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.789256  3590 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:02.789716  3604 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.791657 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1359) took 1875 ms. Trace:
I20260709 12:28:02.792500 26045 rpcz_store.cc:276] 0709 12:28:00.916396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.916602 (+   206us) service_pool.cc:224] Handling call
0709 12:28:00.920105 (+  3503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.684096 (+763991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.791393 (+1107297us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54000f8554. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:02.791578 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559445248}
I20260709 12:28:02.799450 26200 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:28:02.805447 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.816327  3355 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.816903  3355 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.817747  3252 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:02.818313  3252 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:02.819135  3592 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:02.820326  3362 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.822542  3629 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.815840  3546 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.830500 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.831529 26202 raft_consensus.cc:2368] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26.
I20260709 12:28:02.772046  3536 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.839051 26195 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:02.840567 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.842447 26202 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:28:02.843596 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:02.819366  3252 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:02.843648 26194 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:02.695534  3603 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.851363 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.863263 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.848989 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:02.864332 26194 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:02.864357 26786 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:02.775511  3637 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.806689  3610 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.865795  3603 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.881815  3612 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.892894 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.886591 26200 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 23.
I20260709 12:28:02.896921 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.897977 26200 raft_consensus.cc:2368] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33.
I20260709 12:28:02.908975 26199 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:02.849728  3378 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.914981 26197 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:02.887893  3580 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:02.895329  3566 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.886381  3533 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:02.924398 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.904448  3594 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:02.847275  3579 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:28:02.901397  3610 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.900813  3521 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:28:02.932830 26786 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
I20260709 12:28:02.939919 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.940798 26202 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:02.940269 26093 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:02.949026 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 236187us
I20260709 12:28:02.921805 26194 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:02.951272 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:02.952198 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.953101 26200 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:02.920665  3378 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:03.016597 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66715.1us
I20260709 12:28:03.012851 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:02.933418  3564 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.018981 26194 raft_consensus.cc:2410] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 34 because replica has last-logged OpId of term: 31 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:28:03.017565 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:02.942190  3594 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.929719  3566 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:02.944352  3607 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:03.043174 26197 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:02.950573  3612 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:28:03.027539  3642 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 12:28:03.051954 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:02.683471 26051 rpcz_store.cc:276] 0709 12:28:01.479471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.480005 (+   534us) service_pool.cc:224] Handling call
0709 12:28:01.480326 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.480518 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:02.156876 (+676358us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.670994 (+514118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.671136 (+   142us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:02.671205 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.671747 (+   542us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.671873 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24492,"prepare.queue_time_us":476051,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":60,"raft.queue_time_us":36951,"thread_start_us":1064165,"threads_started":2}
W20260709 12:28:03.032565  3605 consensus_meta.cc:287] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.001s
I20260709 12:28:02.931924  3533 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.020030  3408 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:28:03.089421 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 71963.1us
I20260709 12:28:03.013710  3657 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:03.013866  3592 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:02.945182  3610 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:03.112967 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:03.106580 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 156333us
I20260709 12:28:03.139704 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.060504  3536 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.105705  3533 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.147078  3578 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:03.169137 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.103360  3605 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.088204  3647 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:03.108485  3590 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:03.150293  3649 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 12:28:03.085620  3624 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.301s
I20260709 12:28:03.108250  3629 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:03.044952  3594 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.151881  3655 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:03.157130  3408 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:03.164142 26200 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:28:03.172089 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:03.084556  3564 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:03.045001  3608 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.115s
I20260709 12:28:03.186583 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:03.152050  3654 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:28:03.085016  3612 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.121027  3607 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.217522 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.218649 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.189652  3629 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.153388  3650 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:28:03.248572  3629 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:03.249071  3605 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:03.225375 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.231115 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.250325 26201 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 23: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:03.250540  3566 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.251354 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:03.252787 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.254166  3607 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.257545 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4437) took 1873 ms. Trace:
I20260709 12:28:03.258167 26121 rpcz_store.cc:276] 0709 12:28:01.383651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.460568 (+ 76917us) service_pool.cc:224] Handling call
0709 12:28:01.460870 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.552927 (+1092057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.257182 (+704255us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001f81d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:03.257452 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456966016}
I20260709 12:28:03.237844  3594 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:03.240027  3610 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.246610  3604 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:03.246922  3603 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.161234  3592 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.154259  3657 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.225356 26194 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 29 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:28:03.273145 26202 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
W20260709 12:28:03.273533 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48232.7us
I20260709 12:28:03.167624  3536 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.181272  3578 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.278263 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.279917 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:03.265229  3658 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.111s	user 0.005s	sys 0.005s
I20260709 12:28:03.189729  3647 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.284878 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:03.276486  3659 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.121s	user 0.000s	sys 0.003s
I20260709 12:28:03.220867  3564 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.291613 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4440) took 1907 ms. Trace:
I20260709 12:28:03.293255 26128 rpcz_store.cc:276] 0709 12:28:01.384511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.544100 (+159589us) service_pool.cc:224] Handling call
0709 12:28:01.544560 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:02.236429 (+691869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.236485 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:02.237460 (+   975us) log.cc:844] Serialized 52 byte log entry
0709 12:28:02.681509 (+444049us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:02.681557 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.681738 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.290950 (+609212us) raft_consensus.cc:1685] finished
0709 12:28:03.290968 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.291532 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22785,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":71288,"spinlock_wait_cycles":1105792,"thread_start_us":1110807,"threads_started":2,"wal-append.queue_time_us":680101}
I20260709 12:28:03.277549  3601 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:03.297626 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:03.300850 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.314258 26200 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:03.315353 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3403) took 1001 ms. Trace:
I20260709 12:28:03.316998 26202 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:03.286418  3521 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:03.315774 26200 rpcz_store.cc:276] 0709 12:28:02.313575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.287460 (+973885us) service_pool.cc:224] Handling call
0709 12:28:03.313951 (+ 26491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.315002 (+  1051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.315283 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.312882  3518 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:03.312816  3525 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:03.323321  3518 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.335002  3518 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:03.336818 26094 leader_election.cc:400] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26.
I20260709 12:28:03.343039 26094 leader_election.cc:403] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:28:03.343209 26202 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:03.321439  3591 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:03.347236 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3399) took 1036 ms. Trace:
I20260709 12:28:03.336695  3521 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 12:28:03.349013 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3401) took 1037 ms. Trace:
W20260709 12:28:03.350142 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 123968us
I20260709 12:28:03.351236 26198 rpcz_store.cc:276] 0709 12:28:02.310840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.311067 (+   227us) service_pool.cc:224] Handling call
0709 12:28:02.355567 (+ 44500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.694778 (+339211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.694927 (+   149us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.712127 (+ 17200us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.163481 (+451354us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:03.163522 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.163621 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.346979 (+183358us) raft_consensus.cc:1685] finished
0709 12:28:03.347004 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.347168 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35052,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":624795,"threads_started":2,"wal-append.queue_time_us":435230}
I20260709 12:28:03.354516 26199 rpcz_store.cc:276] 0709 12:28:02.311972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.312155 (+   183us) service_pool.cc:224] Handling call
0709 12:28:03.348733 (+1036578us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400300f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:03.348917 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588193536}
I20260709 12:28:03.352756 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3405) took 1029 ms. Trace:
I20260709 12:28:03.356994 26202 rpcz_store.cc:276] 0709 12:28:02.314144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.328189 (+1014045us) service_pool.cc:224] Handling call
0709 12:28:03.342842 (+ 14653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.343640 (+   798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.343828 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.296484  3408 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.357892 26200 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:03.369201 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3409) took 1048 ms. Trace:
I20260709 12:28:03.362903 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.351150  3591 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:03.370409 26200 rpcz_store.cc:276] 0709 12:28:02.320860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.331791 (+1010931us) service_pool.cc:224] Handling call
0709 12:28:03.353441 (+ 21650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.368022 (+ 14581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.369134 (+  1112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.371068 26199 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:03.372859 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2333) took 2365 ms. Trace:
I20260709 12:28:03.373075  3647 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.373770 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:03.371174 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3396) took 1063 ms. Trace:
I20260709 12:28:03.375649 26196 rpcz_store.cc:276] 0709 12:28:02.307995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.308390 (+   395us) service_pool.cc:224] Handling call
0709 12:28:02.308801 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.444064 (+135263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.370890 (+926826us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5400335c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:03.371116 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24941824}
I20260709 12:28:03.376974 26196 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:03.373328 26791 rpcz_store.cc:276] 0709 12:28:01.007697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.009450 (+  1753us) service_pool.cc:224] Handling call
0709 12:28:01.009838 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.389558 (+1379720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.372618 (+983060us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000e2c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:03.372779 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3170973952}
I20260709 12:28:03.369577  3536 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:03.358551  3381 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.359473  3525 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 25. Current term is 26.
I20260709 12:28:03.374253  3657 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.384076  3578 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.373600  3591 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.384189  3564 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.361202  3612 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:03.384920 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48223us
I20260709 12:28:03.391073  3592 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.401698 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.403844 26094 leader_election.cc:400] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33.
I20260709 12:28:03.403821 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.406217 26200 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:03.415056 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:03.424425 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.433597  3610 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:03.413805  3564 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:03.436117 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.440173 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.433151  3591 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:03.431625  3500 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.435959  3610 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.454012 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.454461 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.454941 26054 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:03.458005 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:03.463518 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.463959 26783 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:03.464900 26785 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:03.465307 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:03.465575  3617 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.466392  3603 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:03.467129  3603 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.468703  3521 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:03.469856  3603 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.441110  3647 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:03.470516  3500 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.447852  3607 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:03.472400  3607 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.473008  3607 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:03.436122  3525 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:03.474596 26197 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 20. (term mismatch)
I20260709 12:28:03.468071 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:03.480688 26197 pending_rounds.cc:85] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 19
I20260709 12:28:03.480989 26197 pending_rounds.cc:107] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 28.20
I20260709 12:28:03.481282 26197 raft_consensus.cc:2889] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:03.485692 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.487404  3617 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.487748 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.488022 26045 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:03.488592 26048 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:03.489271 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.489698 26048 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:03.489879 26045 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:03.490444  3584 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:03.491796 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.487998  3617 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.438681  3629 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.493115 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4347) took 1019 ms. Trace:
I20260709 12:28:03.493484 26049 rpcz_store.cc:276] 0709 12:28:02.473231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.526398 (+ 53167us) service_pool.cc:224] Handling call
0709 12:28:03.492943 (+966545us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400020654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:03.493049 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479382400}
I20260709 12:28:03.493959  3617 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.495685 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:03.497836  3632 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:03.445128  3564 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:03.501080  3564 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.503233 26201 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:03.505401 26051 raft_consensus.cc:1240] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:03.507894 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.508951 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:03.509902 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6032219500002611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 12:28:03.510381 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5915219500002422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 12:28:03.510792 26094 connection.cc:256] RPC call timeout handler was delayed by -1.4673219500000414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 12:28:03.511279 26094 connection.cc:256] RPC call timeout handler was delayed by -1.4223219499999686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
W20260709 12:28:03.511662 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3611219500003244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 12:28:03.512013 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2621219500001644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
I20260709 12:28:03.513648 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.528522 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.532806 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.534097 26050 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 27 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:28:03.526170 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.537827 26049 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:03.541076 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.541944 26052 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:03.542034 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.542634 26048 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:03.473480  3500 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.545395 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.541957 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.499818  3632 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:28:03.544402 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.546260 26047 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:03.497892  3584 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:03.509752  3521 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.548609 26052 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:03.549178  3521 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.489888  3382 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:03.514875  3624 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.553547 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.553731 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.515627  3524 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.563951 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.515763  3518 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.590936 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.593135 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.552196  3584 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.551503  3521 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.592026 26045 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:03.595671 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.595671 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.543632  3601 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
I20260709 12:28:03.538432  3634 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.032s
I20260709 12:28:03.544003  3525 consensus_queue.cc:1059] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:28:03.600641 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:03.600643 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.600044 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:03.616765 26787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2337) took 2309 ms. Trace:
I20260709 12:28:03.617462 26787 rpcz_store.cc:276] 0709 12:28:01.307740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.307959 (+   219us) service_pool.cc:224] Handling call
0709 12:28:01.339568 (+ 31609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.384280 (+1044712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.616360 (+1232080us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:03.616673 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2400836992}
I20260709 12:28:03.615314 26050 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:03.590672  3524 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.603135 26049 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:03.601989 26048 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
I20260709 12:28:03.623767 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4445) took 2237 ms. Trace:
I20260709 12:28:03.594617  3639 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:03.616820 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 90207.3us
I20260709 12:28:03.564736  3624 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.633704 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.627441 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.634397 26122 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:03.634495 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.635260 26123 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:28:03.627846 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:03.615164  3647 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:03.631656 26129 raft_consensus.cc:1240] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:03.635826 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.636324 26200 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:03.630970 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:03.639853 26121 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:03.636368 26127 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:03.601238  3584 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:03.634500 26054 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:03.626535 26126 rpcz_store.cc:276] 0709 12:28:01.385847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.171619 (+785772us) service_pool.cc:224] Handling call
0709 12:28:02.171968 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.172193 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:02.175166 (+  2973us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.494878 (+1319712us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b18002b6840. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:28:03.623394 (+128516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.623443 (+    49us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:03.623493 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.623592 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.623709 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6514,"prepare.queue_time_us":132,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"spinlock_wait_cycles":5518592,"thread_start_us":121714,"threads_started":1}
I20260709 12:28:03.645192 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.648898 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.602936  3632 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.653043 26126 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:03.667444 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.668675 26196 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 22.
I20260709 12:28:03.645694  3634 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:03.670045 26126 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:03.670670 26197 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:03.658843  3525 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.676640 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.677449 26051 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:03.682643  3355 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.633548  3524 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.694522 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.696857 26054 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:28:03.698285 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.699554 26127 raft_consensus.cc:2393] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:03.696431 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.711053 26200 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:03.636679  3639 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.713730 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.712072 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.714112 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.715256  3661 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.009s
I20260709 12:28:03.719672 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.715620 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.717268 26054 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:03.717370 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:54832 (request call id 0) took 1372 ms. Trace:
I20260709 12:28:03.672124  3634 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:03.661198  3632 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.722012 26197 rpcz_store.cc:276] 0709 12:28:02.321804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.677763 (+1355959us) service_pool.cc:224] Handling call
0709 12:28:03.693784 (+ 16021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.740978 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.750290 26127 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 28 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 21.
W20260709 12:28:03.717640 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 204100us
I20260709 12:28:03.717567 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.716289 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.708737  3537 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.762538 26121 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:03.750597 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.721971  3546 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:03.753827  3607 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.713634  3382 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:03.761409 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1615) took 1101 ms. Trace:
I20260709 12:28:03.741442  3617 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:03.764537 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:03.771179 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.739920  3590 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:03.776664  3518 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:03.776995  3590 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.765584 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:03.771903 26127 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:03.696372  3525 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.768930 26201 rpcz_store.cc:276] 0709 12:28:02.659946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.252500 (+592554us) service_pool.cc:224] Handling call
0709 12:28:03.761196 (+508696us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400378f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:03.761345 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573815680}
I20260709 12:28:03.748970 26054 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:03.769274 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:03.772931 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56774.2us
I20260709 12:28:03.777117  3314 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:03.787827  3314 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:03.788513  3314 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:03.788635  3639 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.778069 26121 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:03.765590 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.791496  3639 raft_consensus.cc:2764] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
W20260709 12:28:03.768158 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50708.6us
I20260709 12:28:03.796475 26019 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.803593  3639 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:03.804637  3639 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:03.805859  3639 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.812959  3518 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:03.816264 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.803587 26168 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.818548  3439 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:03.819221  3439 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:03.820307  3439 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:03.821702 26168 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.823103  3657 raft_consensus.cc:2804] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:03.824795  3657 raft_consensus.cc:697] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:03.826043  3657 consensus_queue.cc:237] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:03.830714 26000 auto_rebalancer.cc:893] Removing replace marker failed: Replica a981c9a7972c4e99a0903924c64fbb5c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:03.831252 26000 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:28:03.831616 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:28:03.832427 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:03.828415 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58966us
I20260709 12:28:03.845808 26121 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:28:03.847757 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.857234 26197 raft_consensus.cc:2410] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 33 because replica has last-logged OpId of term: 30 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:28:03.822747 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:03.853519 26122 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:03.858672 26756 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:03.766544  3546 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.861158 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.859866 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.770175  3578 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:03.885496 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:03.886417 26201 raft_consensus.cc:2376] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29. Re-sending same reply.
I20260709 12:28:03.777642  3590 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:03.881616 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:03.766710  3521 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:03.893136 26126 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:03.892040 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:03.766939  3500 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 12:28:03.767477  3537 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.769488  3355 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.118s
I20260709 12:28:03.890786  3617 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:03.871608  3382 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:03.871371  3546 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:03.914638  3518 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:28:03.916059  3673 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
W20260709 12:28:03.914103 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.968358 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:03.968966 26051 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:03.937857  3632 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:03.944994 26123 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
W20260709 12:28:03.907994 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56539.7us
I20260709 12:28:03.871479  3579 consensus_queue.cc:1059] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:28:03.913719  3578 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:03.983039 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:03.983415 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.985030 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:03.990378 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:03.996697 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.902961  3521 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.949214  3607 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:03.952762  3675 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:28:03.965631  3617 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:03.861454  3634 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:03.976078  3382 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:04.027274 26020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.021577 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.028697 26054 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:04.039790 26051 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:04.043995 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.047871 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.045979 26127 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:03.998495  3624 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:03.948181  3584 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.056828 26051 raft_consensus.cc:2410] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:28:03.949823  3590 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.088563 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.092768 26054 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:03.978703  3314 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.096535  3624 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:04.097487  3624 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:04.100710 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 244076us
I20260709 12:28:04.102259 26019 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:04.090973  3524 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:04.141362 26052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.019s
W20260709 12:28:04.103225  3591 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.002s
I20260709 12:28:04.142148 26051 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:04.103583  3634 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:04.146951 26052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.006s	sys 0.019s
I20260709 12:28:04.151266 26789 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:04.158228 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.159238 26051 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
W20260709 12:28:04.185494 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 303947us
I20260709 12:28:04.192291 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.201570 26051 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:28:04.207414 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:04.213125 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:04.210011  3565 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:04.217124 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.238387 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.185393  3355 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:28:04.239723 26787 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:28:04.260880 26787 pending_rounds.cc:85] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 17
I20260709 12:28:04.261217 26787 pending_rounds.cc:107] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:28:04.261483 26787 raft_consensus.cc:2889] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:04.201457  3657 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:04.247979 26790 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:04.198400  3592 consensus_queue.cc:1059] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.029s
W20260709 12:28:04.269752 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 167604us
I20260709 12:28:04.201349  3632 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:04.208212 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.263185 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.231532  3615 raft_consensus.cc:3055] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:04.217094  3537 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.151676  3617 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.268895 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.269428 26791 raft_consensus.cc:1240] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:04.269737 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.283828 26783 raft_consensus.cc:2393] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 25: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:04.275107 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.283881 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.284621 26784 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 31 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:28:04.278254 26786 raft_consensus.cc:2368] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:28:04.278802  3615 raft_consensus.cc:740] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:04.296631  3584 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:04.305136 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.279091 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.160758  3382 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.306813 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.307394 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.320776 26791 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
W20260709 12:28:04.325301 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 117696us
I20260709 12:28:04.271148 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:04.327201 26783 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:04.286613  3525 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.332798 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.334745 26785 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:04.334228 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:04.332590 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.333606 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.335873 26791 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:04.326030  3683 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.565s	user 0.005s	sys 0.000s
W20260709 12:28:04.339140 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 68084.4us
I20260709 12:28:04.306329  3706 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:04.276115  3657 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.340444 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:04.342286  3695 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:04.342783  3695 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:04.343179  3695 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:04.346627 26785 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:04.325026  3314 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.347785 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2347) took 1497 ms. Trace:
I20260709 12:28:04.283162  3632 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.333837  3578 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:04.332619  3590 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.350921  3708 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:04.351688  3640 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:04.352902  3548 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:04.339484  3688 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.195s	user 0.002s	sys 0.000s
I20260709 12:28:04.353624 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.354697  3314 consensus_queue.cc:1059] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:28:04.355183  3693 log.cc:927] Time spent T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:28:04.387778  3639 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.007s
W20260709 12:28:04.355930  3691 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:28:04.358633  3690 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:28:04.370201  3706 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:04.359371  3689 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:28:04.392791  3624 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:04.392942  3706 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.393911  3629 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.394565  3629 raft_consensus.cc:3055] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:04.394943  3629 raft_consensus.cc:740] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:04.396137 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:04.396633 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.397778 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.398535  3521 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.398829 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.399111  3639 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:04.399500  3639 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:04.399925  3639 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:04.401041  3709 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:04.403790 26788 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:04.306097  3703 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:28:04.404901 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:04.348205 26782 rpcz_store.cc:276] 0709 12:28:02.850572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.850808 (+   236us) service_pool.cc:224] Handling call
0709 12:28:02.889832 (+ 39024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.893443 (+  3611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.893474 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.894088 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.187481 (+1293393us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b1800425980. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:28:04.325064 (+137583us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:04.325111 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.325242 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.347494 (+ 22252us) raft_consensus.cc:1685] finished
0709 12:28:04.347514 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.347730 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3166,"raft.queue_time_us":12415,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9459,"spinlock_wait_cycles":3293440,"thread_start_us":137382,"threads_started":1,"wal-append.queue_time_us":1440932}
I20260709 12:28:04.405651 26788 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:04.406062 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.406467 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.336525  3697 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.407279 26788 raft_consensus.cc:2410] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 31 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:28:04.407348  3697 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.408488 26786 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:04.409595 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:04.410162 26784 raft_consensus.cc:2376] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25. Re-sending same reply.
I20260709 12:28:04.410226  3697 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.366441  3615 consensus_queue.cc:260] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:04.419600 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:04.420039 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:04.420631 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.422303 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.380797 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.381537 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.407658  3695 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.371826  3708 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.389328 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.379062  3518 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:04.397621  3694 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:04.399618 26124 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:04.453565 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.362295 26129 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:04.360175 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.456126 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.457165 26785 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:04.458441 26785 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:04.424561 26787 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
W20260709 12:28:04.379321 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 402937us
I20260709 12:28:04.389395 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.429421 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:04.434247 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:04.460420 26047 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:04.380373  3640 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:04.435380 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:04.433385 26167 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.463193 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.449738 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.464212 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:04.464473 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:04.464924 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.465135 26124 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:04.465523 26121 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:28:04.460736 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:04.461063 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:04.391853  3657 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.460633 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.399821  3546 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.423177  3599 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.467133 26786 raft_consensus.cc:3060] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:04.435123  3615 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.468784 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:28:04.448949  3683 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:28:04.467172 26195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3402) took 2154 ms (client timeout 1724 ms). Trace:
I20260709 12:28:04.482896 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.395772  3629 consensus_queue.cc:260] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.491266  3680 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:04.472110  3149 consensus_queue.cc:1059] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 12:28:04.499771  3680 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:04.500265  3680 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:04.462071  3697 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:28:04.502769  3640 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:04.493171 26195 rpcz_store.cc:269] 0709 12:28:02.312602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.312839 (+   237us) service_pool.cc:224] Handling call
0709 12:28:04.466993 (+2154154us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:04.467101 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875544320}
I20260709 12:28:04.499969  3149 consensus_queue.cc:1243] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 29.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:28:04.504436  3640 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:04.505232 26167 leader_election.cc:400] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:28:04.505556 26167 leader_election.cc:403] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:04.505892  3525 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:04.506232 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.499246  3629 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.508042 26786 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:04.470541  3624 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:04.505244 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:04.513199  3714 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.513757  3378 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.514065  3634 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:04.514164  3618 raft_consensus.cc:3055] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:04.514552 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:04.434994  3708 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:04.517104 26782 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:28:04.517799 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.519109 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:04.435737  3565 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:04.461737  3408 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 12:28:04.470180  3521 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:28:04.514685  3618 raft_consensus.cc:740] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:04.431528  3696 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 12:28:04.434736  3590 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:04.520205 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.520807 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.497937 26124 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:04.523672 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.521091 25946 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 18 to 22, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 22 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:04.496546 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 99321.4us
I20260709 12:28:04.554634 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.555678 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.525678 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:04.548867 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 193208us
I20260709 12:28:04.472306  3699 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:04.551005 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.431190  3639 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.557397 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:04.557639 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:04.557976 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:04.569270 26783 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:04.561239 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.547816  3260 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:04.603578 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.604718 26121 raft_consensus.cc:2393] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:04.588456 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.551939  3378 raft_consensus.cc:3060] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:04.548668  3657 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:04.514603  3634 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.621892 26052 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:04.606144 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:04.624320 26128 raft_consensus.cc:2393] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:04.520552  3710 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:04.558921  3697 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:04.521140  3599 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.637437 26128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:28:04.638209 26128 raft_consensus.cc:606] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.638813 26128 raft_consensus.cc:614] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:04.587702  3624 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.633760 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:04.553151  3149 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:04.625370 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74986.4us
I20260709 12:28:04.610801  3260 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.513759  3714 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.641013 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:04.622438  3634 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:04.622515  3604 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:28:04.652397 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 94469.6us
I20260709 12:28:04.654577 26128 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:28:04.651762 26121 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:04.514621  3518 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.662729 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.647833  3260 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:04.651489  3685 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:04.684182 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:04.685153 26203 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:04.686565 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:04.651889  3525 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.690389 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.692101 26197 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:28:04.696573 26094 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.326s (SENT)
I20260709 12:28:04.697062 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:04.698617 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.700373 26197 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:04.700896 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4344) took 2225 ms. Trace:
I20260709 12:28:04.707410 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.708079 26200 raft_consensus.cc:2393] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 25: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:04.709019 26201 raft_consensus.cc:1240] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:04.701442 26046 rpcz_store.cc:276] 0709 12:28:02.463806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.463929 (+   123us) service_pool.cc:224] Handling call
0709 12:28:02.464382 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.794317 (+329935us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:02.794618 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.207408 (+412790us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.207470 (+    62us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:03.207539 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.689384 (+1481845us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540022b854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:04.689448 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.689620 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23568,"prepare.queue_time_us":510,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":48,"raft.queue_time_us":27121,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3533,"spinlock_wait_cycles":758165760,"thread_start_us":388783,"threads_started":1,"wal-append.queue_time_us":486515,"wal-append.run_cpu_time_us":2288,"wal-append.run_wall_time_us":1052427}
I20260709 12:28:04.677570  3149 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:04.677039  3685 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:04.676800  3729 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:04.688287  3632 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.660985  3716 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:28:04.637152  3618 consensus_queue.cc:260] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.649544  3624 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.713392  3149 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.651084  3565 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.725081  3685 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:04.740505 26121 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:04.715094  3695 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.742975 26128 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:28:04.770474 26121 raft_consensus.cc:1240] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:28:04.703439  3525 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:04.769609 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.739895  3546 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.746955  3149 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:04.784977 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 88426.3us
I20260709 12:28:04.780503  3728 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:04.776984  3726 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.784760  3618 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:04.799202 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:04.791107  3718 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:28:04.820828 26124 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:04.804913 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4460) took 1129 ms. Trace:
I20260709 12:28:04.818809 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.745888  3727 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.056s
I20260709 12:28:04.824354 26126 rpcz_store.cc:276] 0709 12:28:03.675945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.676105 (+   160us) service_pool.cc:224] Handling call
0709 12:28:03.676407 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.897936 (+221529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.804590 (+906654us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400287654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:04.804845 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497907584}
I20260709 12:28:04.821956 26203 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:28:04.829190 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.818483 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.831236 26201 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:04.825560 26197 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:04.840886 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:04.841281 26197 raft_consensus.cc:3055] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:04.841842 26197 raft_consensus.cc:740] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:04.790714  3639 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:04.817041  3725 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:28:04.843016 26197 consensus_queue.cc:260] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:04.800166  3724 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 12:28:04.821390  3715 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 12:28:04.809006  3717 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:28:04.815608  3729 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.816805  3632 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.862002 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.852396 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.865459 26201 raft_consensus.cc:2410] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
W20260709 12:28:04.848021 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 115031us
I20260709 12:28:04.815485  3695 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.870843  3252 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:04.824852  3408 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:28:04.856796 26197 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.872409  3695 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.885805 26125 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 25 index: 21. (index mismatch)
I20260709 12:28:04.886641  3695 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.893221 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3410) took 2572 ms. Trace:
I20260709 12:28:04.893695 26198 rpcz_store.cc:276] 0709 12:28:02.321239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.354977 (+1033738us) service_pool.cc:224] Handling call
0709 12:28:03.363167 (+  8190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:03.369667 (+  6500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.369711 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:03.386047 (+ 16336us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.359045 (+972998us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18003ce140. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f450d1 00007f5b74360a0f
0709 12:28:04.617239 (+258194us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:04.617283 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.617358 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.892911 (+275553us) raft_consensus.cc:1685] finished
0709 12:28:04.892930 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.893157 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30,"raft.queue_time_us":1188,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9006,"spinlock_wait_cycles":28723968,"thread_start_us":251222,"threads_started":1,"wal-append.queue_time_us":1317709}
I20260709 12:28:04.896744  3714 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:04.867342  3734 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:04.895852 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:04.902755 26194 raft_consensus.cc:2393] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:04.910364  3647 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:04.910844  3647 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:04.911674 26053 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:28:04.912452  3599 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.912861  3603 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:04.922271  3723 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:04.911739  3647 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.924505  3525 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.925503  3647 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.937456  3149 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.939000 26198 raft_consensus.cc:2393] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:04.918967  3708 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.941258  3624 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.941735  3708 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:04.913430  3603 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:04.944972 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3421) took 1005 ms. Trace:
I20260709 12:28:04.945519 26120 rpcz_store.cc:276] 0709 12:28:03.939478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.939625 (+   147us) service_pool.cc:224] Handling call
0709 12:28:03.939941 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.371325 (+431384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.371359 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.383164 (+ 11805us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.640461 (+257297us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:04.640508 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.640582 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.944683 (+304101us) raft_consensus.cc:1685] finished
0709 12:28:04.944715 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.944893 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":462485,"raft.queue_time_us":447768,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":208071,"threads_started":1,"wal-append.queue_time_us":233408}
I20260709 12:28:04.948776  3149 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.949898  3680 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.952363  3699 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:04.866539  3729 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.956040  3603 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:04.956370  3634 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.963676  3709 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.969513  3634 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.969437  3709 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:04.972481  3709 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.973124  3578 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:04.973660  3578 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:04.959781  3726 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:04.964148 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:04.974620 26053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 2975) took 1522 ms (client timeout 1545 ms). Trace:
W20260709 12:28:04.975204 26053 rpcz_store.cc:269] 0709 12:28:03.452780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.454208 (+  1428us) service_pool.cc:224] Handling call
0709 12:28:04.974454 (+1520246us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400227c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:04.974560 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3333780736}
I20260709 12:28:04.978125 26125 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:04.978264 26121 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:04.978392 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:04.979803 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 181664us
I20260709 12:28:04.980727 26094 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:04.979938 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:04.986956 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:04.992946 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:04.977614  3680 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:04.866807  3632 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.002560 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:05.010581 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:05.013120 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.014128 26124 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:04.994756  3408 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:28:05.013176 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:05.015072 26197 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:28:05.021456 26197 pending_rounds.cc:85] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 17
I20260709 12:28:05.021796 26197 pending_rounds.cc:107] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:28:05.022051 26197 raft_consensus.cc:2889] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:05.023288 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.023999 26197 raft_consensus.cc:2393] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 22: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:05.015456 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 190833us
I20260709 12:28:05.016500 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:05.002194  3726 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:05.026074 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:05.026280 26757 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:05.027133 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:05.028486 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.029357 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:05.029924 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:05.030279 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:05.030874 26197 raft_consensus.cc:3060] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.009346  3592 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:05.034034 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:05.036216 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:05.036767 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 479166us
W20260709 12:28:05.040014 25939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:05.014730  3584 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:28:05.038997 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.040805 26757 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:05.021363  3696 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.041990 26169 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:05.044332 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:05.048271 26169 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:05.050349 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.052076 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:05.054636 26197 raft_consensus.cc:2468] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:05.055219 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:05.064837 26124 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:28:05.033552  3592 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.083770  3592 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.042342  3606 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.052657  3603 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:05.090984 26120 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:28:05.032738  3382 raft_consensus.cc:2804] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:28:05.099225 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47274.4us
I20260709 12:28:05.078639  3629 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:05.041273  3705 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:05.102485 26120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:28:05.015642  3685 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:05.106377  3676 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.383s	user 0.002s	sys 0.004s
I20260709 12:28:05.106986 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:05.107326 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4457) took 1472 ms. Trace:
I20260709 12:28:05.107909  3629 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:05.108649  3732 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:05.069705  3714 raft_consensus.cc:2804] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:05.093549  3604 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:05.093711  3565 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:05.067374  3149 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:05.045938  3584 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:05.118984 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 105898us
I20260709 12:28:05.042276  3378 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 27. Current term is 28.
I20260709 12:28:05.103260 26120 raft_consensus.cc:606] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Received request to transfer leadership to TS a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:05.104750 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49646us
I20260709 12:28:05.121692 26120 raft_consensus.cc:614] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:05.123911 26203 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:28:05.103891 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 59213.6us
I20260709 12:28:05.071439  3726 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:05.083106  3735 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:05.107744 26122 rpcz_store.cc:276] 0709 12:28:03.635402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.635589 (+   187us) service_pool.cc:224] Handling call
0709 12:28:05.107151 (+1471562us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001f6dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:05.107273 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337900800}
I20260709 12:28:05.082351  3632 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:05.077736 26788 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:05.100540  3518 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:05.132155  3726 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:05.129961  3748 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 12:28:05.133219  3726 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.118379  3260 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.136801  3685 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.105780 26197 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:05.125785 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:05.125164 26756 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:05.101477  3740 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:28:05.114022  3382 raft_consensus.cc:697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:05.116583  3610 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.121156  3584 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.128737  3745 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:05.109017  3629 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.118610  3604 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:05.144213 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:05.152562 26783 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.124281  3714 raft_consensus.cc:697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:05.144047 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:05.146816 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:05.148248  3378 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.144438  3565 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:05.151295 26122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1"
dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
mode: GRACEFUL
new_leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
 from {username='slave'} at 127.0.0.1:58316
I20260709 12:28:05.149148  3518 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.158524 26788 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:05.163492 26122 raft_consensus.cc:606] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Received request to transfer leadership to TS 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:05.165953 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:05.162379  3632 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:05.168326  3632 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.168529  3729 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.168843  3632 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:05.129668  3732 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.136847  3260 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.172519 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:05.162953  3565 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.181737 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:05.168340  3745 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:05.185823 26122 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:05.187757 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.188699 26195 raft_consensus.cc:2410] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 26 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:28:05.188395  3578 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.189550 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.190110 26195 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:05.191473 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.192024 26195 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:05.193074  3592 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.191893  3382 consensus_queue.cc:237] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.202117 26788 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:05.193100 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.201592 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.213002 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.213620 26783 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:05.214107 26122 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 12.
I20260709 12:28:05.216583 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:05.219962 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:05.184367  3565 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.225092  3578 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:05.224742  3745 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.168852  3714 consensus_queue.cc:237] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.173074  3518 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.228037  3518 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:05.228469 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.177455  3741 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.196025  3592 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.230103 26120 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:05.215648 26167 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:05.175472  3732 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.221129 26122 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.239553 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:05.208300  3149 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:05.245898 26195 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.251641 26783 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:05.259820 26120 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:05.265196 26120 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:28:05.268591 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.269727 26120 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:05.272928 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:05.284652 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.295615 26756 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.542s (SENT) [suppressed 20 similar messages]
I20260709 12:28:05.320873 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.321532 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:05.322352 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.323273 26120 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:05.247728  3612 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:05.335623 26169 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:05.229854  3603 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 12:28:05.373162 26195 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:05.374486 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.317015  3524 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:05.316516  3260 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.395013 26122 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:05.245546  3604 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.401291 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:05.332362  3695 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:05.404436 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:05.404865 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:05.405297 26787 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:28:05.398207 26122 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:28:05.237427  3741 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.410768 26195 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31. Ops: [26.17-30.19]
I20260709 12:28:05.412482 26195 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:28:05.385495  3660 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:05.319227  3606 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 32.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.073s
I20260709 12:28:05.413717 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:05.415171 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:05.415882 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:05.416514 26195 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:05.318725  3149 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.427764 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:05.328099  3439 consensus_queue.cc:1059] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:28:05.398429  3647 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.443112 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:05.409322  3695 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.434753 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:05.427248  3514 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 29 current term: 29 is higher.
I20260709 12:28:05.416074  3500 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:05.413442  3640 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.413710  3518 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.473179 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:05.420053  3355 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.397294  3260 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.397096  3524 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.477557  3695 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.423990  3629 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.514582 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1379) took 1093 ms. Trace:
I20260709 12:28:05.546361 26052 rpcz_store.cc:276] 0709 12:28:04.421246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.421369 (+   123us) service_pool.cc:224] Handling call
0709 12:28:04.421762 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.622528 (+200766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.514320 (+891792us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400153454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:05.514515 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459761024}
I20260709 12:28:05.429487  3660 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:05.420310  3535 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.468716  3703 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.558619 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4468) took 1772 ms. Trace:
I20260709 12:28:05.469318  3743 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.508260  3514 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:05.568077 26127 rpcz_store.cc:276] 0709 12:28:03.786411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.860838 (+ 74427us) service_pool.cc:224] Handling call
0709 12:28:05.558392 (+1697554us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:05.558551 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416355712}
I20260709 12:28:05.570472 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4368) took 1907 ms. Trace:
I20260709 12:28:05.513155  3500 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:05.528374  3726 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.571950 26050 rpcz_store.cc:276] 0709 12:28:03.663193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.664067 (+   874us) service_pool.cc:224] Handling call
0709 12:28:03.673237 (+  9170us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:03.673646 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.673671 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:03.693780 (+ 20109us) log.cc:844] Serialized 52 byte log entry
0709 12:28:04.040916 (+347136us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:04.103107 (+ 62191us) log.cc:844] Serialized 14 byte log entry
0709 12:28:05.409420 (+1306313us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b1800376480. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:28:05.561151 (+151731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.561452 (+   301us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.561507 (+    55us) raft_consensus.cc:1685] finished
0709 12:28:05.569674 (+  8167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.570389 (+   715us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2182,"prepare.queue_time_us":61928,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":3848,"raft.queue_time_us":183333,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":22220,"spinlock_wait_cycles":11168640,"thread_start_us":442468,"threads_started":2,"wal-append.queue_time_us":424697,"wal-append.run_cpu_time_us":3729,"wal-append.run_wall_time_us":1247748}
I20260709 12:28:05.475314  3624 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.094s
I20260709 12:28:05.428113  3708 consensus_queue.cc:1059] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.140s
I20260709 12:28:05.483161  3618 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.098s
I20260709 12:28:05.593076  3703 raft_consensus.cc:3055] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:05.593568  3703 raft_consensus.cc:740] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:05.558624  3592 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.594411  3592 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.591123 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1832) took 1205 ms. Trace:
I20260709 12:28:05.566425  3660 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.595899 26121 rpcz_store.cc:276] 0709 12:28:04.385395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.781250 (+395855us) service_pool.cc:224] Handling call
0709 12:28:04.794821 (+ 13571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.978829 (+184008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.590800 (+611971us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003688d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:05.591029 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421436800}
I20260709 12:28:05.602813  3592 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.594471  3703 consensus_queue.cc:260] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.591151 26197 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:28:05.478653  3647 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.566617  3610 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:05.571070  3514 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.570533  3612 raft_consensus.cc:697] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:28:05.623559 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.581916  3579 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:05.526424  3518 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.622761  3514 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:05.637396  3749 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:28:05.639422  3524 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.641711  3260 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.642427  3647 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.588547  3726 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.648768  3741 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.649972  3726 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.589198  3537 raft_consensus.cc:3055] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:05.582571  3743 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.556087  3640 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.532898  3355 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.554188  3629 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.570912  3535 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.625795 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:05.646463  3584 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.627486  3610 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.818106  3763 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:28:05.800127 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 384938us
W20260709 12:28:05.819119  3632 consensus_meta.cc:287] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.650s	user 0.003s	sys 0.010s
I20260709 12:28:05.819329  3763 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:05.615687  3761 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:05.652155 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:05.651968  3755 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:28:05.704257 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.820359  3632 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:05.828179 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.830165  3632 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:05.820681  3763 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.830775  3355 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.661087 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1616) took 3000 ms. Trace:
I20260709 12:28:05.729421  3543 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:05.832424  3543 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:05.833698  3543 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.831638 26202 rpcz_store.cc:276] 0709 12:28:02.660169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.381889 (+721720us) service_pool.cc:224] Handling call
0709 12:28:03.404678 (+ 22789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.679884 (+275206us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:03.717934 (+ 38050us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.239406 (+521472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.239483 (+    77us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:04.239554 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.660732 (+1421178us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400162e54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:05.660801 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.660974 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46194,"prepare.queue_time_us":65377,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3266,"raft.queue_time_us":241464,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":5438,"spinlock_wait_cycles":632332032,"thread_start_us":517938,"threads_started":2,"wal-append.queue_time_us":307857,"wal-append.run_cpu_time_us":1632,"wal-append.run_wall_time_us":1060875}
I20260709 12:28:05.835057  3543 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:05.836514 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:05.838122  3629 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.665671 26199 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:28:05.831784  3632 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.840969  3612 consensus_queue.cc:237] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:05.842914  3565 consensus_meta.cc:287] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.000s
I20260709 12:28:05.843385  3565 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.847611 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:05.671262 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4469) took 1858 ms. Trace:
I20260709 12:28:05.849433 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:05.652699  3752 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.407s	user 0.001s	sys 0.000s
W20260709 12:28:05.821688 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 196000us
I20260709 12:28:05.652058  3149 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:05.653234  3756 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 12:28:05.658259 26123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3435) took 1714 ms (client timeout 1616 ms). Trace:
I20260709 12:28:05.640091  3524 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.822825 26791 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:05.729116 26093 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:05.861704 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 132682us
I20260709 12:28:05.823836  3584 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.824007  3640 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.727996 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1833) took 1342 ms. Trace:
I20260709 12:28:05.865176 26128 rpcz_store.cc:276] 0709 12:28:04.385551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.781401 (+395850us) service_pool.cc:224] Handling call
0709 12:28:05.558561 (+777160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.558762 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.558794 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:05.558836 (+    42us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:05.558867 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.558939 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.727769 (+168830us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54001f63d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:05.727899 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786468736}
I20260709 12:28:05.862712 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:05.826570  3743 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.819371  3535 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.867535 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:05.876042 26128 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:05.843779 26020 connection.cc:256] RPC call timeout handler was delayed by -1.2601874809997753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 12:28:05.850486 26127 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:05.611295  3660 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:05.660346 26203 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:28:05.848430 26121 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:05.858232 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:28:05.859103 26123 rpcz_store.cc:269] 0709 12:28:03.943495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.990075 (+ 46580us) service_pool.cc:224] Handling call
0709 12:28:05.657938 (+1667863us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001f54d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:05.658155 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2401382272}
I20260709 12:28:06.069156 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:06.070174 26127 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:06.070273 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:06.070840 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 212727us
I20260709 12:28:05.849244 26129 rpcz_store.cc:276] 0709 12:28:03.814865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.815777 (+   912us) service_pool.cc:224] Handling call
0709 12:28:05.671080 (+1855303us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54002cabd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:05.671201 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253216640}
I20260709 12:28:06.073031 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:05.650744  3537 raft_consensus.cc:740] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:05.838399 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:05.670987  3753 log.cc:927] Time spent T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 12:28:06.070580 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 193822us
I20260709 12:28:05.699828  3703 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.840080 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:06.086314 26199 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:05.868942  3149 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:28:05.650645  3732 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:05.840046  3632 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:05.869410  3734 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:05.653777  3604 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:05.869825  3741 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.864153  3518 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:05.875819  3579 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:05.930552  3634 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:05.875483  3578 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:06.071861 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:06.073786 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:05.954809  3708 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.135s
I20260709 12:28:06.099723 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3428) took 1123 ms. Trace:
I20260709 12:28:06.098131  3408 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.130579 26201 rpcz_store.cc:276] 0709 12:28:04.975906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.106525 (+130619us) service_pool.cc:224] Handling call
0709 12:28:05.106906 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:05.107266 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.107292 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:05.107901 (+   609us) log.cc:844] Serialized 52 byte log entry
0709 12:28:05.163497 (+ 55596us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:05.163542 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.163612 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.929712 (+766100us) raft_consensus.cc:1685] finished
0709 12:28:05.929743 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.942561 (+ 12818us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b48000b1b58. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:28:06.099651 (+157090us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2016,"raft.run_cpu_time_us":28423,"raft.run_wall_time_us":828428,"spinlock_wait_cycles":4866816,"thread_start_us":55149,"threads_started":1,"wal-append.queue_time_us":73193}
I20260709 12:28:06.108460 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:06.078702  3745 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.001s	sys 0.016s
I20260709 12:28:06.111380 26128 raft_consensus.cc:2468] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
I20260709 12:28:06.115645 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1839) took 1728 ms. Trace:
I20260709 12:28:06.141589 26125 rpcz_store.cc:276] 0709 12:28:04.386746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.886701 (+499955us) service_pool.cc:224] Handling call
0709 12:28:04.887013 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.978729 (+ 91716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.115387 (+1136658us) spinlock_profiling.cc:243] Waited 90.9 ms on lock 0x7b54001c3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:06.115571 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208960384}
I20260709 12:28:06.114921 26202 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: [27.22-27.22]
I20260709 12:28:06.143421 26128 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: [32.20-32.20]
I20260709 12:28:06.083880  3738 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:06.146705 26201 raft_consensus.cc:1240] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:28:05.869441  3524 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:06.101963  3756 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:28:06.126180 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
W20260709 12:28:06.131677 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 58139.4us
I20260709 12:28:06.100744 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:06.103854  3699 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.011s	sys 0.012s
I20260709 12:28:06.149953 26169 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:06.145691  3624 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:28:06.153156 26203 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:28:06.140326  3578 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.155745  3738 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:06.156450  3738 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:06.157904  3578 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:06.154858  3695 consensus_meta.cc:287] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.595s	user 0.012s	sys 0.003s
I20260709 12:28:06.162909  3766 raft_consensus.cc:2804] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:06.163652  3766 raft_consensus.cc:697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:06.164784  3766 consensus_queue.cc:237] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 33.16, Last appended by leader: 16, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:06.145905  3408 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:06.146353  3661 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:06.167199  3661 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:06.167621  3661 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:06.159713  3695 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.169359 26195 raft_consensus.cc:2368] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28.
I20260709 12:28:06.169480  3708 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:06.169864  3708 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:06.170558  3699 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.011s	sys 0.013s
I20260709 12:28:06.084093  3355 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:28:06.171757  3740 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:06.089900  3640 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
W20260709 12:28:06.180261 26018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.016s	sys 0.031s
W20260709 12:28:06.195205 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.198148 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 1.06951e+06us
W20260709 12:28:06.184720 26122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.017s	sys 0.032s
W20260709 12:28:06.201960 26122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.018s	sys 0.033s
W20260709 12:28:06.203089 26018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.019s	sys 0.033s
W20260709 12:28:06.203845 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 730681us
I20260709 12:28:06.204782 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:06.130311  3734 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:06.110941  3514 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:06.228380 26128 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:06.229349 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.119676  3741 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:06.140920  3745 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.001s	sys 0.017s
I20260709 12:28:06.147508  3579 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:06.230803 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:06.180763 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.239491 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:06.240155 26203 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 25.
W20260709 12:28:06.240278 26198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1637) took 1403 ms (client timeout 1724 ms). Trace:
W20260709 12:28:06.240669 26198 rpcz_store.cc:269] 0709 12:28:04.837332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.895526 (+ 58194us) service_pool.cc:224] Handling call
0709 12:28:06.240029 (+1344503us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b5400335254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:06.240174 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81790720}
W20260709 12:28:06.241106 26167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.000s	sys 0.020s
W20260709 12:28:06.241617 26167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.000s	sys 0.020s
I20260709 12:28:06.181727  3624 consensus_queue.cc:1243] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:28:06.245366 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.245952 26129 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:06.248342 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.249055 26052 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:06.252087 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.252398 26048 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:28:06.252826 26053 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:06.252362 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.253809 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.226162  3408 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:06.213596  3740 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:06.260501 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:28:06.269412 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.314170 26194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.004s	sys 0.025s
W20260709 12:28:06.318630 26194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.004s	sys 0.026s
I20260709 12:28:06.268988  3624 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:06.236224  3579 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:06.325770 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1645) took 1183 ms. Trace:
I20260709 12:28:06.261284 26045 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:06.333318 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.382522 26046 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:06.330840 26194 rpcz_store.cc:276] 0709 12:28:05.142692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.668533 (+525841us) service_pool.cc:224] Handling call
0709 12:28:05.668913 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.669141 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:05.669465 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.325266 (+655801us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:06.325346 (+    80us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:06.325418 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.325513 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.325690 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3155,"prepare.queue_time_us":371,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":3661,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":651380,"threads_started":1}
I20260709 12:28:06.335322 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.385368 26050 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:06.336717 26047 raft_consensus.cc:1275] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:06.170730  3708 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.239853  3709 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:06.349313 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:06.360735 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.390857 26054 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:06.391176 26129 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:06.174726  3537 consensus_queue.cc:260] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.379988 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.400947 26048 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:06.380445 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.244071  3764 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.380445 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:06.352733 26197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.011s	sys 0.039s
I20260709 12:28:06.231609  3696 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 12:28:06.345835  3439 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.014s	sys 0.017s
I20260709 12:28:06.238147  3378 consensus_queue.cc:1059] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 32.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.109s
I20260709 12:28:06.326974  3579 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:06.259793  3565 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:06.404932 26197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.013s	sys 0.042s
I20260709 12:28:06.404944 26053 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:06.314177  3745 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:06.410506 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.345965  3765 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:06.448537 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:06.409976  3439 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.014s	sys 0.018s
I20260709 12:28:06.397008  3709 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:06.442462  3632 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:06.451465  3708 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:06.501885 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1650) took 1066 ms. Trace:
I20260709 12:28:06.502413 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:06.502336 26195 rpcz_store.cc:276] 0709 12:28:05.435515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.442676 (+  7161us) service_pool.cc:224] Handling call
0709 12:28:06.501684 (+1059008us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54002170d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:06.501826 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597739392}
I20260709 12:28:06.503719 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:06.492985  3612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.012s	sys 0.062s
I20260709 12:28:06.441972 26194 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:28:06.443215 26046 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 22.
I20260709 12:28:06.416523 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.443804 26203 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:06.391866  3660 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:06.445340  3537 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:06.382835  3548 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.067s
I20260709 12:28:06.444262 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:28:06.474107  3763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.031s	sys 0.041s
I20260709 12:28:06.635231 26053 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:06.636853 26168 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.026s
W20260709 12:28:06.637184 26168 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.004s	sys 0.026s
W20260709 12:28:06.637451 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 569072us
W20260709 12:28:06.640322 26020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.021s	sys 0.022s
W20260709 12:28:06.566455  3612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.013s	sys 0.062s
I20260709 12:28:06.451628  3745 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:06.640758 26020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.021s	sys 0.022s
I20260709 12:28:06.657227 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:06.641095 26019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.012s	sys 0.019s
W20260709 12:28:06.658361 26019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.012s	sys 0.019s
I20260709 12:28:06.422457 26048 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:06.648140 26053 raft_consensus.cc:2410] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:28:06.628217 26052 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:06.654227 26054 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:28:06.658361 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.668428 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.670516 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.671145 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:06.507485  3260 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:06.677976 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:06.680541 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.684921 26052 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:06.681115 26048 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:06.490861  3565 consensus_queue.cc:1059] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.158s
I20260709 12:28:06.681056 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:06.641021  3763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.031s	sys 0.041s
I20260709 12:28:06.693799 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.441326  3734 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.700160 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:06.623441  3728 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:06.700702 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4480) took 1456 ms. Trace:
I20260709 12:28:06.702363 26053 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:06.704959 26048 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 21.
I20260709 12:28:06.711792 26126 rpcz_store.cc:276] 0709 12:28:05.244874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.245126 (+   252us) service_pool.cc:224] Handling call
0709 12:28:05.257444 (+ 12318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.328582 (+ 71138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.328632 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:05.328686 (+    54us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:05.328737 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.700446 (+1371709us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b540013ef54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:06.700493 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.700637 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163003648}
I20260709 12:28:06.707468 26052 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:06.457805  3765 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.706974 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:06.722383 26202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.016s	sys 0.038s
W20260709 12:28:06.722805 26202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.016s	sys 0.038s
W20260709 12:28:06.720065 26123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.015s	sys 0.044s
I20260709 12:28:06.666471  3696 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:28:06.723691 26123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.016s	sys 0.044s
I20260709 12:28:06.702966  3260 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:06.675697  3524 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:06.728214 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:06.732430 26046 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:06.733376 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.675817  3764 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.735705 26755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.003s	sys 0.051s
I20260709 12:28:06.683174  3640 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.711143  3565 consensus_queue.cc:1243] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 30.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260709 12:28:06.701536  3772 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:28:06.730136 26790 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1625) took 2341 ms (client timeout 1717 ms). Trace:
W20260709 12:28:06.748349 26790 rpcz_store.cc:269] 0709 12:28:04.388889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.389057 (+   168us) service_pool.cc:224] Handling call
0709 12:28:06.725244 (+2336187us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54005cdad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:06.730027 (+  4783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460703616}
I20260709 12:28:06.710501  3514 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:06.739512 26755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.004s	sys 0.051s
I20260709 12:28:06.741640 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.729193  3632 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.736654  3741 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:06.743532 26757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.011s	sys 0.031s
I20260709 12:28:06.723954  3439 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.026s
W20260709 12:28:06.746642 26201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.045s	sys 0.101s
I20260709 12:28:06.728375  3696 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:06.772256 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.771958 26756 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.022s	sys 0.009s
W20260709 12:28:06.773902 26756 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.022s	sys 0.009s
W20260709 12:28:06.768750 26757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.011s	sys 0.031s
W20260709 12:28:06.774565 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 542548us
I20260709 12:28:06.763715 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.738214  3767 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:06.700165  3535 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.013s	sys 0.023s
W20260709 12:28:06.742648 26199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.021s	sys 0.039s
W20260709 12:28:06.833799 26199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.021s	sys 0.039s
I20260709 12:28:06.771055 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4489) took 1478 ms. Trace:
W20260709 12:28:06.775717 26201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.046s	sys 0.101s
I20260709 12:28:06.703313  3709 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:06.788718 26046 raft_consensus.cc:2468] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:06.834760 26122 rpcz_store.cc:276] 0709 12:28:05.292616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.428712 (+136096us) service_pool.cc:224] Handling call
0709 12:28:05.429054 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.643280 (+214226us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:05.643646 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.202688 (+559042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:06.202763 (+    75us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:06.202832 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.770791 (+567959us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540036abd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:06.770842 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.770993 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":685,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":4503,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6860,"spinlock_wait_cycles":491987200,"thread_start_us":558614,"threads_started":1,"wal-append.queue_time_us":762243}
I20260709 12:28:06.846050 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:06.847162 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:06.847659 26122 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:06.839076 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:06.849180 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.839530  3699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.010s	sys 0.034s
I20260709 12:28:06.840262 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:06.841732 26790 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:06.764519  3763 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:06.739099  3378 consensus_queue.cc:1048] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 12:28:06.774681 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 545419us
W20260709 12:28:06.841817 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 163958us
W20260709 12:28:06.855644 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92048us
I20260709 12:28:06.766155  3640 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:06.834733  3770 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.443s	user 0.000s	sys 0.001s
I20260709 12:28:06.862498 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.782363  3696 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:06.859472  3699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.010s	sys 0.034s
I20260709 12:28:06.724373  3728 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:06.742093  3764 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:06.867131 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.870010 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3437) took 1044 ms. Trace:
W20260709 12:28:06.860551  3777 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
I20260709 12:28:06.873158 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.879381  3783 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:06.881054 26125 raft_consensus.cc:1275] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
W20260709 12:28:06.843200  3535 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.013s	sys 0.023s
W20260709 12:28:06.881631 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.882704 26020 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:06.885046 26122 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:06.888984 26201 rpcz_store.cc:276] 0709 12:28:05.825789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.147565 (+321776us) service_pool.cc:224] Handling call
0709 12:28:06.154136 (+  6571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:06.160517 (+  6381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:06.160555 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:06.161119 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.835400 (+674281us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:06.835450 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.835522 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:06.869643 (+ 34121us) raft_consensus.cc:1685] finished
0709 12:28:06.869682 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.869918 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5897,"raft.queue_time_us":7164,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":673953,"threads_started":1,"wal-append.queue_time_us":676071}
W20260709 12:28:06.843856  3685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.011s	sys 0.020s
W20260709 12:28:06.865521  3771 log.cc:927] Time spent T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
I20260709 12:28:06.765743  3772 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:06.931162  3685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.012s	sys 0.021s
I20260709 12:28:06.861505  3612 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.905114 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:06.907271 26122 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:28:06.968814 26125 raft_consensus.cc:1275] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:06.968986 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:06.899556 26168 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:06.918383 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:06.959733 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 272848us
I20260709 12:28:06.895056  3709 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:06.975435 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:06.969710 26198 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:06.900738  3797 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.880339  3791 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:06.886487  3782 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:06.905566  3624 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.900769  3784 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:06.912508  3660 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:28:06.979837 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 80390.5us
I20260709 12:28:06.988664 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:06.882673  3640 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:06.981879  3777 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:28:07.004287  3612 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:07.004757 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:06.979799  3632 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:07.017661 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:07.028707  3738 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.872s	user 0.006s	sys 0.006s
I20260709 12:28:07.029044  3764 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:07.029316  3738 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.030182  3805 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:07.030516 26200 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:07.031117  3805 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:07.031512  3805 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:07.031548 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:06.995489  3791 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.033025 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.032629  3805 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.034054  3805 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:07.034297 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.034757  3791 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:07.034853  3728 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.035539 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:07.018429 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 146865us
I20260709 12:28:06.980921  3579 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:07.021989 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45529.2us
I20260709 12:28:07.027024 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:07.065026  3778 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:28:07.038194  3738 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:07.178046  3703 consensus_meta.cc:287] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.078s	user 0.012s	sys 0.000s
W20260709 12:28:07.181272  3786 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:28:07.181742  3767 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:07.183111 26202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3435) took 1357 ms. Trace:
I20260709 12:28:07.183955 26202 rpcz_store.cc:276] 0709 12:28:05.825442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.143407 (+317965us) service_pool.cc:224] Handling call
0709 12:28:06.143776 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:06.144009 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:06.144035 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:06.144600 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.723238 (+578638us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:06.723282 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.723356 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:07.182803 (+459447us) raft_consensus.cc:1685] finished
0709 12:28:07.182826 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.183028 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":578197,"threads_started":1,"wal-append.queue_time_us":697619}
W20260709 12:28:07.184576 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 279533us
I20260709 12:28:07.188762 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:07.190163 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:06.989415  3797 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:07.040807 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:07.199399 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 158702us
I20260709 12:28:07.200054 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:07.194427 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.200678 26168 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:07.204171 26195 raft_consensus.cc:2393] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:07.195940 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:28:07.075292  3789 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:28:06.998171  3660 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.200974 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.196728 26757 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:06.989629  3808 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.038261  3772 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.065083  3632 raft_consensus.cc:697] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:07.206750 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:07.208534 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.233342 26202 raft_consensus.cc:2410] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 29 because replica has last-logged OpId of term: 28 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 13.
I20260709 12:28:07.235090 26127 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:28:07.238610 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:07.125735  3543 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.290s	user 0.007s	sys 0.010s
I20260709 12:28:07.242674 26045 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:07.208287  3629 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:07.085983  3800 log.cc:927] Time spent T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.189s	user 0.002s	sys 0.000s
I20260709 12:28:07.210567 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:07.106217  3548 raft_consensus.cc:3055] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:07.064857  3378 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:07.199689 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:07.073427  3791 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.044584  3798 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:07.025038  3745 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.258047 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.073451  3260 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:07.193270  3703 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:07.199409  3521 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.222587 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.196632 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1638) took 2359 ms. Trace:
W20260709 12:28:07.085811  3779 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
I20260709 12:28:07.248912 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:07.242825  3772 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.251246 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:07.217674  3660 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:07.198414  3624 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:28:07.264986 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4494) took 1092 ms. Trace:
I20260709 12:28:07.232434  3590 raft_consensus.cc:2804] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:07.218624  3640 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:07.007311  3784 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.202670  3578 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.040s
W20260709 12:28:07.282583 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 49300.7us
I20260709 12:28:07.284404 26201 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:07.272457 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.284775 26196 rpcz_store.cc:276] 0709 12:28:04.837646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.094560 (+256914us) service_pool.cc:224] Handling call
0709 12:28:05.094879 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.095090 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:28:05.182024 (+ 86934us) log.cc:844] Serialized 14 byte log entry
0709 12:28:05.510508 (+328484us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b1800203da0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a
0709 12:28:05.546401 (+ 35893us) log.cc:844] Serialized 14 byte log entry
0709 12:28:07.107342 (+1560941us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b1800203da0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:28:07.195754 (+ 88412us) log.cc:844] Serialized 14 byte log entry
0709 12:28:07.196276 (+   522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:07.196331 (+    55us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:28:07.196378 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.196450 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.196570 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":261938,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":129,"raft.queue_time_us":391959,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":173,"spinlock_wait_cycles":92467712,"thread_start_us":353105,"threads_started":3,"wal-append.queue_time_us":237729}
I20260709 12:28:07.250810 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:07.289209 26045 raft_consensus.cc:2376] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28. Re-sending same reply.
W20260709 12:28:07.263651  3661 consensus_meta.cc:287] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 1.096s	user 0.005s	sys 0.009s
I20260709 12:28:07.292740 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:07.254040  3629 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.252149  3810 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.321354 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.296157 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:07.296712 26123 rpcz_store.cc:276] 0709 12:28:06.172834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.181004 (+  8170us) service_pool.cc:224] Handling call
0709 12:28:06.181516 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:06.182057 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:06.182104 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:06.182753 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.725458 (+542705us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:06.725499 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.725577 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:07.264770 (+539193us) raft_consensus.cc:1685] finished
0709 12:28:07.264795 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.264929 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17192,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":17483,"thread_start_us":542359,"threads_started":1,"wal-append.queue_time_us":657570}
I20260709 12:28:07.254609  3685 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:07.260146  3790 raft_consensus.cc:2804] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:07.255023  3804 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:07.263541  3378 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.257448  3439 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:28:07.260048  3548 raft_consensus.cc:740] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:07.258893  3817 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:07.291373  3260 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.291113  3543 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.252133  3514 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:07.327346 26202 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:28:07.321776 26048 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:07.329197 26050 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:07.323726 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.360458 26201 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:07.361656 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:07.320174 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:07.336062 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:07.324391  3590 raft_consensus.cc:697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 33 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:07.294927  3591 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.034s
I20260709 12:28:07.332016  3640 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.303699  3765 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.291605  3521 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.329459  3629 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:07.346206  3782 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:07.335489  3685 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:07.288971  3791 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.356482  3817 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:07.384905 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48955.9us
I20260709 12:28:07.349104  3378 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:07.344516  3761 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:28:07.364408 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44319.4us
I20260709 12:28:07.349629  3610 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:07.401934 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 202305us
I20260709 12:28:07.353377  3816 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:07.371054 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 160601us
I20260709 12:28:07.409446 26202 raft_consensus.cc:1240] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:07.365989  3745 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.418926 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:07.335410  3632 consensus_queue.cc:237] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.414742 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3089) took 2301 ms. Trace:
W20260709 12:28:07.409739 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.401084  3685 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:07.428160 26197 rpcz_store.cc:276] 0709 12:28:05.113220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.615198 (+501978us) service_pool.cc:224] Handling call
0709 12:28:05.798089 (+182891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:05.798509 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.798540 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:05.799029 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.442722 (+643693us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:06.495824 (+ 53102us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.496448 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.496601 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:07.414522 (+917921us) raft_consensus.cc:1685] finished
0709 12:28:07.414545 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.414676 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124479,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":11175,"raft.queue_time_us":2671,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":7183,"thread_start_us":643305,"threads_started":1,"wal-append.queue_time_us":927269}
I20260709 12:28:07.436014  3832 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:07.435295 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:07.410444  3817 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:07.373806  3543 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.441097 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:07.441972  3543 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.370498  3260 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:07.442420  3818 raft_consensus.cc:2764] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:28:07.443406  3818 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:07.374601  3514 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.398506  3827 consensus_queue.cc:1048] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:28:07.446982 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:07.447114 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:07.448735  3610 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:07.449664 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.458880 26050 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:07.468670 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:07.470194 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:07.471099 26051 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:07.470624 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:07.472373 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.473201 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:07.474189 26197 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:28:07.474725 26051 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
W20260709 12:28:07.480049 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 57713.1us
I20260709 12:28:07.482851 26050 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:07.477478  3610 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.462675  3578 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:07.458272  3790 raft_consensus.cc:697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:07.336107  3810 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.327898  3661 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.519516  3810 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.519650  3661 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.421952  3816 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.420495  3548 consensus_queue.cc:260] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.490008  3610 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:07.599372 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.573451  3591 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
W20260709 12:28:07.636628  3819 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
W20260709 12:28:07.637033  3537 consensus_meta.cc:287] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.998s	user 0.012s	sys 0.000s
I20260709 12:28:07.477842  3590 consensus_queue.cc:237] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:07.638857  3821 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:07.639403  3821 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:07.639788  3821 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.640944  3548 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:07.642302  3831 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:07.642822  3831 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:07.643204  3831 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:07.649829  3514 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:07.649982  3825 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 12:28:07.631652  3812 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.324s	user 0.000s	sys 0.002s
I20260709 12:28:07.575879  3578 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:07.647861  3835 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.665985  3790 consensus_queue.cc:237] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.657172  3591 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.013s
W20260709 12:28:07.670212  3708 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 1.163s	user 0.012s	sys 0.001s
W20260709 12:28:07.675917  3709 consensus_meta.cc:287] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.001s
I20260709 12:28:07.663717  3537 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
W20260709 12:28:07.677738  3828 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 12:28:07.678777  3836 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:28:07.682121  3798 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:07.735555 26048 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:07.681797  3610 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:07.687036  3696 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.003s
I20260709 12:28:07.687201  3548 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.694410  3660 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:07.744067 26050 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:28:07.695585  3824 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
W20260709 12:28:07.700035 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 208798us
W20260709 12:28:07.742619 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 489221us
W20260709 12:28:07.939505  3378 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
I20260709 12:28:07.745707  3548 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.747504 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:08.223888  3837 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:08.224051 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 476579us
W20260709 12:28:07.809244  3769 log.cc:927] Time spent T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.578s	user 0.007s	sys 0.005s
W20260709 12:28:07.803193  3820 log.cc:927] Time spent T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 12:28:07.798825 26197 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 33 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:07.805706 26203 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:28:07.802356  3740 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:07.889158  3817 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:07.818430  3816 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:07.803311  3603 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:08.236739  3603 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:08.222831  3378 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:08.238029  3817 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:08.227072 26756 connection.cc:256] RPC call timeout handler was delayed by -1.7687085399998068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 12:28:07.738613  3734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.031s
W20260709 12:28:07.927925  3805 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.892s	user 0.013s	sys 0.000s
W20260709 12:28:08.201974  3629 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.800s	user 0.009s	sys 0.003s
W20260709 12:28:08.206313  3821 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.004s
I20260709 12:28:07.805339  3660 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:07.888114  3618 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.955814  3695 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:07.799443  3772 consensus_meta.cc:287] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 12:28:07.799062  3810 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.774886  3827 consensus_queue.cc:1059] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.466s
I20260709 12:28:08.228974  3791 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:08.240515  3734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.034s	sys 0.058s
I20260709 12:28:07.732298  3835 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.239971  3848 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:07.743345  3696 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.252480  3772 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:08.248353  3740 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:08.248577  3821 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.262357  3821 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:08.257305  3848 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:08.003904 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.296665 26018 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.591s (SENT)
W20260709 12:28:08.316551 26052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.015s	sys 0.029s
W20260709 12:28:08.317399 26052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.016s	sys 0.029s
W20260709 12:28:08.318145 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.319409 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4412) took 1069 ms. Trace:
I20260709 12:28:08.029747 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:07.816582 26046 raft_consensus.cc:3055] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:08.332471 26046 raft_consensus.cc:740] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:08.334141 26046 consensus_queue.cc:260] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 33.16, Last appended by leader: 16, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:08.335196 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4393) took 1942 ms. Trace:
I20260709 12:28:08.337920 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4380) took 2087 ms. Trace:
I20260709 12:28:08.335668 26047 rpcz_store.cc:276] 0709 12:28:06.392395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.392557 (+   162us) service_pool.cc:224] Handling call
0709 12:28:06.423504 (+ 30947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.423776 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:06.844175 (+420399us) log.cc:844] Serialized 14 byte log entry
0709 12:28:08.334214 (+1490039us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b18002dbbc0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b73f4d4bc 00007f5b7129b48b 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287
0709 12:28:08.334471 (+   257us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:08.334580 (+   109us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:08.334677 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.334793 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.335034 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4323,"prepare.queue_time_us":416019,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":77909,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3197,"spinlock_wait_cycles":81284480,"thread_start_us":447969,"threads_started":3,"wal-append.queue_time_us":46139}
I20260709 12:28:08.338362 26049 rpcz_store.cc:276] 0709 12:28:06.250334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.250493 (+   159us) service_pool.cc:224] Handling call
0709 12:28:06.250840 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.325571 (+ 74731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:06.325604 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:06.325647 (+    43us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:06.325688 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.337382 (+2011694us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b5400030054. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:08.337448 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.337749 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171379584}
I20260709 12:28:08.341408 26053 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:08.343451 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.320010 26052 rpcz_store.cc:276] 0709 12:28:07.249627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.249756 (+   129us) service_pool.cc:224] Handling call
0709 12:28:07.250099 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.746213 (+496114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.746261 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.798206 (+ 51945us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.318268 (+520062us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:08.318373 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.318539 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.318631 (+    92us) raft_consensus.cc:1685] finished
0709 12:28:08.318645 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.318985 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78142,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9236,"thread_start_us":1014947,"threads_started":2,"wal-append.queue_time_us":455385}
I20260709 12:28:08.346858 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.347839 26052 raft_consensus.cc:2376] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26. Re-sending same reply.
I20260709 12:28:08.347957 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4487) took 3064 ms. Trace:
I20260709 12:28:08.349200 26120 rpcz_store.cc:276] 0709 12:28:05.283818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.323868 (+ 40050us) service_pool.cc:224] Handling call
0709 12:28:05.324158 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.449185 (+125027us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:05.511077 (+ 61892us) log.cc:844] Serialized 14 byte log entry
0709 12:28:05.710077 (+199000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:05.710167 (+    90us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:05.710242 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.808479 (+2098237us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:28:08.347327 (+538848us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540006fed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:08.347401 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.347737 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24397,"prepare.queue_time_us":64701,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":5571,"raft.queue_time_us":4463,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":294387840,"thread_start_us":197416,"threads_started":1,"wal-append.queue_time_us":628261}
I20260709 12:28:08.348585 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.353906 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.355139 26049 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:08.356525 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:08.357622 26120 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:08.359239 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:08.360841 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:08.351892 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3445) took 2499 ms. Trace:
I20260709 12:28:08.363349 26121 rpcz_store.cc:276] 0709 12:28:05.851852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.108081 (+256229us) service_pool.cc:224] Handling call
0709 12:28:08.351428 (+2243347us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:08.351650 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5036970752}
I20260709 12:28:08.365597 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:08.366937 26121 raft_consensus.cc:2393] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:08.367511 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.362231 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.362344 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.335930 26046 raft_consensus.cc:3060] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:08.370486 26053 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:08.371713 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.373435 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.374334 26048 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:08.374356 26050 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:08.375706 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.377024 26048 raft_consensus.cc:2468] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:08.378365 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.377425 25946 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 28. New cstate: current_term: 28 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:08.383740 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.384871 26047 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:08.386420 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.387637 26047 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:28:08.371721 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.386089 25946 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 24, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 24 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:08.389257 26053 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:08.389787 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:08.390403 25946 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 25, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 25 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:08.391798 26122 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:08.380393 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.380951  3855 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.382251 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:08.383472 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.820377 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:08.239940 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.237886  3548 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:08.238098  3603 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:08.395251 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.410988 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.247300  3629 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:07.799597  3578 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.415557  3835 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:08.264298  3831 consensus_meta.cc:287] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.000s
W20260709 12:28:08.220685 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 213885us
W20260709 12:28:08.255156  3685 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.794s	user 0.010s	sys 0.003s
W20260709 12:28:08.176676 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.447973 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 280) took 57409 ms (client timeout 15000 ms). Trace:
W20260709 12:28:08.239182 26756 connection.cc:256] RPC call timeout handler was delayed by -1.4194062019996636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:28:08.426111  3799 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.466588 25944 rpcz_store.cc:269] 0709 12:27:11.038781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.960354 (+14921573us) service_pool.cc:224] Handling call
0709 12:28:08.447617 (+42487263us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:54.934811 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:54.935495 (+   684us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:54.935552 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:27:54.935569 (+    17us) tablet.cc:662] Decoding operations
0709 12:27:54.936553 (+   984us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:54.936638 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:54.936672 (+    34us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:27:54.937710 (+  1038us) tablet.cc:701] Row locks acquired
0709 12:27:54.937760 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:27:54.937945 (+   185us) write_op.cc:270] Start()
0709 12:27:54.938105 (+   160us) write_op.cc:276] Timestamp: P: 1783600074937868 usec, L: 0
0709 12:27:54.938129 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:54.941401 (+  3272us) log.cc:844] Serialized 3297 byte log entry
0709 12:27:56.291342 (+1349941us) op_driver.cc:464] REPLICATION: finished
0709 12:27:56.957702 (+666360us) write_op.cc:301] APPLY: starting
0709 12:27:56.957849 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:57.346652 (+388803us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:57.346782 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:57.551670 (+204888us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:57.552017 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:27:57.552073 (+    56us) write_op.cc:312] APPLY: finished
0709 12:27:57.634484 (+ 82411us) log.cc:844] Serialized 81 byte log entry
0709 12:27:57.824871 (+190387us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:57.825533 (+   662us) write_op.cc:454] Released schema lock
0709 12:27:57.825603 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:57.826117 (+   514us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:57.826145 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12267,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":518522,"apply.run_cpu_time_us":45079,"apply.run_wall_time_us":938090,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37615,"num_ops":8,"prepare.queue_time_us":134449,"prepare.run_cpu_time_us":21958,"prepare.run_wall_time_us":432846,"raft.queue_time_us":701778,"raft.run_cpu_time_us":15926,"raft.run_wall_time_us":429837,"replication_time_us":1328250,"spinlock_wait_cycles":1408,"thread_start_us":646146,"threads_started":5,"wal-append.queue_time_us":710865,"wal-append.run_cpu_time_us":20969,"wal-append.run_wall_time_us":2826040}]]}
I20260709 12:28:08.424985 26050 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:08.427066 26120 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:08.431381 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.433512 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.254913  3810 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.251264  3610 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:08.250509  3618 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.445928  3855 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.492941  3618 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:08.493268 26018 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.717s (SENT)
I20260709 12:28:08.447877 26125 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:28:08.459218 26047 raft_consensus.cc:2393] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:08.408317 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:08.472400 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.432739 25946 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:08.475677 26756 connection.cc:256] RPC call timeout handler was delayed by -1.09650003500019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 12:28:08.475924  3831 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:08.426313 26202 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:08.476289  3798 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:08.449061  3378 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:08.501106 26049 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:08.502938 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:08.503713 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:08.494091 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:08.464473  3696 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:08.504060 26123 raft_consensus.cc:2393] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:08.515429 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.448846  3603 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:08.503934 25946 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 17 to 20, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 20 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:08.515018 26126 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 16. (term mismatch)
I20260709 12:28:08.514897 26046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4411) took 1265 ms. Trace:
I20260709 12:28:08.521014 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.509794 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3092) took 1592 ms. Trace:
I20260709 12:28:08.519280 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:08.534806 26126 pending_rounds.cc:85] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:28:08.518139 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2371) took 3488 ms. Trace:
I20260709 12:28:08.545593 26050 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:08.537569 26203 rpcz_store.cc:276] 0709 12:28:06.917422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.919090 (+  1668us) service_pool.cc:224] Handling call
0709 12:28:06.970302 (+ 51212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.229379 (+1259077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.509036 (+279657us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400302854. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:08.509558 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920550784}
I20260709 12:28:08.515672  3808 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.550961 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:08.510090  3844 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:08.559476 26126 pending_rounds.cc:107] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 29.16
I20260709 12:28:08.563604 26126 raft_consensus.cc:2889] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:08.559989  3857 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.559871  3772 consensus_queue.cc:1059] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.125s
I20260709 12:28:08.516626  3843 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.561385  3816 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:08.516393  3799 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.545796 26791 rpcz_store.cc:276] 0709 12:28:05.029237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.031739 (+  2502us) service_pool.cc:224] Handling call
0709 12:28:05.032135 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.870436 (+838301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.517484 (+2647048us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54005c1cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:08.517883 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817919360}
I20260709 12:28:08.567641 26195 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:08.568212 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:08.569868  3848 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:08.570114  3831 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:08.572332  3521 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:08.572918  3827 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.577203 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:08.577242 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:08.560058 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:08.565304 26782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38286 (request call id 2377) took 2839 ms. Trace:
I20260709 12:28:08.588358 26053 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
I20260709 12:28:08.589848 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.520017  3565 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:08.593405 26202 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:08.593940 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:08.594074 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:08.594936 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:08.595139 26789 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:28:08.596619 26127 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:08.599107 26123 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.600384 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3454) took 1071 ms. Trace:
W20260709 12:28:08.600565 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 281) took 29963 ms (client timeout 15000 ms). Trace:
W20260709 12:28:08.601065 25944 rpcz_store.cc:269] 0709 12:27:38.636868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.600295 (+29963427us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:08.600466 (+   171us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:28:08.602138 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 299) took 22734 ms (client timeout 15000 ms). Trace:
W20260709 12:28:08.602468 25944 rpcz_store.cc:269] 0709 12:27:45.878700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.601931 (+22723231us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:08.602061 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:28:08.516793  3685 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.605154 25946 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 24 to 25, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 25 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:08.606452  3685 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:08.607371 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3102) took 1635 ms. Trace:
I20260709 12:28:08.607741 26120 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:08.609088 26786 raft_consensus.cc:2393] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 26: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:08.595829  3565 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.595263 26784 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:28:08.596153 26202 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:28:08.615260 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.615253 26049 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:08.617249 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.618140 26049 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:08.619262 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.607141 25946 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:08.506032  3798 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:08.607822 26195 rpcz_store.cc:276] 0709 12:28:06.972271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.206816 (+234545us) service_pool.cc:224] Handling call
0709 12:28:07.207138 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:08.604463 (+1397325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.606951 (+  2488us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400398354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:08.607296 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3095400320}
I20260709 12:28:08.629156  3864 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.629962 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:08.630071  3864 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:08.630929 26169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.593410 26790 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:08.633345  3536 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:08.634336  3536 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.636646 26052 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:08.637869 26203 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:08.726306 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.943964 26203 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:08.600847 26201 rpcz_store.cc:276] 0709 12:28:07.487886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.799837 (+311951us) service_pool.cc:224] Handling call
0709 12:28:08.408616 (+608779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.412588 (+  3972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.412635 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.413503 (+   868us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.496843 (+ 83340us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:08.496873 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.502296 (+  5423us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.558221 (+ 55925us) raft_consensus.cc:1685] finished
0709 12:28:08.558239 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.559057 (+   818us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":638,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":330,"thread_start_us":82613,"threads_started":1,"wal-append.queue_time_us":133593}
W20260709 12:28:08.916916 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.658408 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58518.8us
I20260709 12:28:08.611747 26782 rpcz_store.cc:276] 0709 12:28:05.726014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.726212 (+   198us) service_pool.cc:224] Handling call
0709 12:28:05.726672 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.726910 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:05.727231 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.101358 (+374127us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.782415 (+681057us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b18001580e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:07.803969 (+1021554us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b18001580e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:08.525487 (+721518us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b18001580e0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:28:08.561961 (+ 36474us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:08.562031 (+    70us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:08.562163 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.562249 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.565088 (+  2839us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82475,"prepare.queue_time_us":1164,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":36411,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":9036,"spinlock_wait_cycles":32747520,"thread_start_us":401323,"threads_started":2,"wal-append.queue_time_us":667565,"wal-append.run_cpu_time_us":4016,"wal-append.run_wall_time_us":1639929}
W20260709 12:28:08.797127 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 303963us
I20260709 12:28:08.861109 26790 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
W20260709 12:28:08.742326 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 67010.8us
I20260709 12:28:08.669929  3685 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:09.014521  3521 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:08.798403 26049 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:08.883090 26050 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:09.019925 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.681706 26053 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:08.742722 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:08.809811 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.022811  3740 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.011s
I20260709 12:28:08.698143 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:08.787994 26123 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:09.040740 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.040644 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.056434  3859 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:08.835348  3565 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.057435  3859 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.862206  3798 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.058085 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.058912 26053 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:09.059211 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.059497 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 3002) took 1425 ms. Trace:
I20260709 12:28:09.060619 26050 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:09.061416 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.062005 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.063055 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 3016) took 1165 ms. Trace:
I20260709 12:28:09.063439 26050 rpcz_store.cc:276] 0709 12:28:07.897593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.061613 (+1164020us) service_pool.cc:224] Handling call
0709 12:28:09.062991 (+  1378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:08.784252  3536 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.059895 26053 rpcz_store.cc:276] 0709 12:28:07.634301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.049381 (+1415080us) service_pool.cc:224] Handling call
0709 12:28:09.059439 (+ 10058us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.064903 26121 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:09.065181 26790 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:28:09.065934 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:08.655584  3844 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.066020 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:09.067216 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:09.068493 26049 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:08.704530 25946 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 18 to 24, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 24 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:09.070029 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.953584  3799 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.053175 26789 raft_consensus.cc:2468] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:28:09.071856 26126 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.072620 26053 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:09.014493  3685 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.073637 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:09.076944 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:09.078320 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.072503  3798 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:09.076392 26047 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.069094  3864 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:08.711243  3843 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.016883 26094 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.083590 26049 raft_consensus.cc:2410] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 31 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 20.
I20260709 12:28:09.082984 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:08.703292  3808 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:08.726787  3827 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.088927 26121 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.075310  3831 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:09.091771 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4416) took 1823 ms. Trace:
I20260709 12:28:09.094566  3843 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:09.095491 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.095541 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.096078  3870 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.086s	user 0.001s	sys 0.001s
I20260709 12:28:09.081141  3816 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:09.122505  3808 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.123478  3536 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.096769  3871 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.200s	user 0.000s	sys 0.002s
I20260709 12:28:09.098925 26782 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:09.099272 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.101402  3754 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 3.701s	user 0.014s	sys 0.008s
I20260709 12:28:09.127614 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.101509 26169 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.128485 26788 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:09.128995 26200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1651) took 2927 ms. Trace:
I20260709 12:28:09.102506  3657 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.129367 26200 rpcz_store.cc:276] 0709 12:28:06.201790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.226372 (+ 24582us) service_pool.cc:224] Handling call
0709 12:28:06.237916 (+ 11544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:07.031125 (+793209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.128502 (+2097377us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400301454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.128795 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822352640}
I20260709 12:28:09.091045 25946 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 25, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 25 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:09.103454 26050 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 31 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:28:09.106474 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.108764  3831 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.104861 26045 rpcz_store.cc:276] 0709 12:28:07.250862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.321969 (+ 71107us) service_pool.cc:224] Handling call
0709 12:28:07.322334 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:07.328758 (+  6424us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:07.349362 (+ 20604us) log.cc:844] Serialized 14 byte log entry
0709 12:28:08.331912 (+982550us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b18003b0620. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:28:08.575298 (+243386us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b18003b0620. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:08.624932 (+ 49634us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:08.637225 (+ 12293us) log.cc:844] Serialized 52 byte log entry
0709 12:28:08.637410 (+   185us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:08.780659 (+143249us) log.cc:844] Serialized 14 byte log entry
0709 12:28:08.997815 (+217156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.040442 (+ 42627us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.040491 (+    49us) raft_consensus.cc:1685] finished
0709 12:28:09.040509 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.074359 (+ 33850us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19496,"prepare.queue_time_us":335277,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":288,"raft.queue_time_us":318512,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":27438,"spinlock_wait_cycles":26821632,"thread_start_us":358996,"threads_started":3,"wal-append.queue_time_us":1175904}
I20260709 12:28:09.113035 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.081580  3798 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:09.142536 26784 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
W20260709 12:28:09.115477  3788 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 2.145s	user 0.000s	sys 0.002s
I20260709 12:28:09.116461  3844 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.084627  3740 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.118551 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.124400  3536 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.118839  3797 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.119333 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.120386 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:08.713418  3857 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.123517 26124 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.107386 26020 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.127652 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.155611  3844 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:09.157521 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 93246.1us
I20260709 12:28:09.159358 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.140980 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.143677 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
W20260709 12:28:09.149475 26018 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.048s (SENT)
I20260709 12:28:09.162381 26197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3108) took 1714 ms. Trace:
I20260709 12:28:09.147581  3726 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:09.147779  3798 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:09.151435 26050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.153110 26126 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 28 because replica has last-logged OpId of term: 27 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 21.
I20260709 12:28:09.154114 26053 raft_consensus.cc:2410] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 31 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 17.
I20260709 12:28:09.164129 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.163412  3629 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.162840 26197 rpcz_store.cc:276] 0709 12:28:07.411677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.475745 (+ 64068us) service_pool.cc:224] Handling call
0709 12:28:07.476123 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:08.228200 (+752077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.125884 (+897684us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54002b3c54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.126238 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741619584}
I20260709 12:28:09.122469  3816 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.162878  3827 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.164320 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.173957 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.175145 26168 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.181322 26783 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:28:09.175794  3260 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.182686 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.185324 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1646) took 4042 ms. Trace:
I20260709 12:28:09.179774  3657 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.186332  3536 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.187523  3355 raft_consensus.cc:3055] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:09.188349 26785 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:09.191076  3657 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.176327  3543 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:09.192843  3543 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:09.187429 26199 rpcz_store.cc:276] 0709 12:28:05.143024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.098737 (+955713us) service_pool.cc:224] Handling call
0709 12:28:06.099105 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:06.102846 (+  3741us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:06.102882 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:06.103496 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:28:06.834350 (+730854us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:06.834396 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.834480 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.114483 (+2280003us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:28:09.184815 (+ 70332us) raft_consensus.cc:1685] finished
0709 12:28:09.184834 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.185125 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3296,"raft.queue_time_us":5447,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2347,"spinlock_wait_cycles":11085184,"thread_start_us":730403,"threads_started":1,"wal-append.queue_time_us":737812}
I20260709 12:28:09.194253  3543 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.188086  3355 raft_consensus.cc:740] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:09.194782  3857 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.196532 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.197338  3355 consensus_queue.cc:260] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.197922  3685 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.198117 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.203462 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.206390 26123 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
W20260709 12:28:09.178818 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 105543us
I20260709 12:28:09.208678 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.191915  3500 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.212991  3590 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:28:09.212357 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83078us
I20260709 12:28:09.213783  3590 raft_consensus.cc:697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:09.200964  3816 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.200891 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.218060  3831 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.218748  3799 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:09.218950 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 202280us
I20260709 12:28:09.152040  3740 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.216439 26050 raft_consensus.cc:2368] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28.
I20260709 12:28:09.225360 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 3019) took 1326 ms. Trace:
I20260709 12:28:09.225817 26050 rpcz_store.cc:276] 0709 12:28:07.898456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.151055 (+1252599us) service_pool.cc:224] Handling call
0709 12:28:09.225291 (+ 74236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.230183  3798 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.232151  3798 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.204810 25946 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:09.235448 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54979.7us
I20260709 12:28:09.236090 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.213215  3500 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:09.177109  3603 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:09.244467  3657 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:09.215185  3590 consensus_queue.cc:237] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.176921  3784 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.180748  3808 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.263080 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.260159 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.264938 26050 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:09.263054 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.257016 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 124564us
I20260709 12:28:09.263475 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.261070 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44840.4us
I20260709 12:28:09.261927 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.263895 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.198945  3355 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:09.259557 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.264709 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:09.260933 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.285640 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2999) took 1712 ms. Trace:
I20260709 12:28:09.286578 26047 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:09.288793 26050 rpcz_store.cc:276] 0709 12:28:07.569969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.245789 (+1675820us) service_pool.cc:224] Handling call
0709 12:28:09.264707 (+ 18918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.282040 (+ 17333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.282322 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.284006 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.290575 26126 raft_consensus.cc:2393] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:09.245627  3843 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.291730 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55715.3us
I20260709 12:28:09.294550 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.267693  3579 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:09.283545 26169 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:09.296093 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.292580 26786 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:28:09.298317  3879 log.cc:927] Time spent T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:28:09.298369 26199 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:28:09.308581 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.310299 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.247622  3378 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.314807 26790 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:09.316721 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3001) took 1678 ms. Trace:
I20260709 12:28:09.317375 26050 rpcz_store.cc:276] 0709 12:28:07.634004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.289904 (+1655900us) service_pool.cc:224] Handling call
0709 12:28:09.306237 (+ 16333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.311546 (+  5309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.311572 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.311631 (+    59us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:28:09.311686 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.311897 (+   211us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.312038 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.291877  3603 raft_consensus.cc:697] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:09.280725  3857 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.323156 26121 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:09.325281 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.327937 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44725.2us
I20260709 12:28:09.306806  3579 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:09.329890 26018 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 2.197s (SENT)
I20260709 12:28:09.330514 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.331312 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:09.331609 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3000) took 1698 ms. Trace:
I20260709 12:28:09.333173 26047 rpcz_store.cc:276] 0709 12:28:07.633618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.289535 (+1655917us) service_pool.cc:224] Handling call
0709 12:28:09.318869 (+ 29334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.325701 (+  6832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.325728 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.325772 (+    44us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:09.331362 (+  5590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.331436 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.331550 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:09.335611 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52165.4us
I20260709 12:28:09.336647  3603 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.337466  3606 raft_consensus.cc:3055] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:09.334272 26200 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
W20260709 12:28:09.340005 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 180824us
I20260709 12:28:09.311671  3843 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.337960  3606 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:09.342801 26018 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.344005 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.344877 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3005) took 1706 ms. Trace:
I20260709 12:28:09.345232 26050 rpcz_store.cc:276] 0709 12:28:07.635124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.326631 (+1691507us) service_pool.cc:224] Handling call
0709 12:28:09.341297 (+ 14666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.341428 (+   131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.341442 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.341469 (+    27us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:09.341487 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.341519 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.341597 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.347127 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.348362  3728 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:09.334192  3579 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.349102  3728 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:09.339291  3543 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:09.342491  3740 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.311623  3657 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.351878 26128 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:09.357249  3728 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.358347 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4418) took 1713 ms. Trace:
I20260709 12:28:09.358729 26050 rpcz_store.cc:276] 0709 12:28:07.644944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.353203 (+1708259us) service_pool.cc:224] Handling call
0709 12:28:09.357718 (+  4515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.357931 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.357961 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.358021 (+    60us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:09.358081 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.358142 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.358263 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:09.359900  3774 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 2.583s	user 0.001s	sys 0.002s
I20260709 12:28:09.355739 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.355311 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.351508 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:09.362007 26127 raft_consensus.cc:2393] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:09.363003 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.363742 26127 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:09.364362 26783 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.365492  3843 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.365952 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.367880 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.368966  3784 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:09.338698  3378 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.369478  3837 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:09.371577  3378 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.369491  3784 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:09.372649  3784 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.373209  3866 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.373880  3866 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.374461 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.381203 26050 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:09.382191 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3014) took 1485 ms. Trace:
I20260709 12:28:09.382694 26050 rpcz_store.cc:276] 0709 12:28:07.897021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.359692 (+1462671us) service_pool.cc:224] Handling call
0709 12:28:09.360104 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.381892 (+ 21788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.382111 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.384991  3866 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:09.388037 26129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56104 (request call id 4493) took 3530 ms (client timeout 3291 ms). Trace:
W20260709 12:28:09.388581 26129 rpcz_store.cc:269] 0709 12:28:05.857293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.072702 (+215409us) service_pool.cc:224] Handling call
0709 12:28:09.387675 (+3314973us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540013f454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:09.387831 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395782912}
I20260709 12:28:09.389850 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.391420 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.394629  3657 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.388041 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:09.391455 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47804 (request call id 3023) took 1035 ms. Trace:
I20260709 12:28:09.400076  3791 raft_consensus.cc:3055] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:28:09.401237 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60276.3us
I20260709 12:28:09.402784 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.403299 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.401418  3791 raft_consensus.cc:740] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:09.404939 26053 rpcz_store.cc:276] 0709 12:28:08.355878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.075022 (+719144us) service_pool.cc:224] Handling call
0709 12:28:09.391259 (+316237us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b540008e3d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:09.391401 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150860672}
I20260709 12:28:09.392071 26129 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:09.414649 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2997) took 1949 ms. Trace:
I20260709 12:28:09.415016 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:09.415113 26045 rpcz_store.cc:276] 0709 12:28:07.465291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.147503 (+1682212us) service_pool.cc:224] Handling call
0709 12:28:09.151382 (+  3879us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:09.210881 (+ 59499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.210914 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:09.211610 (+   696us) log.cc:844] Serialized 52 byte log entry
0709 12:28:09.287549 (+ 75939us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:09.287578 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.287625 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.414361 (+126736us) raft_consensus.cc:1685] finished
0709 12:28:09.414387 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.414561 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":202,"raft.queue_time_us":40009,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":75484,"threads_started":1,"wal-append.queue_time_us":120192}
I20260709 12:28:09.377784  3606 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.421221 26053 raft_consensus.cc:1240] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: [28.22-28.22]
I20260709 12:28:09.421763  3606 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.421980 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1382) took 1522 ms. Trace:
I20260709 12:28:09.422428 26053 rpcz_store.cc:276] 0709 12:28:07.899979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.420561 (+1520582us) service_pool.cc:224] Handling call
0709 12:28:09.420994 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.421703 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.421925 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.423964 26053 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:09.410976  3857 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.424988 26047 raft_consensus.cc:1240] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:09.428663 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.434935 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.437340  3863 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.438345 26787 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:28:09.446166 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:09.450034 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.457355 26783 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:28:09.457602 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.443794 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.416208 26791 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 26: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:09.451457 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1385) took 1152 ms. Trace:
I20260709 12:28:09.452914 26197 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:09.454697 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.458570 26053 rpcz_store.cc:276] 0709 12:28:08.299434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.423242 (+1123808us) service_pool.cc:224] Handling call
0709 12:28:09.423725 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.451189 (+ 27464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.451398 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.459340 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3009) took 1818 ms. Trace:
I20260709 12:28:09.459721 26047 rpcz_store.cc:276] 0709 12:28:07.636157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.339886 (+1703729us) service_pool.cc:224] Handling call
0709 12:28:09.405394 (+ 65508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.453770 (+ 48376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.454036 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.461812 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.463176 26789 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
W20260709 12:28:09.463968 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 168833us
I20260709 12:28:09.466512 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.466974 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.468742 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.473538 26050 raft_consensus.cc:1240] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:28:09.473858 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.474330 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1381) took 1574 ms. Trace:
I20260709 12:28:09.474720 26050 rpcz_store.cc:276] 0709 12:28:07.899661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.455543 (+1555882us) service_pool.cc:224] Handling call
0709 12:28:09.472350 (+ 16807us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.474055 (+  1705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.474270 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304}
I20260709 12:28:09.475456 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.476251 26047 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:09.476873 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1391) took 1174 ms. Trace:
I20260709 12:28:09.477182 26047 rpcz_store.cc:276] 0709 12:28:08.302625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.460460 (+1157835us) service_pool.cc:224] Handling call
0709 12:28:09.472381 (+ 11921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.476635 (+  4254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.476825 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411776}
I20260709 12:28:09.478387 26053 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:28:09.479358 26053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1386) took 1179 ms. Trace:
I20260709 12:28:09.479506 26126 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:09.480393 26050 raft_consensus.cc:1275] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:09.481033 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3024) took 1124 ms. Trace:
I20260709 12:28:09.479794 26053 rpcz_store.cc:276] 0709 12:28:08.299955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.459437 (+1159482us) service_pool.cc:224] Handling call
0709 12:28:09.472881 (+ 13444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.479077 (+  6196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.479304 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.481559 26050 rpcz_store.cc:276] 0709 12:28:08.356293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.475777 (+1119484us) service_pool.cc:224] Handling call
0709 12:28:09.476075 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.480827 (+  4752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.480983 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.482755  3624 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.483359  3624 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.486837  3808 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:09.490974 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.493845 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.495052 26129 raft_consensus.cc:2393] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:09.496790 26757 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:09.498944 26047 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:09.499583 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4421) took 1027 ms. Trace:
I20260709 12:28:09.499579 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.500188 26047 rpcz_store.cc:276] 0709 12:28:08.472416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.489182 (+1016766us) service_pool.cc:224] Handling call
0709 12:28:09.498800 (+  9618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.499407 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.499540 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:09.506673  3734 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.011s
I20260709 12:28:09.507321  3791 consensus_queue.cc:260] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.513151 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.511096 26196 raft_consensus.cc:1240] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:09.503298 26122 raft_consensus.cc:2376] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27. Re-sending same reply.
I20260709 12:28:09.522651 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.470516  3784 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:09.508212  3886 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:28:09.521186 26120 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
W20260709 12:28:09.531138 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.508672  3791 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.539415  3857 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.508447  3661 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:09.518283  3790 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:09.508628  3518 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
W20260709 12:28:09.552750  3841 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.777s	user 0.000s	sys 0.003s
I20260709 12:28:09.554641 26051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4417) took 2093 ms. Trace:
I20260709 12:28:09.555275  3859 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.555052 26051 rpcz_store.cc:276] 0709 12:28:07.461294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.475980 (+ 14686us) service_pool.cc:224] Handling call
0709 12:28:07.476402 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.476841 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.476884 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.479721 (+  2837us) log.cc:844] Serialized 26 byte log entry
0709 12:28:07.566952 (+ 87231us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:07.567002 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.567077 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.551963 (+1984886us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:28:09.554163 (+  2200us) raft_consensus.cc:1685] finished
0709 12:28:09.554186 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.554450 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6686,"spinlock_wait_cycles":242101504,"thread_start_us":86370,"threads_started":1,"wal-append.queue_time_us":179586}
I20260709 12:28:09.491974  3863 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:09.555044 26756 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.546s (SENT)
I20260709 12:28:09.557567 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.558244 26124 raft_consensus.cc:1240] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:28:09.558777  3863 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.560061  3548 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:09.560685  3784 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:08.536118 26046 rpcz_store.cc:276] 0709 12:28:07.249200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.251740 (+  2540us) service_pool.cc:224] Handling call
0709 12:28:07.263578 (+ 11838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:07.816451 (+552873us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54002b32a0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b7431cdc0 00007f5b743a71c0 00007f5b743a8231 00007f5b743a4d8f 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c
0709 12:28:08.514358 (+697907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.514427 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:08.514468 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:08.514526 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.514632 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.514836 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17179,"spinlock_wait_cycles":51436928}
I20260709 12:28:09.563656 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.544672  3790 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.565358  3790 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:09.565812 26052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47798 (request call id 4419) took 1920 ms (client timeout 1909 ms). Trace:
W20260709 12:28:09.566388 26052 rpcz_store.cc:269] 0709 12:28:07.645192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.403711 (+758519us) service_pool.cc:224] Handling call
0709 12:28:09.565497 (+1161786us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400153454. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:09.565666 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338921728}
I20260709 12:28:09.543704 26054 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:28:09.544615 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:09.545676 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.526098  3728 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:09.547966 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.540517  3661 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.568580 26054 pending_rounds.cc:85] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 16
I20260709 12:28:09.571112 26120 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:09.572561 26054 pending_rounds.cc:107] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:28:09.572926 26054 raft_consensus.cc:2889] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:09.576056 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.578358 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:09.579003 26194 raft_consensus.cc:1275] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:28:09.579275 26197 raft_consensus.cc:3060] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.579468 26127 raft_consensus.cc:2410] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 27 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:28:09.580478  3734 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:09.581555  3734 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:09.574863  3661 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:09.579073 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 88531.9us
I20260709 12:28:09.572737  3657 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.572343  3728 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:09.593196  3728 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:09.596697  3863 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:09.597268  3863 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:09.597702  3863 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.608949  3798 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:09.609603  3798 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:09.609908 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 301809us
I20260709 12:28:09.610947 26048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3017) took 1713 ms. Trace:
I20260709 12:28:09.611485 26019 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.612845 26121 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:09.613754  3799 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:09.613933 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 211689us
I20260709 12:28:09.614218 26121 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:09.582077  3734 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:09.617519 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.616273 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 2998) took 2139 ms. Trace:
I20260709 12:28:09.611964  3657 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:09.623191 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 298115us
I20260709 12:28:09.625059  3657 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:09.625795  3840 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.816s	user 0.004s	sys 0.000s
I20260709 12:28:09.621119 26049 rpcz_store.cc:276] 0709 12:28:07.476633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.161351 (+1684718us) service_pool.cc:224] Handling call
0709 12:28:09.161843 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.162398 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:09.162703 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.283283 (+120580us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.428866 (+145583us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b1800148420. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:09.608559 (+179693us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b1800148420. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b71232c0a 00007f5b7a972219 00007f5b712baa40 00007f5b712b54e4 00007f5b712b5455 00007f5b7439d060 00007f5b743af716 00007f5b743c0287 00007f5b743c0208
0709 12:28:09.608747 (+   188us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.609316 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.609505 (+   189us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:09.609547 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.609630 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.615888 (+  6258us) raft_consensus.cc:1685] finished
0709 12:28:09.615917 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.616174 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":325556,"prepare.queue_time_us":3710,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":1824,"raft.queue_time_us":328446,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":11441,"spinlock_wait_cycles":8837632,"thread_start_us":115172,"threads_started":1,"wal-append.queue_time_us":182012}
I20260709 12:28:09.636122 26121 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:09.610014  3798 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:09.611362 26048 rpcz_store.cc:276] 0709 12:28:07.897889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.389170 (+1491281us) service_pool.cc:224] Handling call
0709 12:28:09.389585 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.610537 (+220952us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:09.610577 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.610635 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:09.610673 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.610736 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.610864 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":220544,"raft.queue_time_us":221765,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76}
I20260709 12:28:09.619947 26093 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.655773 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.638439  3500 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:09.685014 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.657835  3831 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:28:09.686869  3783 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:09.687284 26203 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
W20260709 12:28:09.688797 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 71365.1us
W20260709 12:28:09.694481 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52117.5us
W20260709 12:28:09.698563 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 72439.2us
I20260709 12:28:09.688212 26756 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.694777 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:09.690287 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 67401.5us
I20260709 12:28:09.695065 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.703709 26197 raft_consensus.cc:2468] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:09.704910 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:09.713598 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.717555 26197 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:28:09.718880 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 127507us
I20260709 12:28:09.693457  3590 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:09.695897  3783 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:09.740535 26195 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:09.740782  3866 consensus_queue.cc:1048] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:09.713243  3813 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:09.723887  3761 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:09.742769  3579 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.727591  3784 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:09.743836  3784 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:09.744275  3784 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.727859  3696 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.728430 26093 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:09.728967 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70266.8us
I20260709 12:28:09.729244 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.729530  3859 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:09.731833 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.733984  3805 consensus_queue.cc:1059] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:09.735416  3548 consensus_queue.cc:1048] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:09.739162  3578 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:09.740985 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.722748 26018 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:09.755220 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 143813us
I20260709 12:28:09.758529 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.765426 26093 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:09.797273 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:09.804863 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63964.8us
I20260709 12:28:09.805625 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:09.797923 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 80419.6us
W20260709 12:28:09.812961 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54555us
I20260709 12:28:09.798574 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:09.805251 26054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4401) took 3126 ms. Trace:
W20260709 12:28:09.821781 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52346.1us
I20260709 12:28:09.823218 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.827996 26054 rpcz_store.cc:276] 0709 12:28:06.679379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.679576 (+   197us) service_pool.cc:224] Handling call
0709 12:28:06.680048 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.583108 (+2903060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.804930 (+221822us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54000f8f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.805177 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6461480192}
I20260709 12:28:09.760653  3859 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:09.806208  3896 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:28:09.838620 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.766374  3813 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:09.805030  3827 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:09.802827  3732 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:09.803829  3579 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.808521  3660 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.771605  3797 consensus_queue.cc:1048] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.069s
I20260709 12:28:09.799675  3904 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:09.861892 26201 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:28:09.841266  3859 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:09.867995 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:09.741104  3783 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.874572 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.876430 26046 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:09.846763  3813 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.851568  3500 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.852368  3827 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.851336  3803 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:09.869438 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.878360 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.889737 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.852183  3732 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.756450  3734 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:09.895885 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.901410 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.884894 26052 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:28:09.917771 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.918596  3657 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:09.882977  3728 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.919497  3810 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.920686  3728 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.921530 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:09.921764 26122 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:09.922657  3612 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:09.923561 26122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4497) took 1364 ms. Trace:
I20260709 12:28:09.924572 26790 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:09.924624 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.925856 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 81964.8us
I20260709 12:28:09.925994 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:09.927887  3901 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:28:09.892318  3813 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.906383  3827 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.940210 26203 raft_consensus.cc:1240] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: [27.22-27.22]
I20260709 12:28:09.916051  3803 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:09.944111 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:09.945207  3803 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:09.947049 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.919188  3657 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.950083  3657 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.951103  3790 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.951964 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.929075  3500 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.953276 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.953495  3790 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:09.954433  3500 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:09.954617 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.955055 26785 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:09.908427  3732 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.957398 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:09.957549  3408 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:09.958029  3408 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:09.958971  3408 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.959950  3579 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:09.960589 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.961337 26052 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:28:09.961576  3579 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:09.962895  3804 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:09.963303  3804 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:09.964341  3804 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.965590  3804 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.920755 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3446) took 2699 ms. Trace:
I20260709 12:28:09.923950 26122 rpcz_store.cc:276] 0709 12:28:08.580738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.531240 (+950502us) service_pool.cc:224] Handling call
0709 12:28:09.531594 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.922231 (+390637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.923312 (+  1081us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b5400186fd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.923486 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896750848}
I20260709 12:28:09.980397 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.904871  3863 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.986503 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:09.934060 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 74677.3us
I20260709 12:28:09.934074  3782 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:09.993717  3782 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.994172  3782 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.994958 26045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46972 (request call id 1383) took 1918 ms. Trace:
I20260709 12:28:09.934805  3741 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:09.995374 26045 rpcz_store.cc:276] 0709 12:28:08.076603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.417765 (+1341162us) service_pool.cc:224] Handling call
0709 12:28:09.418284 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.619450 (+201166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.619493 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.622819 (+  3326us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.699572 (+ 76753us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:09.699614 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.699680 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.993218 (+293538us) raft_consensus.cc:1685] finished
0709 12:28:09.993257 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.994798 (+  1541us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227234,"raft.queue_time_us":219782,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":10800,"thread_start_us":76349,"threads_started":1,"wal-append.queue_time_us":130474}
I20260709 12:28:09.996457  3741 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:09.935734  3855 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:09.996866 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:09.997614 26195 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.998615 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:09.936967 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:10.000509  3791 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:10.000761 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.002907  3791 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:10.003288  3791 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.003619  3791 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.865967  3660 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.952627 26126 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 24.
I20260709 12:28:09.996893  3741 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:10.006886  3843 raft_consensus.cc:3055] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:10.007318  3843 raft_consensus.cc:740] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:10.008574 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.008355  3843 consensus_queue.cc:260] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.009451 26784 raft_consensus.cc:2393] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:09.994498 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1861) took 1520 ms. Trace:
I20260709 12:28:10.010222 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.010459 26125 rpcz_store.cc:276] 0709 12:28:08.473811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.581204 (+107393us) service_pool.cc:224] Handling call
0709 12:28:08.589092 (+  7888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.788150 (+199058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.788189 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.834738 (+ 46549us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.154998 (+320260us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:28:09.155050 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.994078 (+839028us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b54001f7cd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.994137 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.994193 (+    56us) raft_consensus.cc:1685] finished
0709 12:28:09.994207 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.994412 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38201,"raft.queue_time_us":157070,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":24533,"spinlock_wait_cycles":110413184,"thread_start_us":266098,"threads_started":1,"wal-append.queue_time_us":331441}
I20260709 12:28:10.012247 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.014225  3827 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.966428 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:09.966799  3661 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:09.967352  3813 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:09.967830  3810 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:09.919217 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:09.931892 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 126392us
I20260709 12:28:09.967274 26123 raft_consensus.cc:1275] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:09.768122  3696 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:09.973378 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.973666 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:09.932998 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:09.988184 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:09.988710 26124 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
W20260709 12:28:09.989457 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet de16ce1800b64c3c94c918aa6e9cae3d: Illegal state: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 33 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:09.979372 26196 rpcz_store.cc:276] 0709 12:28:07.221801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.334828 (+113027us) service_pool.cc:224] Handling call
0709 12:28:07.335169 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.543986 (+2208817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.920315 (+376329us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b540039a654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.920651 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5002807040}
I20260709 12:28:09.990888 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:09.992405  3408 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:09.936028 26128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1846) took 3099 ms (client timeout 2114 ms). Trace:
I20260709 12:28:10.026544 26129 raft_consensus.cc:2376] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31. Re-sending same reply.
I20260709 12:28:10.038228 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:10.043496 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.038260 26128 rpcz_store.cc:269] 0709 12:28:06.836902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.839926 (+  3024us) service_pool.cc:224] Handling call
0709 12:28:09.935800 (+3095874us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b54001f5ed4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:09.935939 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5055019776}
I20260709 12:28:10.045934 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:10.046859 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.959400 26018 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.025612 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:10.049252 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 75693.7us
I20260709 12:28:10.056840 26122 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:10.058352 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.060752 26048 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:10.048350 26784 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:28:10.067663 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.057922 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.057546 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.072271 26788 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:10.073161 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.009455  3843 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:10.100598 26128 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:10.101745 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:10.100909 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.102447 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.137540 26129 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:10.101405 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:10.138561 26123 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:10.119617 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.135983 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.067517 26020 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.136457 26786 raft_consensus.cc:1275] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:10.139334 26048 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 26.
I20260709 12:28:10.145473 26124 raft_consensus.cc:3055] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:10.146070 26124 raft_consensus.cc:740] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:10.143450 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76060.7us
I20260709 12:28:10.074165  3734 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:10.074185  3863 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.119170  3590 raft_consensus.cc:2764] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election decision vote started in defunct term 27: won
W20260709 12:28:10.154742 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 195378us
I20260709 12:28:10.154397 26020 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.118950  3624 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.112310  3500 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:10.147364  3855 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:10.146989  3657 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.169461 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.171722 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:10.175449 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 184623us
I20260709 12:28:10.175951 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.164565 26124 consensus_queue.cc:260] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.178586 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.179687 26052 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:28:10.179296 26048 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:10.181545 26124 raft_consensus.cc:3060] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:10.155858  3617 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 29.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.101s
I20260709 12:28:10.187527 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:10.170521  3855 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:10.190207 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 78185.5us
I20260709 12:28:10.191994 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.175385  3624 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.180048  3783 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:10.193691 26129 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:28:10.196964 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.192232 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:10.179817  3355 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.198554 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.199265 26051 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:28:10.203164 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.195288  3855 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:10.195135  3805 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:10.208411 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.216569 26195 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:10.216794 26169 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.219137  3709 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.197621  3783 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:10.200479  3833 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:10.227085  3810 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:10.222438 26169 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:10.233078 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.195822  3657 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.222965  3775 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.210937  3584 consensus_queue.cc:1048] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.062s
I20260709 12:28:10.287916 26124 raft_consensus.cc:1275] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 17. (term mismatch)
I20260709 12:28:10.288462 26124 pending_rounds.cc:85] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 16
I20260709 12:28:10.255018  3632 consensus_queue.cc:1059] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 32.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:28:10.255355 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:10.252504  3612 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:10.228924  3833 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.245339  3661 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.259843 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:10.291599 26048 raft_consensus.cc:2468] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 23.
I20260709 12:28:10.260283  3822 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:28:10.260351 25938 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f720b2e4986f496694c7578beef67cd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a981c9a7972c4e99a0903924c64fbb5c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:10.294956 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:10.262181 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3455) took 2504 ms. Trace:
I20260709 12:28:10.264765 26052 raft_consensus.cc:1240] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:10.203783  3624 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.267364 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:10.267638  3784 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.523s	user 0.004s	sys 0.005s
I20260709 12:28:10.251416  3660 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.270084  3610 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.270340  3696 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.272053 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.312413 26791 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:10.272436  3578 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:10.274268  3741 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.271708  3866 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:10.298897  3833 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:10.211395  3805 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:10.200513  3606 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:28:10.288872 26124 pending_rounds.cc:107] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 32.17
I20260709 12:28:10.352418 26124 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:10.289846 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:10.353288 26195 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:28:10.229347  3783 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:10.363963  3741 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:10.364423 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 163461us
I20260709 12:28:10.365520 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:10.365784  3805 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.367287  3805 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:10.296356 26169 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:10.297852 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.249399  3355 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:10.291183  3612 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.253646  3813 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.289716  3827 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.276795  3791 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.225536  3732 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.303813  3822 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:10.300796 26198 rpcz_store.cc:276] 0709 12:28:07.658395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.958177 (+299782us) service_pool.cc:224] Handling call
0709 12:28:07.958690 (+   513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:07.961370 (+  2680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.961417 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:09.749653 (+1788236us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b2c0010e698. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:28:09.750550 (+   897us) log.cc:844] Serialized 52 byte log entry
0709 12:28:09.905756 (+155206us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:09.933360 (+ 27604us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.084642 (+151282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.150640 (+ 65998us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.150698 (+    58us) raft_consensus.cc:1685] finished
0709 12:28:10.158907 (+  8209us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.163135 (+  4228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138175,"prepare.queue_time_us":27448,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":143558,"raft.run_cpu_time_us":832,"raft.run_wall_time_us":13422,"spinlock_wait_cycles":599042816,"thread_start_us":154659,"threads_started":1,"wal-append.queue_time_us":270589}
I20260709 12:28:10.358255 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.360103  3514 raft_consensus.cc:3055] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:10.298745  3775 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:28:10.312803  3784 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.307940  3803 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:10.367311 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:28:10.376117 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.384526  3612 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:10.386693  3355 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.328845  3610 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.408444 26047 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:10.410703  3791 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.412807  3624 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:10.413686 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 121072us
W20260709 12:28:10.414422 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56246.8us
I20260709 12:28:10.360356  3314 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:10.421826 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 125624us
W20260709 12:28:10.438066 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.405294  3632 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:10.428377 26789 raft_consensus.cc:2368] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30.
I20260709 12:28:10.443413 26127 raft_consensus.cc:2368] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30.
I20260709 12:28:10.428138 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.385690  3741 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:28:10.433045 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.387023  3696 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:10.424268 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 156979us
I20260709 12:28:10.413748  3822 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:10.447623 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.419492  3904 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.449404 26788 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:10.415045  3514 raft_consensus.cc:740] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:10.430233  3661 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:10.429533  3803 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.447194 26019 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:10.437253  3761 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.451009 26756 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.460901  3514 consensus_queue.cc:260] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.462196  3355 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.463732 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:28:10.465806 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.466794 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.469604  3761 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.448633  3741 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:10.481046 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.450546  3632 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:10.452766  3863 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:10.487063 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.487231  3632 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.487396  3863 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:10.488831  3798 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.836s	user 0.014s	sys 0.000s
I20260709 12:28:10.489380  3798 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.483601 26050 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:28:10.454536  3813 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:10.492007 26046 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:10.467368  3803 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:10.518775 26047 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:10.460901  3610 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.472811  3761 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.482537  3741 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:10.522917 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.450068  3660 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:28:10.479848  3500 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:10.485105  3696 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.526152 26053 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:10.450388  3314 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.502064  3578 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:28:10.535555 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 168392us
I20260709 12:28:10.479082  3661 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.533982 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.551079 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.520360  3798 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.529310  3514 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:10.537364  3930 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.533589  3578 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:10.533558  3500 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:10.567368  3804 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.000s
I20260709 12:28:10.576777 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:10.578577 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 112072us
I20260709 12:28:10.561759  3606 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.581612 26168 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.567163  3657 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.570448  3584 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:10.574497  3578 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:10.587211 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50628.1us
I20260709 12:28:10.581584 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:10.594285 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.594717  3922 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:28:10.596735  3660 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:10.600466  3661 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.586400  3632 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:10.609470  3859 consensus_meta.cc:287] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.003s
I20260709 12:28:10.610105  3859 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:10.611527 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 77656.2us
I20260709 12:28:10.592177  3798 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:10.613052  3798 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.613368 26125 raft_consensus.cc:1240] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:10.613466  3798 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:10.593215  3657 raft_consensus.cc:2804] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:10.590340  3606 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.616915  3657 raft_consensus.cc:697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:10.618034  3657 consensus_queue.cc:237] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.618590  3606 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.618381  3632 consensus_queue.cc:1059] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 32.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:10.620072  3632 raft_consensus.cc:3055] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:10.620798  3632 raft_consensus.cc:740] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:28:10.618050 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.622320  3632 consensus_queue.cc:260] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.624316 26122 raft_consensus.cc:1275] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:28:10.624915  3921 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:28:10.626073 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.626801 26127 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:10.632104 26127 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:28:10.634919  3919 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.378s	user 0.001s	sys 0.002s
I20260709 12:28:10.634330  3632 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:10.639129 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.612397 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.583323  3732 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:10.641073 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.642683 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.577600  3775 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 12:28:10.644346  3314 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.645227 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.648033  3784 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.656929 26128 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:10.579491  3590 consensus_queue.cc:1048] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:28:10.666344 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.672196 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.673115 26789 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:10.675743 26122 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:10.677515  3859 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.678785 26094 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:10.688649 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 76348.1us
I20260709 12:28:10.689459 26756 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.689999  3904 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.690608 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:10.697271  3904 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.699587  3904 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.700627  3827 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:10.701090  3827 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:10.701483  3827 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:10.703994  3855 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.652148  3810 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Committing config change with OpId 26.19: config changed from index 12 to 19, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:10.704824  3612 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.706377 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:10.706940  3612 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:10.707028 26127 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:10.708261 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.708797  3863 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.709699 26049 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 31. Current term is 32. Ops: []
W20260709 12:28:10.703933  3734 consensus_meta.cc:287] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.006s
I20260709 12:28:10.718405 26128 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:10.720433 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.707787  3741 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:10.716132 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.714395 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 132872us
I20260709 12:28:10.709131  3610 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:10.692982  3784 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:10.727830 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.731526 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.719926 26789 raft_consensus.cc:2955] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Committing config change with OpId 26.19: config changed from index 12 to 19, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:10.738227 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:10.739138 26194 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:10.740880 26129 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 31 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:28:10.750733 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.751420 26049 raft_consensus.cc:2376] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31. Re-sending same reply.
I20260709 12:28:10.751250  3578 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.752048  3728 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:10.753324  3734 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:10.753723  3734 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:10.754621  3734 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.754832  3803 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.755209 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:10.739830  3813 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:10.756981  3813 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:10.757977  3813 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:10.760546 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 119212us
I20260709 12:28:10.761860 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.763162 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.764428  3408 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.765132  3408 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.765695  3934 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.766356  3934 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.768630 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:10.770804  3782 consensus_meta.cc:287] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.004s
I20260709 12:28:10.770905 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:10.771370  3782 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.772069  3728 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:10.772501  3728 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:10.773139  3728 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:10.774348 26051 raft_consensus.cc:2368] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31.
I20260709 12:28:10.774029  3827 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.775190 26785 raft_consensus.cc:1240] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:10.727316  3866 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.049s
I20260709 12:28:10.760522  3734 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:10.776808  3866 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 log is divergent from this leader: its last log entry 30.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:10.783433 26194 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:10.784678 26018 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:10.785490 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:10.785389 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.727588  3696 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:10.793032 26049 raft_consensus.cc:3060] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:10.791178 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.803983 26788 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:10.805226 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.806246 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.809186 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:10.811192 26051 raft_consensus.cc:2368] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32.
I20260709 12:28:10.811352 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:10.777540  3866 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:10.814568 26120 raft_consensus.cc:1240] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 28. Current term is 29. Ops: []
W20260709 12:28:10.815145 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 44371.3us
I20260709 12:28:10.814909  3822 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.815586  3634 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.816702 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.817808  3634 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.797441  3859 consensus_queue.cc:1059] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:28:10.818969 26094 leader_election.cc:400] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28.
I20260709 12:28:10.819303 26094 leader_election.cc:403] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:10.819430  3859 raft_consensus.cc:3055] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:10.819840  3859 raft_consensus.cc:740] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:10.820333  3829 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.820570 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:10.820854  3829 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.823089  3829 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.823630  3829 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28.
I20260709 12:28:10.824671 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:10.825419 26202 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:10.826172  3741 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.826347  3798 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.803349  3584 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.827450  3584 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.783710  3783 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:10.828583 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:10.829157 26199 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:10.829584  3584 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.814637  3866 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:10.830220 26048 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:10.830421  3783 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.725801  3500 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.830708  3866 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.828591 26054 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:10.827975 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:10.840312 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:10.846016 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.832047  3866 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:10.789187  3782 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:10.803773  3696 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.855247 26194 raft_consensus.cc:1240] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:10.862313 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 268374us
I20260709 12:28:10.855130 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.864130 26785 raft_consensus.cc:2393] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:10.865844 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.867508 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.869626 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:10.740533  3610 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.882073 26120 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:10.869933 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.881270 26790 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:10.845809  3514 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:10.887920 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.894016 26783 raft_consensus.cc:2468] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:10.894380 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.887861 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:10.844588  3784 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:10.846141  3579 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:10.865404  3696 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:10.905427 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:10.902038 26093 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:10.907326 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:10.909020 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:10.909821 26783 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:10.911892  3859 consensus_queue.cc:260] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.895076  3934 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.913226  3408 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:10.913829  3863 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:28:10.916118  3833 consensus_meta.cc:287] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
I20260709 12:28:10.916582 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:10.917147  3822 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.917397 26124 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:10.889550  3798 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.918134 26125 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:10.918520  3590 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:10.924332  3661 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:10.924806  3661 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.925213  3661 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:10.860188 26049 raft_consensus.cc:2468] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
W20260709 12:28:10.932715 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 147319us
I20260709 12:28:10.934355 26199 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:10.934940 26125 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:10.855571  3829 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.046s
I20260709 12:28:10.894397  3314 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.913156  3859 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:10.938922 26127 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
I20260709 12:28:10.966722 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:10.971065 26783 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
W20260709 12:28:10.974610 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.987792 26169 leader_election.cc:400] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30.
I20260709 12:28:10.988263 26169 leader_election.cc:403] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Cancelling election due to peer responding with higher term
W20260709 12:28:10.988713 26167 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.502s (SENT)
I20260709 12:28:10.932803  3624 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.993338 26124 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 29. Current term is 30. Ops: []
W20260709 12:28:10.986930 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 47076.5us
W20260709 12:28:11.037140 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.048491 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:11.054769  3514 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:28:11.049432 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62799.3us
W20260709 12:28:11.050372 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 87019.9us
I20260709 12:28:10.940070  3822 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:11.066826 26168 leader_election.cc:400] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31.
W20260709 12:28:11.054826 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.943084  3803 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:10.940277  3617 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.942703  3798 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:11.078420 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 117499us
I20260709 12:28:10.916589  3833 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.933213  3784 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.937525  3579 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:10.987402  3934 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:11.067288  3822 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:11.108906 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 54900.6us
I20260709 12:28:10.947122  3314 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.098528 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:11.198143 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:11.228189 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 123856us
W20260709 12:28:11.229089 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 119138us
I20260709 12:28:11.078522  3355 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:11.237553 26049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.247399 26049 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:11.266459 26048 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:28:11.268560 26788 raft_consensus.cc:2368] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 32. Current term is 33.
I20260709 12:28:11.071977  3844 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:28:11.109520  3548 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.275688 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:11.113144  3822 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:11.104616  3943 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 29. Reason: Vote denied by peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611) with higher term. Message: Invalid argument: T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30.
I20260709 12:28:11.293779 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.099913  3804 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.149s
I20260709 12:28:11.295753 26127 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:11.298548 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.305455 26048 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:11.094329  3798 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.291463 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.280869 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.310005 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
W20260709 12:28:11.314191  3945 log.cc:927] Time spent T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.374s	user 0.000s	sys 0.001s
I20260709 12:28:11.261463 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.238101  3827 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:11.327247 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 65860.6us
I20260709 12:28:11.321640 26120 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:11.332305 26789 consensus_meta.cc:287] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.000s
I20260709 12:28:11.246215  3578 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:11.245381  3606 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.090s
I20260709 12:28:11.331513  3943 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:11.278952  3829 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:28:11.321437  3548 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:11.336709 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.351975 26048 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
W20260709 12:28:11.339699  3952 log.cc:927] Time spent T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:28:11.341902 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:11.343271 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet d3b95127d51d4e5db31f5dbda1a935c0: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:11.112767  3934 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:11.320773  3728 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.004s
I20260709 12:28:11.363842 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:11.369477 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:11.269148  3514 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.085s
I20260709 12:28:11.345678  3314 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:11.277330  3355 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.372812  3584 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:28:11.425547 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83759.1us
I20260709 12:28:11.359572  3612 consensus_queue.cc:1059] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.043s
I20260709 12:28:11.402585  3514 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:11.441581 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:28:11.435882 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 343658us
I20260709 12:28:11.440881 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.443126 26127 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:11.420742  3584 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:11.450238  3948 log.cc:927] Time spent T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.382s	user 0.001s	sys 0.000s
I20260709 12:28:11.380286  3728 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.404970  3941 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:28:11.463191 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.465699 26195 raft_consensus.cc:2393] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 24: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:11.468250 26756 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.441948  3514 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:11.460641  3741 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:11.477775 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.481913 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.477782 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.479132 26197 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:11.438344  3695 consensus_queue.cc:1048] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.355s
I20260709 12:28:11.486730 26127 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:11.489097 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.490387 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.467080  3833 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:11.493880 26196 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.000s	sys 0.012s
W20260709 12:28:11.496865 26129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.009s	sys 0.003s
W20260709 12:28:11.497255 26129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.009s	sys 0.003s
I20260709 12:28:11.501863 26120 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 33. Current term is 34. Ops: []
W20260709 12:28:11.502278 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 124740us
W20260709 12:28:11.506137  3960 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:28:11.479256  3827 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:11.513046  3934 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:11.516404  3827 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:11.517434  3827 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.518163  3728 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:11.521136  3956 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:28:11.522639 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:11.523291  3548 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:11.486400  3810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44004 (request call id 4) took 6265 ms. Trace:
I20260709 12:28:11.529142 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:11.487841  3829 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:11.491496  3784 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:11.536010 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67844.7us
I20260709 12:28:11.535985  3829 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:11.537086  3829 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:11.537480  3949 log.cc:927] Time spent T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:28:11.538955  3959 log.cc:927] Time spent T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:28:11.535455  3355 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:11.539686  3584 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:11.516125  3866 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.001s
I20260709 12:28:11.484802  3579 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:11.532557  3734 consensus_meta.cc:287] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.755s	user 0.007s	sys 0.005s
W20260709 12:28:11.512085  3859 consensus_meta.cc:287] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.001s
I20260709 12:28:11.535002  3514 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.533784 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:11.569152 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.562096 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:11.552255 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 477267us
I20260709 12:28:11.556406 26785 raft_consensus.cc:2410] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 32 because replica has last-logged OpId of term: 31 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:28:11.556946 26169 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:11.588537 26128 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:11.565697 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.587975 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:11.590306 26198 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:11.601064 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44202.4us
I20260709 12:28:11.606858 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:11.607755 26203 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:11.620023 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:11.636060 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70441.2us
I20260709 12:28:11.576686  3798 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:11.575178  3784 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:11.562189  3863 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:11.606467  3548 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:11.600164  3855 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:11.652789 26053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.000s	sys 0.009s
W20260709 12:28:11.653823 26053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.001s	sys 0.009s
W20260709 12:28:11.618631  3661 consensus_meta.cc:287] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.680s	user 0.005s	sys 0.008s
I20260709 12:28:11.616936  3761 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:11.654556  3661 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.655021 26786 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:11.655638 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:11.605794  3355 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:11.656718  3661 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:11.656891  3355 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.657675  3859 consensus_queue.cc:1059] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:28:11.659199  3355 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:11.665256 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 227522us
I20260709 12:28:11.670189 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.670187 26203 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:11.645980  3783 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:11.672148 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:11.675792 26128 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:28:11.680645 26121 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:11.683148 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.683271 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.684000 26048 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:11.688649 26788 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:11.689883 26789 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:11.649482  3784 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.692698 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.694010 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.694077 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.681553 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.701568 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:11.699577 26196 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:11.707518 26018 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:11.708571 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.708913 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:11.709184 26047 raft_consensus.cc:3060] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:11.706192 26048 raft_consensus.cc:2468] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 34.
I20260709 12:28:11.709928 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:11.709553 26786 raft_consensus.cc:2368] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31.
W20260709 12:28:11.704371 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 73654.1us
I20260709 12:28:11.712476 26052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.712965 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:11.713361 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.712968 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.714174 26046 raft_consensus.cc:3060] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:11.714363 26125 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:11.666641  3971 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:11.715552 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.712476 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:11.716228 26018 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.724097 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.724411 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:11.674878  3783 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:11.698755  3784 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:11.728185 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:28:11.732966  3926 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:28:11.733739 26018 leader_election.cc:400] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 32. Current term is 33.
I20260709 12:28:11.734179 26018 leader_election.cc:403] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:11.736563  3579 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:11.736809 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:11.737298  3579 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:11.737906  3859 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:11.738866  3981 consensus_queue.cc:1059] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:28:11.740034  3981 raft_consensus.cc:3055] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:11.740465  3981 raft_consensus.cc:740] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:11.741428  3981 consensus_queue.cc:260] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.742971  3981 raft_consensus.cc:3060] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:11.736860  3964 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:28:11.748807  3968 log.cc:927] Time spent T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:28:11.748659  3829 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.649443  3709 consensus_queue.cc:1059] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:28:11.750468 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.751194  3829 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:11.751274  3709 raft_consensus.cc:3055] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:11.751760  3709 raft_consensus.cc:740] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:11.752285 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:11.752723  3709 consensus_queue.cc:260] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.753885  3709 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:11.756280 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.757537 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.758857 26125 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:11.738719  3579 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.660115  3863 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.762416 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:11.713380  3732 raft_consensus.cc:3055] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:11.712850  3355 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:11.781556 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.667047  3548 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.751957  3829 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:11.760371 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:11.730376  3783 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:11.765396 26047 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:11.767916 26045 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:11.726603  3855 raft_consensus.cc:2804] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:11.727931  3866 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.728446  3963 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:11.788964  3904 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:11.523923  3810 rpcz_store.cc:276] 0709 12:28:05.212968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.213108 (+   140us) service_pool.cc:224] Handling call
0709 12:28:09.122263 (+3909155us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b58005606d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743010aa 00007f5b74383447 00007f5b743a070a 00007f5b743a176a 00007f5b743a091b 00007f5b743ac03c 00007f5b743ab6b3 00007f5b743a39fa 00007f5b7936b4c9 00007f5b728e7c54
0709 12:28:09.189871 (+ 67608us) log.cc:844] Serialized 501 byte log entry
0709 12:28:09.557061 (+367190us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54002ca6d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743ab735 00007f5b743a39fa 00007f5b7936b4c9 00007f5b728e7c54 00007f5b728e7be2 00007f5b728e7b11 00007f5b728e7a8c 00007f5b728e6d42 00007f5b726e048c 00007f5b726df8c6
0709 12:28:09.937663 (+380602us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b18003328c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b83f3 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:11.475132 (+1537469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30670,"raft.queue_time_us":35918,"raft.run_cpu_time_us":709,"raft.run_wall_time_us":2180,"spinlock_wait_cycles":1145100800}
I20260709 12:28:11.797438 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.769721  3804 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:11.768429  3798 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:11.719911  3965 consensus_queue.cc:1059] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:28:11.814177 26046 raft_consensus.cc:2468] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:28:11.818351 26123 raft_consensus.cc:2468] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:11.819720 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:11.820034 26052 raft_consensus.cc:1240] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34. Ops: [29.20-33.22]
I20260709 12:28:11.769568  3732 raft_consensus.cc:740] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:11.828200 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.828722  3732 consensus_queue.cc:260] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.830657  3732 raft_consensus.cc:3060] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:11.821262 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:28:11.810642  3696 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.870s	user 0.005s	sys 0.006s
I20260709 12:28:11.794790  3803 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:11.796727  3855 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:11.839764 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.841920 26045 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:11.842876 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
W20260709 12:28:11.838877 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 112585us
I20260709 12:28:11.867246 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.868093 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:11.868146 26047 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:11.868676 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:11.823886  3355 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.869342 26756 leader_election.cc:400] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Vote denied by peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989) with higher term. Message: Invalid argument: T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32.
I20260709 12:28:11.869333 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.873709 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.824134  3963 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:11.874420 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:11.882565 26047 raft_consensus.cc:2393] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:11.885643 26045 raft_consensus.cc:2393] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 33: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:11.834303  3973 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:11.846235  3855 raft_consensus.cc:3060] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:11.823913  3904 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.889518 26756 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.891500 26045 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:28:11.819694  3866 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.839933  3844 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 32. Current term is 33.
I20260709 12:28:11.893492 26047 raft_consensus.cc:1240] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:28:11.873118  3971 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:11.884292  3355 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.900890 26047 pending_rounds.cc:85] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting all ops after (but not including) 13
I20260709 12:28:11.842468  3696 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.902793 26197 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:11.903867 26047 pending_rounds.cc:107] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 12:28:11.904383 26047 raft_consensus.cc:2889] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:11.904744  3977 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:28:11.910543  3963 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.924204 26198 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:11.923663 26756 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:11.844256  3803 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.935844 26120 raft_consensus.cc:2468] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:11.940685 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.941766 26194 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
I20260709 12:28:11.943439 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:11.923013  3990 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:11.893504  3973 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.979414 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:11.955300  3606 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.955017  3994 raft_consensus.cc:2804] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:11.980206 26054 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:11.985961  3995 raft_consensus.cc:2804] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:11.986644  3995 raft_consensus.cc:697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:11.976994  3579 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:11.964270  3987 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.962666  3972 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:11.976110  3943 consensus_meta.cc:287] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.000s
I20260709 12:28:11.989056 26046 raft_consensus.cc:3060] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:11.994256 26166 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.994468 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:11.979993  3617 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:11.979904  3863 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:11.997733 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.007640 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:12.008440 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:11.988271  3579 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.012378  3696 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:12.016104  3980 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:28:12.018059  3973 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:11.996330  3943 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:12.023495 26046 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
W20260709 12:28:12.030520 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.018510  3843 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:11.994077  3987 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.987744  3995 consensus_queue.cc:237] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.005646  3617 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:11.989742  3606 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:12.046490 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.006294  3548 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.054589 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.067991 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:12.056370  3822 consensus_meta.cc:287] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.001s
I20260709 12:28:12.056092  3610 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:11.995844  3972 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:11.979722  3971 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:12.072227 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 78987.6us
I20260709 12:28:12.071470 26122 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
W20260709 12:28:12.057351 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 1.09083e+06us
I20260709 12:28:12.070150 26168 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.073860 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.074373 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.075690 26167 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.081048 26168 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:12.065335  3866 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.072465  3943 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:12.072062  3863 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:12.081632  3966 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:12.095252 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:12.078405  3612 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:12.087891  3657 raft_consensus.cc:3055] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:12.089290  3695 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:12.072324  3989 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:28:12.077445  3822 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.098810 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:12.082674  3994 raft_consensus.cc:697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:12.086185  3904 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.094846  3943 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:12.160663 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.098083  3657 raft_consensus.cc:740] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:12.160769 26045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:12.098961  3973 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:12.096276  3355 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.182721 26045 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:12.115289  3863 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:12.116734  3997 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:12.192269 26757 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:12.188982  3632 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:12.163259  3966 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:12.196756 25939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 5)
I20260709 12:28:12.166436  3943 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:12.200266 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:12.201840  3783 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.202898  3514 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:12.204185 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:12.209072  3829 raft_consensus.cc:3055] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:12.104224  4001 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.116s
W20260709 12:28:12.236650 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 161059us
I20260709 12:28:12.236770 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.236572 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.237694 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.211719 26051 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:12.213135  3612 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:12.214790 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:12.198922  3657 consensus_queue.cc:260] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.219488 26196 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:12.223074 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.223312 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.224964 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.210855  3822 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.273000 26053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152"
I20260709 12:28:12.238699 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:12.220315  3632 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:12.283933 26788 raft_consensus.cc:2393] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:12.285712 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 70999.6us
I20260709 12:28:12.273499 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.275318 26048 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:12.220485  3548 raft_consensus.cc:2804] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:12.238123 26786 raft_consensus.cc:2468] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:28:12.279498 26783 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:12.279866 26053 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:12.278245 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 197260us
I20260709 12:28:12.274819 26782 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:12.291391 26168 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:12.234768  3829 raft_consensus.cc:740] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:12.294873 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 185352us
I20260709 12:28:12.235262  3966 raft_consensus.cc:697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:12.291792 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:12.277235  3612 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:12.302495 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:12.303649  3657 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:12.304009  3829 consensus_queue.cc:260] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.304621 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:12.305085  3829 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:12.308256  3783 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.308710  4010 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:12.296655  3696 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:28:12.309201  4010 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.309578  4010 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:12.309688 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:12.310474 26787 raft_consensus.cc:2393] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:12.301633  3548 raft_consensus.cc:697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:12.273387  3994 consensus_queue.cc:237] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.312155  3548 consensus_queue.cc:237] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.314035  3632 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.317776  3617 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.319743 26051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:12.320957 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:12.321727  3784 consensus_meta.cc:287] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.001s
I20260709 12:28:12.322044  3782 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:12.322166  3784 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.322904 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.324477  3624 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:12.324222 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.338703 26045 raft_consensus.cc:2468] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
I20260709 12:28:12.290565  3989 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:12.315842  3632 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.291316  4008 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:12.347100  4009 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:12.347810 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45264.9us
W20260709 12:28:12.342366  3775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.004s	sys 0.019s
I20260709 12:28:12.322821  3782 raft_consensus.cc:2804] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:12.371007 26053 raft_consensus.cc:2468] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:12.374310 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1660) took 1509 ms. Trace:
I20260709 12:28:12.329666  3612 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.374907 26784 rpcz_store.cc:276] 0709 12:28:10.862677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.870659 (+  7982us) service_pool.cc:224] Handling call
0709 12:28:10.903293 (+ 32634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.903578 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:10.939749 (+ 36171us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.226603 (+1286854us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b180037e4c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b7129ae07 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b 00007f5b73f44d23 00007f5b73f4523a 00007f5b743afa2a
0709 12:28:12.356262 (+129659us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.356403 (+   141us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:12.356442 (+    39us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:12.356492 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.356553 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.356663 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8207,"prepare.queue_time_us":43617,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":6156672,"thread_start_us":120936,"threads_started":1}
I20260709 12:28:12.385800 26128 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
W20260709 12:28:12.365056  3775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.004s	sys 0.019s
I20260709 12:28:12.351647  3981 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:12.364473  4008 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.364951  3782 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:12.390002  3798 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.000s
I20260709 12:28:12.364168  3696 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:28:12.442168 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.377847  3783 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:12.415218 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:12.476025 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 171445us
I20260709 12:28:12.364677  4009 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.373687  3987 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:12.482797 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 209355us
I20260709 12:28:12.509471 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.389827  3695 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:28:12.299815  3863 consensus_queue.cc:1175] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" responder_term: 28 status { last_received { term: 28 index: 18 } last_committed_idx: 18 last_received_current_leader { term: 28 index: 18 } }
I20260709 12:28:12.412442  3579 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.492434 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:12.513371 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet 40485ee0f2b44c1290c368cf529eade1: Illegal state: Replica 8fd5643e7b494fa79c3d0aaf1e6e8152 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 32 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:12.495904 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.378937  4004 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.132s
I20260709 12:28:12.479494  3612 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:12.482404  3782 raft_consensus.cc:3060] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:12.363945  3855 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.487895  3798 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:12.481245  4008 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.538957 26203 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
W20260709 12:28:12.573556 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.487869  3696 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:12.625492 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 92394.9us
W20260709 12:28:12.601110 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 244446us
I20260709 12:28:12.649537 26168 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:12.560221 26019 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:12.650512 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 90424.9us
I20260709 12:28:12.522114  3695 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:12.651206 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:12.515246  4009 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:12.656375 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.658876  3579 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:28:12.663094 26121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1893) took 1725 ms (client timeout 1999 ms). Trace:
I20260709 12:28:12.546725  3514 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:12.669402  3996 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.684s	user 0.002s	sys 0.000s
I20260709 12:28:12.669090  3696 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:12.660008  3969 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 1.101s	user 0.000s	sys 0.014s
I20260709 12:28:12.670235 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3478) took 1696 ms. Trace:
W20260709 12:28:12.671459 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.477715  3981 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:12.676294 26121 rpcz_store.cc:269] 0709 12:28:10.938009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.938125 (+   116us) service_pool.cc:224] Handling call
0709 12:28:12.662847 (+1724722us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54001f8bd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:12.662988 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371172736}
I20260709 12:28:12.699117 26127 rpcz_store.cc:276] 0709 12:28:10.974022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.493324 (+519302us) service_pool.cc:224] Handling call
0709 12:28:11.493621 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.525585 (+ 31964us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:11.545296 (+ 19711us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.550139 (+  4843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.550189 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:11.550237 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.669994 (+1119757us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54001874d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:12.670041 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.670180 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4624,"prepare.queue_time_us":25334,"raft.queue_time_us":5020,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":15740,"spinlock_wait_cycles":80336128}
W20260709 12:28:12.674357 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 293787us
I20260709 12:28:12.675530  3695 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:12.678028 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.670225  3803 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.643954  3784 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.524892  3987 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.151s
I20260709 12:28:12.550040  3783 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:12.668710  4008 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.680895  3500 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:12.682978  3579 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:12.729718 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:12.724938 26054 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: []
W20260709 12:28:12.732795  4016 log.cc:927] Time spent T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:28:12.748099 26121 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:12.721746  3973 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:12.754752 26046 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
W20260709 12:28:12.757805 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.679026  4011 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 12:28:12.529425  3966 consensus_queue.cc:237] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.774986 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:12.776897 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:12.775950 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:12.734118  4026 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.733508  3500 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:12.745901  3514 raft_consensus.cc:2804] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:12.789204 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:12.789860 26194 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:12.788053 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:12.798956 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:12.801100 26121 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
W20260709 12:28:12.803903 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet 1e8b57c1b80240d483c7ca4e95d74673: Illegal state: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:12.774045  4025 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:12.856112 26785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38282 (request call id 1662) took 1982 ms (client timeout 2216 ms). Trace:
W20260709 12:28:12.856791 26785 rpcz_store.cc:269] 0709 12:28:10.873600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.873788 (+   188us) service_pool.cc:224] Handling call
0709 12:28:12.855846 (+1982058us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400409a54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:12.856023 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519782144}
I20260709 12:28:12.856866  4025 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.858959 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
I20260709 12:28:12.863819 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:12.864102  3657 consensus_meta.cc:287] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 12:28:12.864727 26789 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:12.865406  3972 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:12.866735  4025 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.804656 26129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1888) took 1943 ms. Trace:
W20260709 12:28:12.806425  4019 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:28:12.875790 26200 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
W20260709 12:28:12.807183  4020 log.cc:927] Time spent T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:28:12.877331 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:12.880599  3732 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:12.881347  3732 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.812394  4014 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.170s
W20260709 12:28:12.813004 26120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1899) took 1291 ms (client timeout 1588 ms). Trace:
W20260709 12:28:12.904614 26120 rpcz_store.cc:269] 0709 12:28:11.521160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.522307 (+  1147us) service_pool.cc:224] Handling call
0709 12:28:12.812771 (+1290464us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x7b54001d1ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:12.812913 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91491584}
W20260709 12:28:12.813395  4017 log.cc:927] Time spent T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:28:12.816443  4008 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:12.818092  4031 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:12.818714 26784 raft_consensus.cc:2468] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:12.822315  3590 consensus_queue.cc:1048] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:12.800089  3624 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:12.789193  3500 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:12.827708  3606 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:12.828526  3579 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.829596  4029 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:12.916877  3584 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:12.840375 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:12.839109  3943 consensus_meta.cc:287] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.001s
W20260709 12:28:12.841871  3829 consensus_meta.cc:287] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.000s
I20260709 12:28:12.843268 26046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:12.940766 26046 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:12.844671 26052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3035) took 1090 ms. Trace:
I20260709 12:28:12.848788 26054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" is_pre_election: true
I20260709 12:28:12.852833  3855 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:12.848109  4010 consensus_meta.cc:287] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.538s	user 0.005s	sys 0.005s
I20260709 12:28:12.869503 26129 rpcz_store.cc:276] 0709 12:28:10.861097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.861324 (+   227us) service_pool.cc:224] Handling call
0709 12:28:10.861663 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:10.869754 (+  8091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.869785 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:10.895565 (+ 25780us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.497597 (+602032us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:11.497631 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.497690 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:12.659400 (+1161710us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:28:12.800616 (+141216us) raft_consensus.cc:1685] finished
0709 12:28:12.800641 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.804590 (+  3949us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":466,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9407,"spinlock_wait_cycles":102571904,"thread_start_us":554498,"threads_started":1,"wal-append.queue_time_us":621876}
I20260709 12:28:12.789951  4026 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:12.882807 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.983s (SENT) [suppressed 38 similar messages]
W20260709 12:28:12.891252  3632 consensus_meta.cc:287] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.004s
W20260709 12:28:12.899436  3989 consensus_meta.cc:287] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
I20260709 12:28:12.954891 26129 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:12.930713 26789 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
W20260709 12:28:12.931979 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.931339 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:12.938150 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:12.941932 26052 rpcz_store.cc:276] 0709 12:28:11.753969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.756357 (+  2388us) service_pool.cc:224] Handling call
0709 12:28:11.774967 (+ 18610us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:11.827630 (+ 52663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.844409 (+1016779us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b5400359dd4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:12.844612 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53984128}
I20260709 12:28:12.927500  4025 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.965560 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:12.951285 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:12.951367 26125 raft_consensus.cc:1275] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:28:12.967892 26125 pending_rounds.cc:85] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:28:12.968264 26125 pending_rounds.cc:107] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:28:12.968564 26125 raft_consensus.cc:2889] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:12.954964 26121 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:12.839766 26049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4450) took 1575 ms. Trace:
I20260709 12:28:12.970209 26049 rpcz_store.cc:276] 0709 12:28:11.264209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.264392 (+   183us) service_pool.cc:224] Handling call
0709 12:28:11.264768 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.265016 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.265046 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:12.482305 (+1217259us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b2c000014c8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:28:12.482736 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:28:12.786910 (+304174us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:12.786957 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.787048 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:12.839431 (+ 52383us) raft_consensus.cc:1685] finished
0709 12:28:12.839453 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.839680 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17288064,"thread_start_us":138911,"threads_started":1,"wal-append.queue_time_us":290460}
I20260709 12:28:12.934080  4032 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.958047 26120 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
W20260709 12:28:12.959149 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.926589  3782 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.976567  3855 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:12.977070 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:12.977406 26168 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:12.977597  4002 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:12.979401 26122 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 27: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:12.951943  3761 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.008s	sys 0.001s
W20260709 12:28:12.980530 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.906232  4009 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.984735 26757 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:13.016801  3829 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.017508  4009 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.017383  3829 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.986667 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.018189  4009 raft_consensus.cc:2804] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:13.018766 26129 raft_consensus.cc:2410] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:28:13.019578  3829 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.971891  3989 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:12.990671  4027 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:12.992369  3990 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:12.991989  4011 consensus_queue.cc:1048] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:12.993113  3606 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:12.994580  3855 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:13.034961  3855 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.037225  3855 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.022280  4027 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.038131  4027 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:13.021027  3761 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.366s	user 0.009s	sys 0.001s
I20260709 12:28:13.040684  3761 raft_consensus.cc:697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:13.041800  3761 consensus_queue.cc:237] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.045840  3990 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:13.047266  3981 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:12.981376  4032 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.002130  3732 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.024314  3829 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.053542  3950 log.cc:927] Time spent T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.785s	user 0.005s	sys 0.000s
I20260709 12:28:13.006544  4026 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:12.939653  4029 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.944937  3943 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:12.948496  4010 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.014690  4014 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:13.018730  4009 raft_consensus.cc:697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:12.988246 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.061182 26124 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:13.016821 26757 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:13.062287 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45538.2us
I20260709 12:28:13.025874 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.028273 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:12.974967  4025 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.028286 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.028777 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.030460 26198 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:13.031260 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:13.046893  3990 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:12.995437  3782 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.004535  3695 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.086301  3983 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.087143  3983 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.090093  3983 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:13.060146 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 177348us
I20260709 12:28:13.064744 26200 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
W20260709 12:28:13.066529 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 398475us
I20260709 12:28:13.077121 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.108280 26198 raft_consensus.cc:3060] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:13.108600 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:13.110528  3632 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.110759 26167 leader_election.cc:400] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31.
I20260709 12:28:13.111135 26167 leader_election.cc:403] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:28:13.111277  3632 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:13.111655  3863 consensus_queue.cc:1059] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:13.112861 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:13.112880  3863 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865) with higher term. Message: Invalid argument: T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for earlier term 30. Current term is 31.
I20260709 12:28:13.113935  3863 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:13.113030  3775 raft_consensus.cc:3055] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:13.125348 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:13.132139 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.058943  3829 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.153658  4035 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:13.168344  3500 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.170238  3829 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.171219  4035 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:13.171480  3500 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.172413  3863 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:13.173559  3863 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.174340 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.171813  4035 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.175767 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.176819  4035 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.168040 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.177820 26126 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:13.178722  3695 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.180163 26202 raft_consensus.cc:2393] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 27: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:13.181597 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.182219 26202 raft_consensus.cc:2468] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
W20260709 12:28:13.182577 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.183106 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.166898  4032 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.064318  3514 raft_consensus.cc:697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:28:13.144872  3999 log.cc:927] Time spent T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 1.157s	user 0.003s	sys 0.002s
I20260709 12:28:13.146879 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.148257  4030 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:13.149963  3943 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:13.144409  3981 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:13.151924  4029 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.153630  4010 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:13.153770 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.154958  4025 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.155808  4034 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:13.157145  4009 consensus_queue.cc:237] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.129761  3775 raft_consensus.cc:740] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:28:13.160354 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.161382 26203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3488) took 1486 ms. Trace:
I20260709 12:28:13.161700  4031 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:13.162045 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3489) took 1560 ms. Trace:
I20260709 12:28:13.165172 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.165802  4002 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.166986 26196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48534 (request call id 3489) took 1443 ms. Trace:
I20260709 12:28:13.075239  4026 raft_consensus.cc:2804] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:13.151634 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1687) took 1455 ms. Trace:
I20260709 12:28:13.184865  3970 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.201865  3970 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.072674  4014 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.202471 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.206485  4014 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.206967  4027 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.186669  3514 consensus_queue.cc:237] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.197288 26203 rpcz_store.cc:276] 0709 12:28:11.675341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.700631 (+ 25290us) service_pool.cc:224] Handling call
0709 12:28:13.161185 (+1460554us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54002166d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:13.161301 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890773760}
I20260709 12:28:13.199023 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.200812 26196 rpcz_store.cc:276] 0709 12:28:11.723687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.730210 (+  6523us) service_pool.cc:224] Handling call
0709 12:28:13.166827 (+1436617us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5400397954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:13.166928 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607380224}
I20260709 12:28:13.214411 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.185722  3863 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.211495 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.201406 26195 rpcz_store.cc:276] 0709 12:28:11.696024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.696456 (+   432us) service_pool.cc:224] Handling call
0709 12:28:11.696738 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.794104 (+ 97366us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:11.825796 (+ 31692us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.890620 (+ 64824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.890683 (+    63us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:11.890768 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.668876 (+778108us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:28:12.860110 (+191234us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b18003abe20. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712862a6 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419
0709 12:28:13.151284 (+291174us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b5400396f54. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:13.151352 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.151543 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30987,"prepare.queue_time_us":32985,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":53,"raft.queue_time_us":1437,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":3721,"spinlock_wait_cycles":233954176,"thread_start_us":63302,"threads_started":1,"wal-append.queue_time_us":130400}
I20260709 12:28:13.217419 26126 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:13.197907 26123 rpcz_store.cc:276] 0709 12:28:11.601544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.725020 (+123476us) service_pool.cc:224] Handling call
0709 12:28:13.161895 (+1436875us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b540010e5d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:13.161991 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126895872}
I20260709 12:28:13.228655 26124 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:13.230168 26202 raft_consensus.cc:2368] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34.
I20260709 12:28:13.230976 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.231788 26203 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:13.235651 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.236881 26196 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:13.247700 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.250886  3634 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.250635 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65229.9us
I20260709 12:28:13.253985 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:13.252676 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:13.256570 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.189877  3696 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.255311 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:13.261986 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.201309  4026 raft_consensus.cc:697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:13.268851 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.270169 26123 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:13.272439 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.273330 26202 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:28:13.266849 26203 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:13.230010  3732 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:13.271234 26198 raft_consensus.cc:2468] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:13.245322  3965 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:13.282454 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:13.285687 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.190754  3981 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:13.297811 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.299403 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.308842 26047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 3050) took 1518 ms. Trace:
I20260709 12:28:13.309481 26047 rpcz_store.cc:276] 0709 12:28:11.790910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.900517 (+109607us) service_pool.cc:224] Handling call
0709 12:28:11.900762 (+   245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.905051 (+  4289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.905108 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.905765 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.978437 (+ 72672us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:11.978480 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.978613 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.144146 (+1165533us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b73f4086e 00007f5b73f3fbb6 00007f5b73f3f7d8 00007f5b73f49221 00007f5b73f491d9 00007f5b73f49169
0709 12:28:13.286558 (+142412us) raft_consensus.cc:1685] finished
0709 12:28:13.286602 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.308755 (+ 22153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5705472,"thread_start_us":72299,"threads_started":1,"wal-append.queue_time_us":81352}
I20260709 12:28:13.268761  4014 raft_consensus.cc:2804] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:13.267791  3798 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.193904  4034 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.315981 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.342579 26200 raft_consensus.cc:2368] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31.
I20260709 12:28:13.317574 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.342701 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.318589 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:13.343994 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.318868 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.345616 26194 raft_consensus.cc:2410] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 32 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:28:13.187074  4030 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.320308 26201 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:13.321666 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:13.323643 26094 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:13.320925 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:13.322749 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:13.200555  4002 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.337657 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:13.370136 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 49328.9us
I20260709 12:28:13.340839 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:13.369401 26197 raft_consensus.cc:2393] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:13.279052  3783 raft_consensus.cc:2804] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:13.354915 26125 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:13.299983  3970 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:13.380134 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56595.6us
I20260709 12:28:13.381619 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.381927 26785 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:13.382572 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:13.374303 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 352938us
W20260709 12:28:13.385771 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64089us
I20260709 12:28:13.388797 26126 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:13.390477 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.302498  3965 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.396572  3965 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.399430 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:13.399585 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 76937.9us
I20260709 12:28:13.195591  3775 consensus_queue.cc:260] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.407436 26197 raft_consensus.cc:2410] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 30 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:28:13.263825  3634 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.315223  4031 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.405707 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:13.369457  3829 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.401098 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.323876  4014 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:13.337239  4044 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:13.384315  3783 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:13.406862 26786 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:28:13.385593  3810 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:13.339018  3822 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:13.474903 26126 raft_consensus.cc:2468] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:28:13.339218  3813 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:13.285668  3732 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.445789 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.493212 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.413642 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:13.385421  3617 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:13.458225  3634 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.460503 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:13.495664 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 82095.3us
I20260709 12:28:13.463080  3782 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:13.454530  4014 raft_consensus.cc:3060] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:13.386000  4043 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.141s
W20260709 12:28:13.464766  4037 log.cc:927] Time spent T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.297s	user 0.002s	sys 0.001s
I20260709 12:28:13.465371  3775 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:13.406955  4013 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:13.445839  4031 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:13.546804 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.473508  3783 raft_consensus.cc:3060] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:13.410099  3863 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:13.474210  4039 log.cc:927] Time spent T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:28:13.442204 26201 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:13.405254  3804 consensus_queue.cc:1059] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.086s
I20260709 12:28:13.369333  3971 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.372838  3798 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.450145  3829 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.464128 26200 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:13.510063 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.565493 26198 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:13.506647 26757 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:13.511121 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:13.512535 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.569022 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.519017 26195 raft_consensus.cc:2468] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:13.430078  3970 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.083s
I20260709 12:28:13.544677 26129 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:13.546207 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.573714 26126 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:13.532552  4001 raft_consensus.cc:2764] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 32: won
I20260709 12:28:13.509765  3660 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:13.539230  3579 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:28:13.509557  3941 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:13.531567  3997 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:13.560818 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.492163  3822 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.558964 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.600932 26201 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.542439  3617 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:13.570694 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64148.9us
I20260709 12:28:13.550446  3863 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.553895  3855 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.578778 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.579615 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.613263 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.578845  3963 raft_consensus.cc:2804] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:13.611363 26200 raft_consensus.cc:2376] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31. Re-sending same reply.
W20260709 12:28:13.615933 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 226246us
I20260709 12:28:13.508474  3500 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:13.536986  3782 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.553949  4027 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:13.469010  4026 consensus_queue.cc:237] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:13.592453 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.562908  3971 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.616500 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.612463  4001 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.635551 26200 raft_consensus.cc:2376] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27. Re-sending same reply.
I20260709 12:28:13.639526 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:13.624257 26094 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:13.621109 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.623198 26124 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:13.647884 26195 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:28:13.626526  3798 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.627957  4031 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:13.628768  4047 log.cc:927] Time spent T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:28:13.655856  4034 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.645078  4001 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.656946 26050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47798 (request call id 4454) took 2391 ms. Trace:
I20260709 12:28:13.658344  4001 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.657437 26050 rpcz_store.cc:276] 0709 12:28:11.268387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.278887 (+ 10500us) service_pool.cc:224] Handling call
0709 12:28:11.279338 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:11.320102 (+ 40764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.320148 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:13.075213 (+1755065us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b2c000014c8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b72c9ea11 00007f5b72cae372 00007f5b743e0a01 00007f5b743a543a 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2
0709 12:28:13.131122 (+ 55909us) log.cc:844] Serialized 52 byte log entry
0709 12:28:13.523757 (+392635us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:13.523800 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.523873 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.656549 (+132676us) raft_consensus.cc:1685] finished
0709 12:28:13.656576 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.656869 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14994,"raft.queue_time_us":20350,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":22172,"spinlock_wait_cycles":24220032,"thread_start_us":381538,"threads_started":1,"wal-append.queue_time_us":505127}
I20260709 12:28:13.660997 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.661983 26195 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:13.663652 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:13.664659 26195 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:13.593389 26198 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:13.609855  3934 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:13.634898  3963 raft_consensus.cc:697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:13.611868  3863 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:13.641044 26200 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.620548 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.629825  4002 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.632774  4009 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.630154  4030 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:13.622735  3981 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.596846  3660 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.600553  3941 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:13.629390  4051 log.cc:927] Time spent T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:28:13.581892 26194 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:13.655537  3971 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.618312  3855 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.677804 26198 raft_consensus.cc:1240] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:13.672578 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48464.1us
I20260709 12:28:13.674711  4034 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:13.677392 26126 raft_consensus.cc:2468] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:13.686018  3990 raft_consensus.cc:3055] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:13.717847 26198 raft_consensus.cc:1240] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:13.688990 26201 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:13.631209  3500 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.703184 26194 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:13.674636  3617 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.717389  4044 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:28:13.690763  4009 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.731767  4044 consensus_queue.cc:1059] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:28:13.736887  4057 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:28:13.740654  4034 raft_consensus.cc:3055] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:13.741215  4034 raft_consensus.cc:740] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:13.742084  4034 consensus_queue.cc:260] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.743852  3782 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.745661  3660 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.745796  3855 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.746452  3829 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.753228  3941 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.753890  3500 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.718043  3990 raft_consensus.cc:740] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:13.755575  3990 consensus_queue.cc:260] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.756784  3990 raft_consensus.cc:3060] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:13.763314  3963 consensus_queue.cc:237] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.725669 26169 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:13.768071 26194 raft_consensus.cc:1240] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:28:13.769546 26202 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
I20260709 12:28:13.772547  3500 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.707594  3732 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.788558 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.789505 26121 raft_consensus.cc:3060] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:13.788648 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.797711 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.766114  4014 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.780902  3941 raft_consensus.cc:2804] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:13.801245 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.804000 26128 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:13.807224 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:13.804000 26122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.021s	sys 0.036s
W20260709 12:28:13.808854 26122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.021s	sys 0.036s
W20260709 12:28:13.801700 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 386594us
I20260709 12:28:13.825745 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.806043 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:13.796887  3732 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.831137 26128 raft_consensus.cc:2393] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 33: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:13.834151 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.842947 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:13.841084 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:13.845811 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:13.802824  4009 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:13.847976 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:13.844131  3941 raft_consensus.cc:697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:13.870692 26128 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:13.867305 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Not found: tserver 9e23389325a6411c9b0634f4f6af54c3 not available for placement
I20260709 12:28:13.829376 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:13.849813 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 338789us
I20260709 12:28:13.845543  3732 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.786837  3500 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.859655 26121 raft_consensus.cc:2468] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
I20260709 12:28:13.864125 26127 raft_consensus.cc:1240] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 31. Current term is 32. Ops: [31.21-31.21]
W20260709 12:28:13.867074 26120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.031s	sys 0.018s
I20260709 12:28:13.847575  4002 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:13.875813 26122 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:13.863901  4040 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:13.879733 26019 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:13.887897 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:13.884128 26120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.032s	sys 0.018s
I20260709 12:28:13.882184 26169 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:13.847188  3804 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.027s
W20260709 12:28:13.885628 26168 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.899296 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:13.928403 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 48734us
I20260709 12:28:13.929950 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:13.878674  3578 raft_consensus.cc:3055] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:13.940402 26128 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 31 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:28:13.933252 26019 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:13.903637 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:13.942344 26128 raft_consensus.cc:1240] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:13.944324 26128 raft_consensus.cc:1275] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:28:13.887996  4005 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:13.946983 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:13.888630  4040 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:13.936573  4060 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.089s	user 0.001s	sys 0.001s
I20260709 12:28:13.961612  3660 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:13.946241  3798 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:28:13.968215  3696 consensus_meta.cc:287] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.003s
I20260709 12:28:13.937233  4009 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:13.926232  3657 raft_consensus.cc:2804] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:13.969480  3657 raft_consensus.cc:697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:13.970467  3657 consensus_queue.cc:237] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.945475  3579 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.904505  3971 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:13.973768  3971 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.974130  3971 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:13.974468 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.947402  3514 consensus_queue.cc:1059] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
W20260709 12:28:13.977690  4072 log.cc:927] Time spent T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:28:13.980930 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.981236  4030 raft_consensus.cc:3055] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:13.981675  4030 raft_consensus.cc:740] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:13.982115 26788 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:13.982463  4030 consensus_queue.cc:260] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.936312  3578 raft_consensus.cc:740] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:13.983596  3934 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:13.984284  3934 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:13.985986  3941 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.962340  3500 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:13.957755  3803 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:13.988842  3803 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:13.989697  3803 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:13.964763 26196 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
W20260709 12:28:13.965487  4054 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:28:13.988266  3970 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:13.993866  3970 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.994309  3970 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:13.995723  3584 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:14.011114 26200 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:14.012061 26195 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:14.009339 26128 raft_consensus.cc:1275] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:28:14.014245 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.014750 26789 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:28:14.015405 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:13.997093  4040 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:14.024276 26128 raft_consensus.cc:1275] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:28:14.032069 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:14.040968 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.062487 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.064585 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.066254 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.068284 26194 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 25 index: 21. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:28:14.068444 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:14.070905 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:14.073872 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:14.022882  3961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.007s	sys 0.011s
I20260709 12:28:14.073884 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.085178 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.085990 26197 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
W20260709 12:28:14.079852  3961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.007s	sys 0.011s
I20260709 12:28:14.127887  4014 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.129985  3634 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.130748  3634 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.132413  3829 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.133124  3829 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.135696  3829 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:14.142637  3934 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.147027  3732 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.156965 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56090 (request call id 3496) took 1101 ms. Trace:
I20260709 12:28:14.157562 26125 rpcz_store.cc:276] 0709 12:28:13.055589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.214102 (+158513us) service_pool.cc:224] Handling call
0709 12:28:14.156699 (+942597us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54001f86d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:14.156860 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317215616}
I20260709 12:28:14.159610 26125 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:14.039942  3863 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.166785  3863 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.172900  3934 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:14.173856  3934 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.177670  3934 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:14.179029  3634 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.217283 26195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48540 (request call id 3115) took 1047 ms. Trace:
I20260709 12:28:14.218050 26195 rpcz_store.cc:276] 0709 12:28:13.170195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.671525 (+501330us) service_pool.cc:224] Handling call
0709 12:28:14.011902 (+340377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.012432 (+   530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.216964 (+204532us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b5400399750. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:14.217191 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778048896}
W20260709 12:28:14.221387 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.226248 26122 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:14.251601 26129 raft_consensus.cc:1275] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:14.256342 26124 raft_consensus.cc:3055] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:14.257026 26124 raft_consensus.cc:740] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:14.258065 26124 consensus_queue.cc:260] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.18, Last appended by leader: 18, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.259784 26124 raft_consensus.cc:3060] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:14.278175  3934 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.088904  3783 consensus_meta.cc:287] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.003s
I20260709 12:28:14.286728  3934 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.301988  3579 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:14.278869  3997 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.082131  3855 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:14.081869  4071 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:14.311214 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.311920 26126 raft_consensus.cc:1275] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:14.076702  3660 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.252s
I20260709 12:28:14.086474  3500 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.247s
W20260709 12:28:14.366045 26198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.025s	sys 0.059s
W20260709 12:28:14.358043  3590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.008s	sys 0.030s
I20260709 12:28:14.367043  4025 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:14.389864 26094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.069s	sys 0.083s
I20260709 12:28:14.332919  3783 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:14.391839 26198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.026s	sys 0.059s
I20260709 12:28:14.345769  3997 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:14.401306 26094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.069s	sys 0.084s
I20260709 12:28:14.341231  3971 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:14.375248  3617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.014s	sys 0.029s
W20260709 12:28:14.412397 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 697791us
W20260709 12:28:14.388979  4026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.014s	sys 0.029s
I20260709 12:28:14.414109 26094 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:28:14.397408  3590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.009s	sys 0.030s
I20260709 12:28:14.419273 26094 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:14.412096  3617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.014s	sys 0.031s
W20260709 12:28:14.415782  3963 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.037s	sys 0.042s
I20260709 12:28:14.427696 26791 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38294 (request call id 2032) took 1055 ms. Trace:
W20260709 12:28:14.439010  3963 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.037s	sys 0.042s
I20260709 12:28:14.394512  3660 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:14.418682  4026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.014s	sys 0.030s
W20260709 12:28:14.442868 26201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.020s	sys 0.028s
W20260709 12:28:14.443281 26201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.020s	sys 0.028s
I20260709 12:28:14.439083 26791 rpcz_store.cc:276] 0709 12:28:13.372359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.372465 (+   106us) service_pool.cc:224] Handling call
0709 12:28:13.372783 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.442272 (+ 69489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.442319 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.464739 (+ 22420us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.772386 (+307647us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:13.772426 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.415038 (+642612us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x7b5400821254. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:14.415115 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.415244 (+   129us) raft_consensus.cc:1685] finished
0709 12:28:14.415265 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.427603 (+ 12338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":18076,"spinlock_wait_cycles":91461504,"thread_start_us":307158,"threads_started":1,"wal-append.queue_time_us":494719}
I20260709 12:28:14.428558  3829 raft_consensus.cc:2804] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:14.427289  4063 raft_consensus.cc:2804] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:14.404155  4040 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:14.451161 26169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.020s	sys 0.100s
W20260709 12:28:14.463135 26169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.020s	sys 0.100s
W20260709 12:28:14.463486 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 572734us
I20260709 12:28:14.452718 26124 raft_consensus.cc:1275] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 34 index: 18. (term mismatch)
I20260709 12:28:14.464623 26124 pending_rounds.cc:85] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 17
I20260709 12:28:14.464924 26124 pending_rounds.cc:107] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 33.18
I20260709 12:28:14.465193 26124 raft_consensus.cc:2889] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:14.469226 26757 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.440817  3987 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.416576  4025 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:14.501684 26168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.015s	sys 0.007s
W20260709 12:28:14.503432 26168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.015s	sys 0.007s
W20260709 12:28:14.504105 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 618455us
I20260709 12:28:14.471904  3829 raft_consensus.cc:697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:14.498799  3970 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.504s	user 0.016s	sys 0.000s
I20260709 12:28:14.489406  4063 raft_consensus.cc:697] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:14.444751  3660 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.491051  3863 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:14.515008  3934 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:14.548282 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.562723 26127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.006s	sys 0.028s
W20260709 12:28:14.505980  3904 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.019s	sys 0.025s
I20260709 12:28:14.519817  3660 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:14.495232  3987 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.517654  4010 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:28:14.565038 26127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.006s	sys 0.029s
W20260709 12:28:14.572096 26121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.011s	sys 0.019s
W20260709 12:28:14.572517 26121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.011s	sys 0.019s
I20260709 12:28:14.574450 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1923) took 1376 ms. Trace:
I20260709 12:28:14.574962 26121 rpcz_store.cc:276] 0709 12:28:13.198186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.980329 (+782143us) service_pool.cc:224] Handling call
0709 12:28:13.980599 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.980788 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:13.981081 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.573984 (+592903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.574068 (+    84us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:14.574136 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.574222 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.574378 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":937,"prepare.queue_time_us":1745,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"thread_start_us":591543,"threads_started":1}
W20260709 12:28:14.563427  4013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.013s	sys 0.019s
W20260709 12:28:14.566857  3904 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.019s	sys 0.026s
W20260709 12:28:14.564399  3798 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.017s	sys 0.002s
W20260709 12:28:14.568671  3657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.009s	sys 0.018s
W20260709 12:28:14.597535  3798 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.018s	sys 0.002s
I20260709 12:28:14.597963  3798 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:14.583184  4013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.013s	sys 0.020s
I20260709 12:28:14.599320  4013 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:14.600625  4013 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.517102  4063 consensus_queue.cc:237] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.604976  4025 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:14.516180  3829 consensus_queue.cc:237] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:14.608162  4080 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:28:14.611958 26123 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:14.615146 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35926 (request call id 1925) took 1328 ms. Trace:
I20260709 12:28:14.617259  3987 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:14.599314  3798 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.618487 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.620764 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.622337 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.571638  4010 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:14.624145  4010 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.625892  4010 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:14.615628 26123 rpcz_store.cc:276] 0709 12:28:13.286405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.408311 (+121906us) service_pool.cc:224] Handling call
0709 12:28:14.614902 (+1206591us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540013f954. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:14.615068 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571744512}
I20260709 12:28:14.626996  3971 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:14.520253  3863 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.628697  4040 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:14.519915  3970 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.636701 26785 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:14.638984  3970 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:14.645519 26198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3509) took 1220 ms. Trace:
I20260709 12:28:14.648931 26198 rpcz_store.cc:276] 0709 12:28:13.424945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.718973 (+294028us) service_pool.cc:224] Handling call
0709 12:28:13.719352 (+   379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:13.729744 (+ 10392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.729777 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:13.730393 (+   616us) log.cc:844] Serialized 52 byte log entry
0709 12:28:14.428602 (+698209us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:14.428660 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.428760 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.645183 (+216423us) raft_consensus.cc:1685] finished
0709 12:28:14.645214 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.645431 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13353,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":697752,"threads_started":1,"wal-append.queue_time_us":781063}
W20260709 12:28:14.703713 26784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.022s	sys 0.024s
I20260709 12:28:14.640121  4034 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:14.648352  3863 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:14.648556  3657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.010s	sys 0.018s
W20260709 12:28:14.778283 26784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.022s	sys 0.032s
I20260709 12:28:14.781500  4044 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:14.783860  4025 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:14.781268  3579 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.812791  3500 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.814150  3579 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:14.814901  3500 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:14.777264 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.774238  3803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.082s	sys 0.098s
I20260709 12:28:14.811465 26199 raft_consensus.cc:1240] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 33. Current term is 34. Ops: []
W20260709 12:28:14.826072 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.826820 26194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.008s	sys 0.046s
W20260709 12:28:14.827349 26194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.008s	sys 0.046s
W20260709 12:28:14.826727  3941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.023s	sys 0.025s
I20260709 12:28:14.810981  3934 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.838343 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.818473  3803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.082s	sys 0.098s
W20260709 12:28:14.841809  3941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.023s	sys 0.025s
W20260709 12:28:14.858924  4081 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:28:14.859794 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.861910 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.858994 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.817854  4044 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.865375 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:14.872153 26167 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.873178 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 368249us
I20260709 12:28:14.873288 26787 raft_consensus.cc:1240] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:14.823290  4025 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:14.873929 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 443597us
I20260709 12:28:14.847175  3579 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:14.876117 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:14.877588 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:14.878091 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:14.878418 26201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3514) took 1380 ms. Trace:
I20260709 12:28:14.882803 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.846589  3934 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:14.889981 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:14.878849 26201 rpcz_store.cc:276] 0709 12:28:13.498266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.774469 (+276203us) service_pool.cc:224] Handling call
0709 12:28:13.775822 (+  1353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:13.784386 (+  8564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.784417 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:13.793192 (+  8775us) log.cc:844] Serialized 52 byte log entry
0709 12:28:14.446682 (+653490us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:14.863513 (+416831us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.873520 (+ 10007us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.878111 (+  4591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.878187 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.878224 (+    37us) raft_consensus.cc:1685] finished
0709 12:28:14.878236 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.878367 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1318,"raft.queue_time_us":1348,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":11792,"thread_start_us":991841,"threads_started":2,"wal-append.queue_time_us":775374}
I20260709 12:28:14.886266 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:14.887738 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.893973 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:14.914765 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:14.923717 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.923947 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.924431 26198 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
W20260709 12:28:14.925655 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:14.925539 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.925539 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:14.928716 26094 connection.cc:256] RPC call timeout handler was delayed by -1.6092000759999792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 12:28:14.929762 26169 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:14.929314 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5741000760003772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 12:28:14.938225 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:14.943686 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:14.916164  4063 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:14.976217 26197 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:14.919298  3803 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.916505  4085 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:14.926118  4087 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:28:14.986261 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.941917  3514 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:15.014679  3514 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:15.015280  3514 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:14.973011  3500 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.992075  4063 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:15.037189  4090 log.cc:927] Time spent T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:28:14.981659  3863 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:15.047335  3821 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:15.053273  3934 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:15.024011  4084 log.cc:927] Time spent T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:28:15.060436  4082 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.973704  3617 consensus_queue.cc:1048] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.048s
I20260709 12:28:15.061179  4082 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.063664  4082 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.063764  4044 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.064294  3775 raft_consensus.cc:493] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:15.064831  3775 raft_consensus.cc:515] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.067003  3775 leader_election.cc:290] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:14.942198  4026 consensus_queue.cc:1048] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.078s
I20260709 12:28:14.972040  3804 consensus_queue.cc:1048] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.080s
I20260709 12:28:15.007304  4087 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:15.074430  4089 log.cc:927] Time spent T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:28:15.013365  3657 consensus_queue.cc:1048] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 17, Time since last communication: 0.066s
W20260709 12:28:15.075866  4086 log.cc:927] Time spent T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:28:15.011356  4083 consensus_queue.cc:1048] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 12:28:15.075493  4025 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:15.096920 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1912) took 2013 ms. Trace:
I20260709 12:28:15.101037 26127 rpcz_store.cc:276] 0709 12:28:13.084338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.882559 (+798221us) service_pool.cc:224] Handling call
0709 12:28:13.882932 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.883380 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.883419 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.901220 (+ 17801us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.572001 (+670781us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:14.572043 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.572118 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.096690 (+524572us) raft_consensus.cc:1685] finished
0709 12:28:15.096711 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.096859 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1687,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2839,"thread_start_us":610652,"threads_started":1,"wal-append.queue_time_us":886985}
I20260709 12:28:15.012174  3761 consensus_queue.cc:1048] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:28:15.073740  3804 consensus_queue.cc:1243] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer 9e23389325a6411c9b0634f4f6af54c3 log is divergent from this leader: its last log entry 30.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260709 12:28:15.005810  3803 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.075953  4087 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:15.104265  4030 raft_consensus.cc:3060] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:15.095963  3934 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.150951  3934 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.154054  3934 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.109620 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.111208 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:15.111516 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.155153 26121 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:15.155227 26126 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:15.112493 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:15.156620 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.157011 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.157653 26129 raft_consensus.cc:2468] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 34.
I20260709 12:28:15.158023 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.158401 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.159629 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.113828 26125 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:28:15.114697 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:15.115636 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.117527  3822 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.163483 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.118464  4063 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.167373 26121 raft_consensus.cc:2468] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:15.121577 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.068931  3578 consensus_queue.cc:260] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:15.168025 26094 connection.cc:256] RPC call timeout handler was delayed by -1.5003000760002578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 12:28:15.179652 26199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48534 (request call id 3495) took 3270 ms. Trace:
I20260709 12:28:15.180032 26199 rpcz_store.cc:276] 0709 12:28:11.909551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.923100 (+ 13549us) service_pool.cc:224] Handling call
0709 12:28:11.923437 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.818552 (+2895115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.179447 (+360895us) spinlock_profiling.cc:243] Waited 2.89 s on lock 0x7b5400336154. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:15.179599 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6641131392}
W20260709 12:28:15.180864 26094 connection.cc:256] RPC call timeout handler was delayed by -1.3788000760000614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 12:28:15.181324 26094 connection.cc:256] RPC call timeout handler was delayed by -1.2123000760002469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 12:28:15.181622 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1781000760001916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
W20260709 12:28:15.181900 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1691000760001771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
W20260709 12:28:15.182183 26094 connection.cc:256] RPC call timeout handler was delayed by -1.1331000760001189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 12:28:15.133586 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:15.135273 26196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53816 (request call id 1703) took 1921 ms (client timeout 1822 ms). Trace:
W20260709 12:28:15.185775 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 259142us
W20260709 12:28:15.185814 26196 rpcz_store.cc:269] 0709 12:28:13.214058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.255964 (+ 41906us) service_pool.cc:224] Handling call
0709 12:28:15.135097 (+1879133us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b5400217ad4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:15.135202 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626577792}
I20260709 12:28:15.137703  3783 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.205509 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.208590 26125 raft_consensus.cc:2410] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 34 because replica has last-logged OpId of term: 30 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
W20260709 12:28:15.186599 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:15.209446 26168 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.160175 26129 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:15.176513 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4572) took 1956 ms. Trace:
W20260709 12:28:15.176813 26788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38294 (request call id 2027) took 1836 ms (client timeout 1827 ms). Trace:
I20260709 12:28:15.175690 26122 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:15.207746 26194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53816 (request call id 1711) took 1136 ms. Trace:
I20260709 12:28:15.212143 26128 rpcz_store.cc:276] 0709 12:28:13.220339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.025467 (+805128us) service_pool.cc:224] Handling call
0709 12:28:14.169185 (+143718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.169469 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.169508 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.169563 (+    55us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:14.169618 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.169695 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.176313 (+1006618us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54001f86d0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a48ad 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:15.176450 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329491328}
W20260709 12:28:15.211647 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Not found: tserver 9e23389325a6411c9b0634f4f6af54c3 not available for placement
I20260709 12:28:15.214936 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:15.216049 26128 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:15.216358 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.217756 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.218559 26129 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:15.219550 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:15.142653 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.221851 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:15.176008 26126 raft_consensus.cc:2468] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 26.
W20260709 12:28:15.212222 26788 rpcz_store.cc:269] 0709 12:28:13.340327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.340496 (+   169us) service_pool.cc:224] Handling call
0709 12:28:15.176640 (+1836144us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b5400640354. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a92fa 00007f5b7936ab17 00007f5b728e56d4 00007f5b728e5662 00007f5b728e5591 00007f5b728e550c 00007f5b728e47c2 00007f5b726e048c 00007f5b726df8c6 00007f5b726df26f
0709 12:28:15.176748 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405150976}
I20260709 12:28:15.210496 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:15.221077 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:15.212433 26194 rpcz_store.cc:276] 0709 12:28:14.071199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.079588 (+  8389us) service_pool.cc:224] Handling call
0709 12:28:14.083134 (+  3546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.083486 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.083544 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.084026 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.827714 (+743688us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:14.827754 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.827828 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.207523 (+379695us) raft_consensus.cc:1685] finished
0709 12:28:15.207542 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.207687 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1116,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":743334,"threads_started":1,"wal-append.queue_time_us":893041}
I20260709 12:28:15.160663  3579 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.161043  3500 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.253799  4063 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:15.253749  3579 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.174320  3904 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:15.248754 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:15.257942 26129 raft_consensus.cc:2410] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 31 because replica has last-logged OpId of term: 28 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 21.
W20260709 12:28:15.255219 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.204612  3775 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:15.287832 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.209147  3941 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:15.206710  3863 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:15.226533  4044 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.254473  4063 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.302150 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.323566 26129 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:28:15.324745 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.225670  3822 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:15.306241 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 325535us
I20260709 12:28:15.308347  3775 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.207118  3990 consensus_queue.cc:1059] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:28:15.327350 26129 raft_consensus.cc:2393] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:28:15.332305 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 83650.3us
I20260709 12:28:15.346487 26094 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:15.333876 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.347676 26121 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
W20260709 12:28:15.327543 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 177196us
I20260709 12:28:15.308666  3941 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.339105 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:15.341004 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:15.325619  4103 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.350921 26094 leader_election.cc:400] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34.
W20260709 12:28:15.353206 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 132245us
I20260709 12:28:15.305898  4085 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:15.346331 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 205329us
I20260709 12:28:15.267196  3500 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:15.401077 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 179315us
W20260709 12:28:15.367777 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.398191 26756 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.338192  4083 raft_consensus.cc:493] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:15.389451 26094 leader_election.cc:403] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Cancelling election due to peer responding with higher term
I20260709 12:28:15.301674  3904 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:15.405185 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 54477.5us
I20260709 12:28:15.405913 26094 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:15.399428 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.407912 26125 raft_consensus.cc:2410] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 35 because replica has last-logged OpId of term: 33 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:28:15.335566  4034 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:15.402289 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:15.356897 26128 raft_consensus.cc:2468] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:15.355398  3981 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:15.414397 26094 leader_election.cc:400] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31.
I20260709 12:28:15.414458 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.414947 26094 leader_election.cc:403] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:28:15.322499  3863 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.359620  3783 raft_consensus.cc:2804] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:15.415089 26169 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:15.415961 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:15.425879 26121 raft_consensus.cc:2410] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
W20260709 12:28:15.422930 26756 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:15.427196 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:15.427942 26756 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.345469  3971 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.055s
I20260709 12:28:15.390333  3761 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:15.428357 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:15.340212  4044 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:15.432485 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 223057us
I20260709 12:28:15.344363  3822 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:15.403795  4010 consensus_meta.cc:287] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.000s
I20260709 12:28:15.397588  4009 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:15.444854 26168 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.450137 26125 raft_consensus.cc:1275] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:28:15.451686 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:15.452521 26168 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:15.454129 26168 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.454666 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:15.413673  3943 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:15.413414  4043 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election lost for term 33. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 33. Current term is 34.
I20260709 12:28:15.459410 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:15.460029 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:15.479274 26124 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:15.472029 26168 leader_election.cc:336] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.431738  4025 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:15.432973  3783 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.437363  3804 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:15.432056  3829 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:15.479238 26756 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.442986  3761 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:15.475823 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60824us
I20260709 12:28:15.398671  4103 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:15.481032 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:15.490561 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 76314.3us
I20260709 12:28:15.460980  3981 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261) with higher term. Message: Invalid argument: T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for earlier term 30. Current term is 31.
I20260709 12:28:15.491376 26124 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:15.449014  3997 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.491844 26127 raft_consensus.cc:1275] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:28:15.481760 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:15.421176  4085 consensus_queue.cc:1059] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.057s
I20260709 12:28:15.423455  4083 raft_consensus.cc:515] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.507826 26169 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:15.449764  4009 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:15.484573 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 79742.3us
W20260709 12:28:15.502178 26756 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:15.480446  3617 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:15.511413 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.519125 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:15.493749  3783 raft_consensus.cc:3060] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:15.527400 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:15.527302 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 45535.2us
I20260709 12:28:15.503302  3761 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:15.500291  4025 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.507366  3943 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:15.535202 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:15.532527 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:15.520856  4009 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:15.541776 26094 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:15.499933  3514 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.530148  3997 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.534790  3981 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:15.539862  3584 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:15.562384  4100 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.241s	user 0.001s	sys 0.001s
I20260709 12:28:15.550899  4116 raft_consensus.cc:2804] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:15.542994  4026 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.023s
I20260709 12:28:15.560351  4103 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.548189  4120 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:28:15.580668 26784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38282 (request call id 1679) took 1585 ms. Trace:
I20260709 12:28:15.567521  3579 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.582000 26784 rpcz_store.cc:276] 0709 12:28:13.995034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.995421 (+   387us) service_pool.cc:224] Handling call
0709 12:28:13.995791 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.808746 (+812955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.808776 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.817621 (+  8845us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.998428 (+180807us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:14.998478 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.998586 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.580362 (+581776us) raft_consensus.cc:1685] finished
0709 12:28:15.580396 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.580579 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6479,"raft.queue_time_us":11728,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5537,"thread_start_us":965099,"threads_started":2,"wal-append.queue_time_us":363450}
I20260709 12:28:15.549463  3943 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.588701  4119 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:15.589943  4107 log.cc:927] Time spent T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:28:15.590487  4119 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:15.590981  4119 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.591387  4119 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:15.577206  4129 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:15.592347  4116 raft_consensus.cc:697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:15.592993  4129 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.593736  4116 consensus_queue.cc:237] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.580008  4106 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:15.596268  4106 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.598560  4106 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.559532  3997 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.602087  3997 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.602679  3997 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:15.603042  3997 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.603377  3997 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:15.599939  3904 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.605006  3904 raft_consensus.cc:2804] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:15.605427  3904 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.605793  3904 raft_consensus.cc:3060] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:15.608528  4101 log.cc:927] Time spent T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 12:28:15.610023  4102 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:28:15.567700  4063 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:15.589059  4112 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:28:15.572491  4114 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.573268  3775 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:15.645905  4085 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 31.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:15.622511  3514 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.630414  4087 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.632860  3981 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:15.632853  3761 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.575024  3584 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:15.616807  3941 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:15.685177 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.707414  4102 log.cc:927] Time spent T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:28:15.649886  4127 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:15.650002  3863 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.674563  3500 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:15.648121  3775 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:15.710083  4044 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.708997  3584 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:15.645238  4114 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:15.779369 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 244224us
I20260709 12:28:15.765677  3584 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:15.796201 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.807673 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 246969us
I20260709 12:28:15.808341 26194 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:15.760815  4127 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.785434  4083 leader_election.cc:290] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.773901  3775 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.794327  3863 raft_consensus.cc:493] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.790040  3970 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
W20260709 12:28:15.820516  4121 log.cc:927] Time spent T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.086s	user 0.001s	sys 0.001s
I20260709 12:28:15.823408 26197 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 33. Current term is 34. Ops: []
W20260709 12:28:15.815352  4133 log.cc:927] Time spent T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:28:15.813849  4025 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:15.853785 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 321322us
I20260709 12:28:15.855005 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:15.857645 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.861099 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:15.862030 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.862835 26203 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:15.866070 26201 raft_consensus.cc:1240] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:15.866194 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:15.867390 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.867624 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.863113  4124 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:15.869311 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.870101  3578 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:15.857967  3941 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.871039  3941 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:15.872637 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.873277  3941 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:15.873839  4136 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:15.874439  4136 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.859364  4083 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.877737  4083 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.879848  4083 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.868482 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.869421  4124 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:15.881024 26197 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:15.863091 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:15.858732 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:15.871970  3803 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.830729  3775 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:15.885464 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.852694  3761 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:15.828267  3863 raft_consensus.cc:515] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.829952  3970 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:15.986792 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.952669 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.986320 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.999960 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddc5e875aa744c50b603a5072bc7305b" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.952391 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.005643 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.006706 26201 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:15.858698  3821 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.000929  4030 consensus_meta.cc:287] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.825s	user 0.009s	sys 0.005s
I20260709 12:28:15.901810 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:15.976714 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.016997 26788 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:16.019508 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.865077  3783 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:16.022943 26756 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:16.024021 26756 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.024554 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.025321 26789 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:16.026232 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:16.027695 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.028193 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:16.028357 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.028867 26203 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
W20260709 12:28:16.029414 26756 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.860849  3514 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.038058 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.029994 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.011790  3904 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.045396  3584 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.046746  4127 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.047576  3904 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.048502  4136 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.030089  3821 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.050598  3970 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:16.050926  3821 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:16.050433 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.995836  3803 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:15.995263  4087 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:16.042513 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.055181 26200 raft_consensus.cc:1240] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:16.056430 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.059460 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:16.057915 26756 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.954249  3798 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:16.062602 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.063149 26201 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
W20260709 12:28:16.062007 26756 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.989828  3997 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.065860 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:15.989858  4119 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.043422  3514 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.067750 26200 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:16.179965 26194 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: []
W20260709 12:28:16.123735  4009 consensus_meta.cc:287] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.002s
I20260709 12:28:16.180670 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.133656  3863 leader_election.cc:290] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.142891  3997 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.181996 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.134672 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.183383 26195 raft_consensus.cc:2376] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27. Re-sending same reply.
I20260709 12:28:16.183491 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.183781  3514 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:16.078390 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.135021 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:16.139048  4148 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:28:16.136102  4114 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.191927 26127 pending_rounds.cc:85] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 21
I20260709 12:28:16.221306 26127 pending_rounds.cc:107] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 30.22
I20260709 12:28:16.221477 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.134060 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.221758 26127 raft_consensus.cc:2889] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.222390 26201 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:28:16.193264 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.222781 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.223634 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.224929 26201 raft_consensus.cc:2393] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:16.064782 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.227239 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.228044 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.230485 26200 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:16.231693 26195 raft_consensus.cc:1240] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:16.057457  3783 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:16.233943 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 171957us
W20260709 12:28:16.234581 26756 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:16.235102 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.064757  3798 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.236483 26019 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:16.238363 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.064358  3584 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.061275  3803 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.244578 26019 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.246068 26785 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:16.249302 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.237059  4005 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:16.254220  4147 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:16.254653  4125 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:16.255173 26790 raft_consensus.cc:1240] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 32. Current term is 33. Ops: [30.19-32.20]
I20260709 12:28:16.255488 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.255806 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.257330 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:28:16.259125 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.262059 26785 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:16.245662  3804 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.237720  3798 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.180860  4009 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.243451  3783 raft_consensus.cc:2804] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:16.250404  4085 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:16.267519  4144 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:16.274479 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.277017  4144 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.284970 26782 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:16.285338  4144 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:16.285915  4152 log.cc:927] Time spent T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:28:16.269577  3783 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:16.290511  3783 raft_consensus.cc:3060] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.295275  4144 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.298274  4144 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.252023  4124 consensus_queue.cc:1059] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:28:16.301329  4124 raft_consensus.cc:3055] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:16.301828  4124 raft_consensus.cc:740] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:16.302593  3579 raft_consensus.cc:3055] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:16.303089  3579 raft_consensus.cc:740] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:16.268255  4125 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.307740  4125 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.317569 26122 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:28:16.309020  4085 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.303009  4124 consensus_queue.cc:260] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.276041  4114 consensus_queue.cc:1059] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 12:28:16.271418  3804 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:16.339692 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 90447.6us
W20260709 12:28:16.340480 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93355.5us
W20260709 12:28:16.342777 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.337541  4119 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.340924  3863 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.304177  3579 consensus_queue.cc:260] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.352490  4009 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:16.352926  3579 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.352551  3863 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.359442  3863 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:16.360973  3803 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.340551  4085 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.363721  4085 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.365669  4124 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:16.371343  3943 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.372362  3798 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.377512  3783 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.384378 26019 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.387460  3798 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:16.392737 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44214us
I20260709 12:28:16.393322  3798 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:16.394083 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.394686  3783 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.397679 26120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3514) took 1261 ms. Trace:
W20260709 12:28:16.394761 26168 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.398756 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:16.395359  3798 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.400496  3798 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.402832  3798 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:16.404621 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.405773  3798 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.407189  3798 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.414186 26121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56090 (request call id 3504) took 1284 ms. Trace:
I20260709 12:28:16.414633 26121 rpcz_store.cc:276] 0709 12:28:15.129909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.486732 (+356823us) service_pool.cc:224] Handling call
0709 12:28:15.507557 (+ 20825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.507769 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:15.533911 (+ 26142us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.413687 (+879776us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.413860 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.413904 (+    44us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:16.413955 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.414020 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.414129 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":870289,"prepare.queue_time_us":45551,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":24871,"threads_started":1,"wal-append.queue_time_us":185294}
I20260709 12:28:16.416316  4103 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.416564 26757 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:16.416937  4103 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.421365 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1934) took 1292 ms. Trace:
I20260709 12:28:16.421466  3578 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:16.419917  4103 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.421929 26125 rpcz_store.cc:276] 0709 12:28:15.129005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.435254 (+306249us) service_pool.cc:224] Handling call
0709 12:28:15.435685 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.451070 (+ 15385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.421131 (+970061us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b54001f63d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:16.421305 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28408192}
I20260709 12:28:16.400538  3822 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.419821  4025 consensus_queue.cc:1059] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.010s
I20260709 12:28:16.398068 26120 rpcz_store.cc:276] 0709 12:28:15.135954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.486882 (+350928us) service_pool.cc:224] Handling call
0709 12:28:15.487321 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.530512 (+ 43191us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:15.530557 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.530601 (+    44us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:15.530633 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.991548 (+460915us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b480021c3d8. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b792ceaa1 00007f5b792c5b37 00007f5b792c59ca 00007f5b793bd26f 00007f5b793bcf37 00007f5b793d155c 00007f5b793d1508 00007f5b793d1488 00007f5b793d1443 00007f5b793d06ec 00007f5b782faa96
0709 12:28:16.397451 (+405903us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b54001c4654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:16.397501 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.397622 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":438280,"raft.queue_time_us":8319,"raft.run_cpu_time_us":2644,"raft.run_wall_time_us":454607,"spinlock_wait_cycles":95339136}
W20260709 12:28:16.435474 26092 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.442377 26790 raft_consensus.cc:1275] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 21. (term mismatch)
I20260709 12:28:16.442552 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.442932 26790 pending_rounds.cc:85] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 20
I20260709 12:28:16.443262 26790 pending_rounds.cc:107] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 31.21
I20260709 12:28:16.443730 26790 raft_consensus.cc:2889] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.440385 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.449117 26120 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 24.
I20260709 12:28:16.443208 26789 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:16.451061  4009 raft_consensus.cc:2804] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:16.451606 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.437821  4125 raft_consensus.cc:3055] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:16.455163  4009 raft_consensus.cc:697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:16.455562  4125 raft_consensus.cc:740] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:16.456385  4009 consensus_queue.cc:237] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.459414  4025 consensus_queue.cc:1243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer a981c9a7972c4e99a0903924c64fbb5c log is divergent from this leader: its last log entry 31.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:28:16.460078  4025 raft_consensus.cc:3055] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Stepping down as leader of term 30
W20260709 12:28:16.460139 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 65403.1us
I20260709 12:28:16.460529  4025 raft_consensus.cc:740] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:16.460788  4129 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:16.461357  4129 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:16.461426  4025 consensus_queue.cc:260] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.456614  4125 consensus_queue.cc:260] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.462782  4125 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:16.462815  4025 raft_consensus.cc:3060] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:16.466738 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.467918 26167 leader_election.cc:304] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.469183  3971 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.469702  3971 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.470417 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.471220  3775 raft_consensus.cc:2749] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:16.473632  3971 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:16.423470  3798 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.438577 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.477063  3971 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:28:16.477656  4005 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:16.478089  4005 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:16.478708  3804 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.479035  4005 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.481726  4005 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:16.483881  4160 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.154s	user 0.000s	sys 0.002s
I20260709 12:28:16.493225 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:16.492172 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Not found: tserver 9e23389325a6411c9b0634f4f6af54c3 not available for placement
I20260709 12:28:16.493366  3775 raft_consensus.cc:2764] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:28:16.496517  3500 raft_consensus.cc:3060] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:16.497648  3634 raft_consensus.cc:2804] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:28:16.497808 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 956002us
I20260709 12:28:16.498257  3634 raft_consensus.cc:697] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:16.498787 26094 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.499163  3634 consensus_queue.cc:237] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.497057 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.503924 26020 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.504712  4071 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:16.505342  4071 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:16.506880  4147 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:16.507206  4128 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.510771  3500 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:16.513116 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 61878us
W20260709 12:28:16.522123 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48765.2us
I20260709 12:28:16.524092 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:16.524610 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 47149.9us
I20260709 12:28:16.525893 26756 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.529464 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.529690 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.530150 26121 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:16.530306 26129 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.530531 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.529134 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.531096 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.513549 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:16.516948 26126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4591) took 1129 ms. Trace:
I20260709 12:28:16.520084 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.535703  3829 consensus_queue.cc:1048] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:16.536077 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.536638 26122 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.532490 26126 rpcz_store.cc:276] 0709 12:28:15.387078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.729694 (+342616us) service_pool.cc:224] Handling call
0709 12:28:15.733821 (+  4127us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:16.423461 (+689640us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:16.468448 (+ 44987us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.513611 (+ 45163us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:16.514168 (+   557us) log.cc:844] Serialized 52 byte log entry
0709 12:28:16.514326 (+   158us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:16.514534 (+   208us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.514724 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.514799 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.516751 (+  1952us) raft_consensus.cc:1685] finished
0709 12:28:16.516770 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.516890 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":725464,"prepare.queue_time_us":10653,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":727471,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":242,"thread_start_us":53017,"threads_started":2,"wal-append.queue_time_us":7459}
I20260709 12:28:16.533322 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:16.538295 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:16.553649  3732 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:16.554253  3732 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:16.554782 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.554886 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.555171  3732 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.556144 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.459436 26123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56104 (request call id 4592) took 1072 ms. Trace:
I20260709 12:28:16.556478 26019 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:16.557242 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.557879  3904 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:16.558115 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.558571 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.559238 26120 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:16.556728 26123 rpcz_store.cc:276] 0709 12:28:15.387380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.730693 (+343313us) service_pool.cc:224] Handling call
0709 12:28:15.763320 (+ 32627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.424193 (+660873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.424265 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.424767 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.458842 (+ 34075us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:16.458879 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.458941 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.458978 (+    37us) raft_consensus.cc:1685] finished
0709 12:28:16.458995 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.459379 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":747949,"raft.queue_time_us":661299,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":268,"spinlock_wait_cycles":1335680,"thread_start_us":33178,"threads_started":1,"wal-append.queue_time_us":31477}
I20260709 12:28:16.562333 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.563289 26121 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 31 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:28:16.563421 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:16.560554 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:16.565692 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.558429  3904 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:16.579871 26125 raft_consensus.cc:1240] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:16.549618  4103 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:16.580653 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.581697 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.549418  4026 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:16.561553  3822 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:16.570904 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:16.592892 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.593003 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.614107 26202 pending_rounds.cc:85] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 20
I20260709 12:28:16.614195 26785 raft_consensus.cc:1813] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Handling vote request from an unknown peer a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.614539 26202 pending_rounds.cc:107] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 31.21
I20260709 12:28:16.614650 26785 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:16.614871 26202 raft_consensus.cc:2889] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:16.615274 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 44406.6us
I20260709 12:28:16.595547 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.616672 26788 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:28:16.617502 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.595924 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.597029 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.569461  3798 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:16.621130 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.599109 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.566946  4071 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:16.612576 26120 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:16.634534 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:16.634824 26017 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.566586  4128 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:16.567265  3971 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:16.608772 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.639729 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.618932 26197 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:16.640671 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.641373 26782 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 33: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:16.598241 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.619436 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.641819 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.622500 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.623015 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.569468  4074 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:16.656157 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.657296 26120 raft_consensus.cc:2393] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 31: Already voted for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in this term.
I20260709 12:28:16.660175 26019 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.635519 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:16.661171 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.635888 26123 raft_consensus.cc:1240] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:28:16.634711 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.621053 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.586930  4125 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:16.653501 26195 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:16.662581 26194 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:28:16.638659 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 197276us
I20260709 12:28:16.663542 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.669135 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.669361 26199 raft_consensus.cc:2376] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28. Re-sending same reply.
I20260709 12:28:16.569234  3829 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:28:16.666956 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.604099  4103 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:16.669760 26788 raft_consensus.cc:1240] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:16.670706 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.692274 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.670298 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.697628 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.653301  3971 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.584652  3578 consensus_queue.cc:1059] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.075s
I20260709 12:28:16.637897  3798 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.613749  3822 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:16.697103 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.698179 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.634698  4071 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.677465 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:16.700016 26123 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:16.718910 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.719939 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.702620 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.720726 26789 raft_consensus.cc:2468] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:16.703207 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:16.667449 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:16.717113 26756 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.634451  4105 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:16.717554 26198 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:16.724119 26197 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:16.742085 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.744824 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.654474  4074 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:16.634289  4001 consensus_queue.cc:1059] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.080s
I20260709 12:28:16.744952 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.670049 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.759621 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.760294 26790 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:16.718303 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.760715 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.760893 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.745946 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:16.746438 26200 raft_consensus.cc:2376] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27. Re-sending same reply.
I20260709 12:28:16.747239 26198 raft_consensus.cc:2468] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:16.747081 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.698706  4103 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:16.698201  4139 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:16.755461 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:16.698283  3855 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:16.770522 26168 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.757148 26126 raft_consensus.cc:1275] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 34 index: 18. (index mismatch)
I20260709 12:28:16.702863  3971 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:16.662962  4132 consensus_queue.cc:1059] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.066s
I20260709 12:28:16.773062 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:16.764604 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.718753  3822 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:16.721583  4117 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.739670  4071 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:16.780508 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.756942 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:16.763900 26198 raft_consensus.cc:1275] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
W20260709 12:28:16.779529 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:16.782406 26168 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.717538  3798 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.777567  4009 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.016s
I20260709 12:28:16.785813 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.724130  4063 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:16.760097  4074 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:16.745996  4105 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:16.826315 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adcc76aa423a40eb95d31d98a8c8776f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.820871 26122 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:16.827353 26120 raft_consensus.cc:2410] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 30 because replica has last-logged OpId of term: 27 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 21.
I20260709 12:28:16.805121 26127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1941) took 1371 ms. Trace:
I20260709 12:28:16.776443  3617 raft_consensus.cc:3055] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:16.778772  3578 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:16.776055 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 58972.1us
W20260709 12:28:16.832782 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.833071 26128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1930) took 1717 ms. Trace:
I20260709 12:28:16.783363 26196 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 24.
I20260709 12:28:16.780308  3943 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:16.826215 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.827885 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:16.833388 26756 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.836854 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.838541 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:16.828997 26785 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:28:16.828258 26127 rpcz_store.cc:276] 0709 12:28:15.434287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.191173 (+756886us) service_pool.cc:224] Handling call
0709 12:28:16.191629 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:16.505903 (+314274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.505935 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:16.506485 (+   550us) log.cc:844] Serialized 52 byte log entry
0709 12:28:16.564672 (+ 58187us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:28:16.564710 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.564769 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.804858 (+240089us) raft_consensus.cc:1685] finished
0709 12:28:16.804882 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.805032 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":282929,"raft.queue_time_us":289491,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":57780,"threads_started":1,"wal-append.queue_time_us":193947}
I20260709 12:28:16.758708  3904 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.846274 26127 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:16.818395  4063 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:16.854350 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:16.827853 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 130050us
I20260709 12:28:16.847908 26782 raft_consensus.cc:1240] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:16.833534 26128 rpcz_store.cc:276] 0709 12:28:15.115618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.415642 (+300024us) service_pool.cc:224] Handling call
0709 12:28:15.997523 (+581881us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b580011b298. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b71280b71 00007f5b71280a22 00007f5b7a1998d9 00007f5b792f3ee0 00007f5b793bd0ab 00007f5b793bd131 00007f5b7937efc0 00007f5b79369929 00007f5b793693d2 00007f5b728e0bd4
0709 12:28:15.997801 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.426121 (+428320us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:28:16.686068 (+259947us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.832744 (+146676us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.832794 (+    50us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:28:16.832845 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.832908 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.833014 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428650,"prepare.queue_time_us":6697,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":428083,"raft.run_cpu_time_us":45,"raft.run_wall_time_us":45,"spinlock_wait_cycles":20540288,"thread_start_us":366771,"threads_started":2}
I20260709 12:28:16.825472  4105 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:16.851359 26756 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.897500 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.856416 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:16.901028 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 44704us
I20260709 12:28:16.903509 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:16.906745  3578 consensus_queue.cc:1048] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:16.908180  4125 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:16.913097  4178 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:16.927583 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.783615 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.852957  3804 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:16.866014 26124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1940) took 1459 ms. Trace:
I20260709 12:28:16.919705 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:16.876163 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 196) took 38907 ms (client timeout 15000 ms). Trace:
I20260709 12:28:16.868253  4147 raft_consensus.cc:3055] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:17.185290  4147 raft_consensus.cc:740] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:17.186316  4147 consensus_queue.cc:260] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.187602  4147 raft_consensus.cc:3060] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:17.185089 25946 rpcz_store.cc:269] 0709 12:27:37.968968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.829600 (+14860632us) service_pool.cc:224] Handling call
0709 12:28:16.876049 (+24046449us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:09.904857 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:09.968601 (+ 63744us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:09.968669 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:28:09.968694 (+    25us) tablet.cc:662] Decoding operations
0709 12:28:10.154533 (+185839us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:10.161810 (+  7277us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:10.164096 (+  2286us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:28:10.257448 (+ 93352us) tablet.cc:701] Row locks acquired
0709 12:28:10.257511 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:28:10.257693 (+   182us) write_op.cc:270] Start()
0709 12:28:10.257859 (+   166us) write_op.cc:276] Timestamp: P: 1783600090257616 usec, L: 0
0709 12:28:10.257885 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:28:10.258662 (+   777us) log.cc:844] Serialized 7557 byte log entry
0709 12:28:10.278149 (+ 19487us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b
0709 12:28:11.055804 (+777655us) op_driver.cc:464] REPLICATION: finished
0709 12:28:11.353465 (+297661us) write_op.cc:301] APPLY: starting
0709 12:28:11.370133 (+ 16668us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:12.205137 (+835004us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:12.205178 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:13.164252 (+959074us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:13.165183 (+   931us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:28:13.165234 (+    51us) write_op.cc:312] APPLY: finished
0709 12:28:13.165699 (+   465us) log.cc:844] Serialized 172 byte log entry
0709 12:28:13.797369 (+631670us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b4400022b4c. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b71140db1 00007f5b711dec85 00007f5b711df6a2 00007f5b73f511db 00007f5b73f4caaf 00007f5b7129b0ac 00007f5b7a0f59a9 00007f5b712b769b 00007f5b712b593f 00007f5b712b7c02 00007f5b73f3f20b
0709 12:28:14.121219 (+323850us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:14.121491 (+   272us) write_op.cc:454] Released schema lock
0709 12:28:14.121596 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:14.122152 (+   556us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:14.122194 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8496205,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":246294,"apply.run_cpu_time_us":55224,"apply.run_wall_time_us":2819673,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":199839,"prepare.run_cpu_time_us":23955,"prepare.run_wall_time_us":511818,"raft.queue_time_us":448578,"raft.run_cpu_time_us":16564,"raft.run_wall_time_us":56576,"replication_time_us":797750,"spinlock_wait_cycles":102467328,"thread_start_us":1520197,"threads_started":5,"wal-append.queue_time_us":1493367,"wal-append.run_cpu_time_us":63737,"wal-append.run_wall_time_us":2569513}]]}
W20260709 12:28:17.190447 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 197) took 16994 ms (client timeout 15000 ms). Trace:
W20260709 12:28:17.190797 25946 rpcz_store.cc:269] 0709 12:28:00.195789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.190208 (+16994419us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:17.190387 (+   179us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:28:16.870833  4063 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.193353  4063 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:16.877235  4173 log.cc:927] Time spent T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.103s	user 0.002s	sys 0.000s
I20260709 12:28:16.782166  4117 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.201288  4117 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:16.883121  4002 consensus_queue.cc:1048] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:16.830267  3617 raft_consensus.cc:740] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:16.891479  3971 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.202524  3617 consensus_queue.cc:260] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.24, Last appended by leader: 24, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.203545  3971 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.917145  4103 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.946404  3579 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:16.932317 26124 rpcz_store.cc:276] 0709 12:28:15.406314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.878257 (+471943us) service_pool.cc:224] Handling call
0709 12:28:15.878574 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.425412 (+546838us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:28:16.569508 (+144096us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.832086 (+262578us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.832276 (+   190us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.844678 (+ 12402us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.844782 (+   104us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:28:16.844803 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.844894 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.865787 (+ 20893us) raft_consensus.cc:1685] finished
0709 12:28:16.865807 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.865948 (+   141us) inbound_call.cc:177] Queueing success response
0709 12:28:16.882799 (+ 16851us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b18001e64c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
0709 12:28:16.884884 (+  2085us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b18001e64c0. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7a180ab1 00007f5b7a18d281 00007f5b712b8522 00007f5b712ba7c1 00007f5b712ba779 00007f5b712ba709 00007f5b712ba6d1 00007f5b712b99cd 0000000000388634 0000000000388479 00007f5b7129b83a
Metrics: {"mutex_wait_us":546429,"prepare.queue_time_us":8259,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":549755,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":12925,"spinlock_wait_cycles":180731392,"thread_start_us":334930,"threads_started":2,"wal-append.queue_time_us":265265}
I20260709 12:28:16.866355 26129 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:16.872653 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:16.990518 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 325566us
W20260709 12:28:17.472537 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 599902us
I20260709 12:28:16.985441 26784 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
I20260709 12:28:16.952093 26787 raft_consensus.cc:1275] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 21. (term mismatch)
I20260709 12:28:17.477972 26787 pending_rounds.cc:85] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 20
I20260709 12:28:17.478268 26787 pending_rounds.cc:107] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 30.21
I20260709 12:28:17.478505 26787 raft_consensus.cc:2889] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:17.479784  3943 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:16.867167 26790 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
W20260709 12:28:17.252957 26754 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.302771 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.481472 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.308686 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.482225 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.482671 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.309394  3732 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:17.289224  3579 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.488806  3732 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.489521  3579 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:16.975274  3798 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.495107  3798 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.977566  3822 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.505195  3732 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.506668  3822 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:17.203590  3617 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:16.974555  4071 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.951648  4074 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.205699  4103 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.530395  4103 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.530498  4074 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.530958  4103 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.531693  4071 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:17.532886  4103 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:16.984391  4105 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:16.932296  4125 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:17.124007  4124 raft_consensus.cc:3055] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:28:17.327561 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 432185us
I20260709 12:28:16.931561 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:17.336190 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 416566us
I20260709 12:28:17.541327 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.542467 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:17.367213 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 515845us
I20260709 12:28:17.395814  4082 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:17.397643 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.406101  4010 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.546123  4010 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.547065 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:17.411092 26783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.020s	sys 0.023s
I20260709 12:28:17.548597 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:28:17.548733 26783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.020s	sys 0.023s
I20260709 12:28:17.548539 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:17.549489 26782 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 34: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:17.437446  4034 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.550196  4034 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.551105 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:17.548810  3798 raft_consensus.cc:3060] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:17.446360 25945 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 12 to 16, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 16 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:17.552163 26167 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.776s (SENT)
I20260709 12:28:17.553467 25945 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
W20260709 12:28:17.554196 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.454869 26125 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:17.558221 26125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35926 (request call id 1945) took 1002 ms. Trace:
I20260709 12:28:17.558565 26125 rpcz_store.cc:276] 0709 12:28:16.556169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.603880 (+ 47711us) service_pool.cc:224] Handling call
0709 12:28:16.641029 (+ 37149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.556503 (+915474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.557954 (+  1451us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54003d1654. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430e571 00007f5b743a5e01 00007f5b743a483e 00007f5b79369419 00007f5b728e0bd4 00007f5b728e0b62 00007f5b728e0a91 00007f5b728e0a0c 00007f5b728dfcc2 00007f5b726e048c 00007f5b726df8c6
0709 12:28:17.558159 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1871074048}
I20260709 12:28:17.469560 26126 raft_consensus.cc:1240] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:17.559356  3798 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:17.480391  3943 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:17.559907  3943 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:17.548539 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:17.481629  3775 consensus_queue.cc:1048] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
I20260709 12:28:17.499249  3579 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.510255 26123 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:17.568346  3775 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:17.544986  4082 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.570626  4193 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:17.571516  4082 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.571969  4034 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:17.504002 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Not found: tserver 9e23389325a6411c9b0634f4f6af54c3 not available for placement
I20260709 12:28:17.535414  4105 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:17.476898 26094 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.536146  4124 raft_consensus.cc:740] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:17.565305  4103 consensus_queue.cc:1059] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:28:17.576507 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 99647.5us
I20260709 12:28:17.574465 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:17.577268 26094 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.577253  4147 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:28:17.578007 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.585436 26120 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:28:17.580615 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:17.591457 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:17.592213 26124 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:17.592960 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:17.596035 26120 raft_consensus.cc:1275] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:17.593844 26123 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
I20260709 12:28:17.598318  4103 raft_consensus.cc:3055] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:17.598738  4103 raft_consensus.cc:740] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:17.599588  4103 consensus_queue.cc:260] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.600795  4103 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:17.601678  4074 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:17.598270 26020 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.606604 26124 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:17.606709  3855 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:17.607332  3855 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:17.608407  3855 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.597901  3579 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:17.611223  4071 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:17.614017 26123 pending_rounds.cc:85] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 15
I20260709 12:28:17.614485 26123 pending_rounds.cc:107] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 12:28:17.614823 26123 raft_consensus.cc:2889] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:17.584213  4010 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.611248  3579 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:17.612628  4071 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:17.621061  3904 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.629729 26199 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:28:17.631469 26120 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:17.632645 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:17.633878 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:17.634322 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:17.634646 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:17.639791 26127 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:17.577201  4124 consensus_queue.cc:260] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.645095 26202 raft_consensus.cc:1275] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:17.647873 26202 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:28:17.635713 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:17.637472 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 126330us
W20260709 12:28:17.663775 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 880093us
W20260709 12:28:17.665091 26168 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.642583  3904 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.666225 26127 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:28:17.666760 26168 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:17.624029  3943 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.665540  3584 consensus_queue.cc:1059] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.002s
I20260709 12:28:17.653788  4117 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.682848 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:17.689958 26757 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:17.696379 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.697609 26202 raft_consensus.cc:2468] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:17.693475 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:17.700644 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 156275us
I20260709 12:28:17.656953  3798 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.707386 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:17.702920 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:17.653889  4125 consensus_queue.cc:1175] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e23389325a6411c9b0634f4f6af54c3" responder_term: 31 status { last_received { term: 31 index: 19 } last_committed_idx: 17 last_received_current_leader { term: 31 index: 19 } }
I20260709 12:28:17.710120 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:17.667127  4124 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:17.739514 26203 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:17.746516 26199 raft_consensus.cc:2393] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 34: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:17.746917 26167 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:17.692606  4010 raft_consensus.cc:493] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.748176 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48729us
I20260709 12:28:17.755036 26094 leader_election.cc:304] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.763126 26198 raft_consensus.cc:1240] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 31. Current term is 32. Ops: [31.24-31.24]
W20260709 12:28:17.763790 26168 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.700932  4082 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:17.668058  4005 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:28:17.764671 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:17.765592 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:17.708351  4009 raft_consensus.cc:2804] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:28:17.766194 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 52457.4us
I20260709 12:28:17.769428  4010 raft_consensus.cc:515] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.781005  4010 leader_election.cc:290] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.781947 26202 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:17.782765 26195 raft_consensus.cc:3060] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:17.786146 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:17.694119  3617 consensus_queue.cc:1048] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.070s
W20260709 12:28:17.788165 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 85569.3us
I20260709 12:28:17.761920  4043 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.769466  4197 log.cc:927] Time spent T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:28:17.814566 26195 raft_consensus.cc:2468] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:17.693035  3579 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.703269  4194 consensus_queue.cc:1059] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 32.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.105s
I20260709 12:28:17.790401  4193 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:17.784698  4002 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:17.864650 26019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.855780 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 100810us
I20260709 12:28:17.704043  4117 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.746241  3798 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.872882 26094 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.864792 26120 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:17.870903 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:17.803711  4009 raft_consensus.cc:697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:17.876120 26127 raft_consensus.cc:2393] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:17.888818 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 141984us
I20260709 12:28:17.862475  4005 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:17.891610 26167 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:17.768812  4071 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.899565 26167 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:17.816220  4043 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.919685  4117 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:17.921316 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.921492  4117 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.922358  4117 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.925127  4117 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:17.927546  4117 raft_consensus.cc:2804] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:17.928082  4117 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:17.928655  4117 raft_consensus.cc:3060] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:17.873430  4002 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.932121  4147 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.932776  4147 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:17.934721  4002 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:17.934988  4147 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:17.936115  3943 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.936902  4009 consensus_queue.cc:237] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:17.937446  3943 raft_consensus.cc:3055] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:17.937860  3943 raft_consensus.cc:740] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:17.938503 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:17.900353 26094 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.893383  4005 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:17.940766 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:17.905010  4074 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:17.916355  4082 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.939388  4010 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.936939  4043 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:17.971519 26094 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.945595  4063 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:17.968464 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 182364us
W20260709 12:28:17.983618 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.984674 26094 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997
I20260709 12:28:17.938618  3943 consensus_queue.cc:260] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:17.988456 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.996579 26019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 1.602s (SENT) [suppressed 120 similar messages]
I20260709 12:28:18.000300 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:18.004164 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 240503us
W20260709 12:28:18.004967 26168 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.005373 26168 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:18.007906 26168 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.008394 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:17.987550  3579 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.966799 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96261.6us
W20260709 12:28:18.009341 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.997433  4216 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:18.004210 26094 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:17.985744  4217 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.986545  4222 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:17.998147  4074 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:17.996970  4213 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.982213 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.011574 26020 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:18.007090  4043 raft_consensus.cc:2764] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:28:18.011080  3943 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:18.006352  4063 consensus_queue.cc:1048] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.024s
W20260709 12:28:18.023411 26168 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.022794  4218 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:28:18.047009 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 64883.1us
I20260709 12:28:18.045784 26168 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.023360  4214 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:18.047947 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:18.043442  4199 log.cc:927] Time spent T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:28:18.049089  3855 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 28.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:18.046425  4219 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:18.051167  3798 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.051304  4224 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:18.052103 26122 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:28:18.022850  4222 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.019116  3579 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.056008  3971 raft_consensus.cc:3055] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:18.024583  4211 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
W20260709 12:28:18.056623  4208 log.cc:927] Time spent T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:28:18.059646  4125 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:18.038600  4216 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:18.067376 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 43987.2us
I20260709 12:28:18.124891 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:17.991601  4005 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.041667  4074 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:18.094177 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 46346.5us
I20260709 12:28:18.127166 26018 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
W20260709 12:28:18.095651 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 223781us
I20260709 12:28:18.026307  3965 consensus_queue.cc:1048] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.050s
W20260709 12:28:18.120364 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 221072us
I20260709 12:28:18.068631  4214 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:18.133539 26094 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:18.129992 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:18.135017 26201 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.080766  4224 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:18.132133 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:18.093086  4207 log.cc:927] Time spent T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:28:18.042593  4217 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:28:18.158006 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.123791  3584 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.162752 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.079459  4043 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:18.130985 26168 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.185271 26168 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.081534  3971 raft_consensus.cc:740] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:18.080582  4222 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.123850  4218 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:28:18.190070 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 193525us
I20260709 12:28:18.091931  4117 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.172420  4125 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:18.171185  4010 raft_consensus.cc:493] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
W20260709 12:28:18.218981 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77410.6us
I20260709 12:28:18.183341  4005 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.157529  4213 consensus_queue.cc:1048] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.061s
W20260709 12:28:18.222815 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91865.9us
I20260709 12:28:18.140741  3775 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:18.213192 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 48110.4us
W20260709 12:28:18.226164 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68772.6us
I20260709 12:28:18.228438 26201 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:18.182930  4215 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.141705  3578 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:18.287676  4251 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:18.288486  4251 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:18.287737  3578 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:18.289029  4235 log.cc:927] Time spent T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:28:18.289428  4251 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.225566  3822 raft_consensus.cc:2749] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:18.241791 26168 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.239270  4125 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:18.243333  4236 log.cc:927] Time spent T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:28:18.250603  3904 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.298202  4193 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.232134  4218 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.189205  3584 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.270907 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:18.232692  3855 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:18.230535  4105 raft_consensus.cc:2749] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:28:18.286717  4242 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.294042 26168 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.218184  4043 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.264451  4216 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.298894  4193 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:18.238946  4010 raft_consensus.cc:515] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.269841  4196 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:18.237128  3971 consensus_queue.cc:260] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.319054  3584 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.324651  3578 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.315923  4125 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:18.325539 26812 debug-util.cc:398] Leaking SignalData structure 0x7b0800331d00 after lost signal to thread 24865
W20260709 12:28:18.352530 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4de0 after lost signal to thread 25986
I20260709 12:28:18.353066 26203 raft_consensus.cc:2410] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 33 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
W20260709 12:28:18.353269 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002cbc00 after lost signal to thread 26033
W20260709 12:28:18.353574 26812 debug-util.cc:398] Leaking SignalData structure 0x7b0800342500 after lost signal to thread 26034
W20260709 12:28:18.353835 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002cb880 after lost signal to thread 26078
W20260709 12:28:18.354799 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08002cbac0 after lost signal to thread 26153
W20260709 12:28:18.355808 26812 debug-util.cc:398] Leaking SignalData structure 0x7b080034f780 after lost signal to thread 26227
W20260709 12:28:18.356134 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08003311a0 after lost signal to thread 26228
I20260709 12:28:18.272001  3775 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:18.351442 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 96876.6us
I20260709 12:28:18.309602 26129 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:18.341038  4253 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:18.340592 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 162873us
I20260709 12:28:18.257313  4074 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:18.333822 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92142.6us
W20260709 12:28:18.384590 26168 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.385166 26168 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:18.360503  3971 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:18.357451 26812 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7a40 after lost signal to thread 26815
I20260709 12:28:18.340106  4244 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:18.335947  4242 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.291222  4224 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.300463  4215 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.326268  3855 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.369884 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.344249  4216 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.392464 26129 raft_consensus.cc:2468] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:28:18.397969  4231 log.cc:927] Time spent T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 12:28:18.401106 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.410923 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:18.402200 26168 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:18.370450  4124 consensus_meta.cc:287] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Time spent flushing consensus metadata: real 0.609s	user 0.006s	sys 0.010s
I20260709 12:28:18.404706  3943 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:18.412542 26168 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.413393 26198 raft_consensus.cc:2393] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:18.386376  3584 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.416332  3579 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.413226  3970 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:18.417105  3775 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.418243  4215 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.406821  4244 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.440435  4254 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.441036  4254 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.445590  4254 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.445788 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.446491 26198 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:18.422628  4216 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.424266  4242 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:18.428020 26168 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.447822 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:18.433575  4245 log.cc:927] Time spent T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:28:18.438881  3579 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:18.459398 24862 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 26079)
I20260709 12:28:18.434988  3970 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.476842 26203 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:18.477319  4205 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:18.477945  4205 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:18.478910  4205 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.480942 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.482334 26203 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:18.484534 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:18.485363 26203 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:18.487869  4218 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.494194 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:18.495178 26201 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:18.499785 26755 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.503826 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:18.504774 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:18.505129 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:18.505877 26129 raft_consensus.cc:2410] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 28 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:18.506182 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.506704 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.507321 26196 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:18.507532  4043 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.508329 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.508932 26194 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:18.511855  4221 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:18.512495  4221 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.512830 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.514675  4221 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.518868 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.519933 26200 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:18.526196 26200 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:28:18.528318 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:18.512101 26201 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 25.
I20260709 12:28:18.531165 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:18.535892 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.536749 26787 raft_consensus.cc:2410] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 for term 32 because replica has last-logged OpId of term: 31 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 23.
W20260709 12:28:18.532042 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.545408 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.519021 26019 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Timed out: RequestConsensusVote RPC to 127.24.71.131:38261 timed out after 2.184s (SENT)
I20260709 12:28:18.554435 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:18.556620 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:18.570845 26169 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.441592  4244 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:18.574175 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 55196.8us
I20260709 12:28:18.575042 26019 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:18.586354 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 200982us
I20260709 12:28:18.586516 26195 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:18.588183 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.589015 26195 raft_consensus.cc:2393] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:18.590376 26203 raft_consensus.cc:1240] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 32. Current term is 33. Ops: [32.19-32.19]
I20260709 12:28:18.480177  4205 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.434474  3584 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.595678 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:18.589677 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.597790 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:18.568951  3970 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.470225  4216 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.593349 26124 raft_consensus.cc:1240] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 30. Current term is 31. Ops: []
W20260709 12:28:18.609573 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 177383us
I20260709 12:28:18.609757 26127 raft_consensus.cc:1240] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 24e61c63097e4d3dbb149050e6fcc997 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:18.581519  4220 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:18.458437  3579 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.593490  4071 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:18.603303  4194 raft_consensus.cc:2804] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:18.635147 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.635417 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.641285 26784 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:18.635887 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.642053 26785 raft_consensus.cc:1813] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Handling vote request from an unknown peer a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:18.642843 26785 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:28:18.601610  4117 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.637977 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b765338c61436481fe416f17637ab2" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:18.609002  4242 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:18.638381 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ad94af29be4656bf3ce8a023d19f7d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.635607 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.645870  4194 raft_consensus.cc:697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:18.646039 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.665012 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.665235 26791 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:18.665664 26785 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:18.647185 26782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.665939 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.666543 26782 raft_consensus.cc:2410] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:28:18.666666 26784 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:18.648492 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955755ce63f743b9a01d5871aff94f6c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.657943 26788 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 25.
I20260709 12:28:18.668196  3970 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.646502 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:18.669569 26786 raft_consensus.cc:2376] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34. Re-sending same reply.
W20260709 12:28:18.650766  4239 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.434s	user 0.002s	sys 0.004s
I20260709 12:28:18.670500 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.644358  4215 consensus_queue.cc:1059] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:28:18.671219  3970 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.672966 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.673179  4010 leader_election.cc:290] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.673632 26791 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:18.676249 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.676599  4194 consensus_queue.cc:237] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.677644  3855 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.671985  3970 raft_consensus.cc:3055] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:18.681685  3970 raft_consensus.cc:740] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:18.682778  3970 consensus_queue.cc:260] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.684122  3970 raft_consensus.cc:3060] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:18.689844 26788 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 20. (term mismatch)
I20260709 12:28:18.690290 26788 pending_rounds.cc:85] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 19
W20260709 12:28:18.681345 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.690573 26788 pending_rounds.cc:107] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 28.20
I20260709 12:28:18.700637 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.701211  3584 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.701503 26783 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:18.705502  3579 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.707727 26788 raft_consensus.cc:2889] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:18.633071  4216 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.710520  4216 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:18.719586  4222 consensus_meta.cc:287] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.004s
I20260709 12:28:18.730132  4222 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:18.769102 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.789367 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.792869 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.794073 26199 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 27.
I20260709 12:28:18.775617 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.795396  3970 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.796900 26019 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.796793  3970 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.801270 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.802321  3970 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.804847  3579 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:18.805430  3579 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.805881  3579 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:18.808568 26167 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.808849  4125 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:18.823119  4222 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.824527  4230 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:18.828927 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bede0bf927e45a7b18a1bf2ad0bb594" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.830322 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.837667 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:18.838243 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.838656 26194 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.843091 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:18.844888 26202 raft_consensus.cc:2410] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 29 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:28:18.845077  3855 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:18.846670  3855 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.847267  3855 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.849045 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:18.844903  4278 log.cc:927] Time spent T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 12:28:18.865288  4280 log.cc:927] Time spent T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:28:18.818192 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 198112us
I20260709 12:28:18.873346  3578 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.875852  3578 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:18.879629 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 68996.1us
I20260709 12:28:18.888504  4224 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.889772  4224 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.893216 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:18.894574 26788 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:18.898450 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 308720us
I20260709 12:28:18.900812  4103 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:18.901893  4103 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:18.904910 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:18.906028 25945 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 290) took 99248 ms (client timeout 15000 ms). Trace:
I20260709 12:28:18.906240  4233 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.907235  4233 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.908569  4254 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.909403  4254 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:18.906245 26124 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:18.917630  4224 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.920003  4233 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:18.922245 26168 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:18.922972 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.923259 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.925549  4214 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:18.925113 26788 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:18.926460  4254 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:18.928047 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:18.927897 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:18.929322 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.930728  4283 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:18.931118 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34019ceec5a4e4e877bed7351d1d91d" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.931476  4283 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:18.932602  4283 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:18.933199 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.933998 26783 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:18.937917  4082 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:18.938565  4082 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.939011  4082 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:18.899878 26094 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:18.930258  4233 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:28:18.951388 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 51784.1us
I20260709 12:28:18.960891 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:18.961303 26000 auto_rebalancer.cc:288] Could not retrieve cluster info: Not found: tserver 9e23389325a6411c9b0634f4f6af54c3 not available for placement
I20260709 12:28:18.964085  4264 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:18.965809 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 43647.1us
I20260709 12:28:18.956674 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:18.946550  3578 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:18.952911  4233 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.974709 26194 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:18.903901  4103 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:18.977277  4233 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:18.975481  4254 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:18.987350 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 187943us
I20260709 12:28:18.987067 26788 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 27.
I20260709 12:28:18.991906 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:18.993161 26121 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:18.991334 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:18.996759  4264 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:18.997360  4264 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.997874  4264 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:19.000381 26094 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.001657  4105 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:19.004618 26756 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.017297 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:19.019670  4193 consensus_meta.cc:287] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.669s	user 0.018s	sys 0.004s
I20260709 12:28:19.024833  4193 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.026189  4267 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:19.032963  4240 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.045022 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.034418 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.032779 26167 leader_election.cc:304] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.037640  4264 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.049306 26093 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.056819  4194 raft_consensus.cc:2749] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:19.057016  4218 consensus_queue.cc:1059] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:19.057461  3970 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.042901  3943 consensus_meta.cc:287] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: Time spent flushing consensus metadata: real 0.614s	user 0.014s	sys 0.006s
I20260709 12:28:19.062178 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:18.906690 25945 rpcz_store.cc:269] 0709 12:26:39.657188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.660166 (+  2978us) service_pool.cc:224] Handling call
0709 12:26:40.972082 (+1311916us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b743adf3a 00007f5b7a163e9c 00007f5b7a26204e 00007f5b787232a4 00007f5b78723232 00007f5b78723161 00007f5b787230dc 00007f5b78722392 00007f5b726e048c 00007f5b726df8c6
0709 12:28:18.905684 (+97933602us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:17.873185 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:17.989250 (+116065us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:17.991196 (+  1946us) write_op.cc:435] Acquired schema lock
0709 12:28:17.994589 (+  3393us) tablet.cc:662] Decoding operations
0709 12:28:17.995158 (+   569us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:17.995219 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:17.995257 (+    38us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:28:17.995522 (+   265us) tablet.cc:701] Row locks acquired
0709 12:28:17.995545 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:28:17.995780 (+   235us) write_op.cc:270] Start()
0709 12:28:17.995907 (+   127us) write_op.cc:276] Timestamp: P: 1783600097995727 usec, L: 0
0709 12:28:17.995930 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:28:17.996692 (+   762us) log.cc:844] Serialized 987 byte log entry
0709 12:28:18.448639 (+451947us) op_driver.cc:464] REPLICATION: finished
0709 12:28:18.475529 (+ 26890us) write_op.cc:301] APPLY: starting
0709 12:28:18.475622 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:18.478436 (+  2814us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:18.478537 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:18.484992 (+  6455us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:18.486065 (+  1073us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:28:18.486125 (+    60us) write_op.cc:312] APPLY: finished
0709 12:28:18.486333 (+   208us) log.cc:844] Serialized 33 byte log entry
0709 12:28:18.487661 (+  1328us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:18.487780 (+   119us) write_op.cc:454] Released schema lock
0709 12:28:18.487827 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:18.490619 (+  2792us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:18.490649 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":144742,"reported_tablets":47,"spinlock_wait_cycles":76318592,"child_traces":[["op",{"apply.queue_time_us":26640,"apply.run_cpu_time_us":4857,"apply.run_wall_time_us":15060,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12499,"num_ops":2,"prepare.queue_time_us":191181,"prepare.run_cpu_time_us":13502,"prepare.run_wall_time_us":212958,"raft.queue_time_us":148429,"raft.run_cpu_time_us":8441,"raft.run_wall_time_us":26937,"replication_time_us":452527,"thread_start_us":261878,"threads_started":4,"wal-append.queue_time_us":183640}]]}
I20260709 12:28:19.067351  4240 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.040963  4254 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:28:19.072669 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 95041.7us
I20260709 12:28:19.059810 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.079156 26125 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 29: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:19.081346 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.082291 26121 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:19.079099  3970 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.086720  3798 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:19.088701 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 54559us
I20260709 12:28:19.089747 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:19.090698  3798 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:19.091058  3798 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.091368  3798 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:19.092125 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:19.093477  4216 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:19.093946  4216 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:19.094187 26167 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.094395  4216 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:19.058463  4218 raft_consensus.cc:3055] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:19.097710  4218 raft_consensus.cc:740] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:19.098862  4218 consensus_queue.cc:260] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.100409  4251 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:19.100522  4194 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:19.101001  4251 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.101192  4194 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:19.101421  4251 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:19.102275  4194 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.104682 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.108785  3578 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:19.109370  3578 raft_consensus.cc:493] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.109807  3578 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:19.109676 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:19.118124  3971 consensus_meta.cc:287] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.015s
I20260709 12:28:19.119134 26790 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:19.124353  3855 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:19.125825 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50839.6us
I20260709 12:28:19.060930  4082 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.129196  3855 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.091238  4240 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.132043  4264 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.134090  4240 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:19.137010  3943 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:19.138336 26198 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:19.138818  4218 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:19.088052  3970 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.096050 26167 leader_election.cc:304] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:19.153038 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 176932us
W20260709 12:28:19.159358 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 92955.1us
I20260709 12:28:19.074692  4254 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.151304  4240 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.164100 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.172493 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.173094 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.174728 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.176695 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.177681 26129 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 34.
I20260709 12:28:19.164393  3943 raft_consensus.cc:2749] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:19.177963 26168 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.162755  4105 raft_consensus.cc:2749] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:19.178889 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 139636us
I20260709 12:28:19.164883  4254 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:19.165714  4082 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.187258  3971 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:19.190904 26019 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.192059  3855 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:19.192674  3855 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:19.193177  4224 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:19.193785  3855 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.195928  4233 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.196573 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
W20260709 12:28:19.196612 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.197331 26127 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:19.179600  4193 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:19.201929 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.202646 26790 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:19.208424  4103 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.209245  4103 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.211517  4103 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.216462  4219 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:19.217296  4219 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.220403  4219 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.220876  4240 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.223277  4240 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.223882  4240 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.224625 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 128701us
I20260709 12:28:19.224568  4240 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.229475 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:19.230055  4240 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.230378  3943 raft_consensus.cc:493] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 24e61c63097e4d3dbb149050e6fcc997)
I20260709 12:28:19.231209 26200 raft_consensus.cc:1240] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:19.231261  3579 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.230242 26198 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 29: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:19.230988  3943 raft_consensus.cc:515] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.234625  3943 leader_election.cc:290] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.237474  4216 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.239972  4216 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.240906 26790 raft_consensus.cc:2468] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:19.243525 26093 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.211673  3578 raft_consensus.cc:515] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:19.256559 26094 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.243944  4233 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.259583 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.260619 26198 raft_consensus.cc:2393] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:19.257638 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.253515  4105 raft_consensus.cc:2804] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:28:19.272953 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 108979us
I20260709 12:28:19.273723 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:19.273988 26756 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.274044  4105 raft_consensus.cc:697] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:19.275300  4105 consensus_queue.cc:237] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.281538 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.283574 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.270562  4233 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:19.285218  3970 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:19.303620 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:19.310413 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.293486  4283 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:19.318383 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 70944.3us
I20260709 12:28:19.293936  4251 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.325650 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:19.305737  3798 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.331967 26194 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:28:19.336887 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62947us
I20260709 12:28:19.336817 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.338480 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.346887  3578 leader_election.cc:290] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.351892 26202 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:19.356878  4251 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.358052 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.358644 26790 raft_consensus.cc:3060] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:19.358676 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.359494  4244 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:19.360001  4244 raft_consensus.cc:493] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.335599  4254 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.360373  4244 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:19.362120  4254 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.362942 26125 raft_consensus.cc:1240] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:28:19.371434  4224 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.353579 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:19.353837  4283 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.354012 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:19.354735 26094 leader_election.cc:336] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.355471 26129 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:19.355679 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.359607  3798 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.374668 26194 raft_consensus.cc:1240] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:19.352728 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.376298 26199 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:19.378274 26094 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:19.341774  3579 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.379438 26196 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 32.
I20260709 12:28:19.388902 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.388965 26167 leader_election.cc:304] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.372432  4224 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.396799 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:19.399147 26120 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:19.398525 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 63628.1us
I20260709 12:28:19.400347 26790 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
W20260709 12:28:19.401647 26168 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.402012 26093 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.402123 26168 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:19.403512 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.390729  4254 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:19.405647 26196 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:28:19.404075 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.400441  4216 raft_consensus.cc:2749] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:19.409662  4283 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:19.410001  3798 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.409721 26129 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:28:19.410737  3798 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.411393 26020 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:19.412562 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 68074.5us
I20260709 12:28:19.412894  3798 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.413321 26020 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.414247  3855 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:19.416352 26167 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Timed out: RequestConsensusVote RPC to 127.24.71.132:39865 timed out after 1.745s (SENT)
I20260709 12:28:19.416878 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.418147 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.416368  4242 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.420754  4242 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.426048  4242 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:19.421914  4105 raft_consensus.cc:2804] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:19.427628  4105 raft_consensus.cc:697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:19.423391 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 81336.3us
W20260709 12:28:19.429086 26094 leader_election.cc:336] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.429523 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.429674  4105 consensus_queue.cc:237] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:19.431031 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.431516  4254 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:19.432614 26020 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.413480  3798 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.440204 26199 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
W20260709 12:28:19.434096 25950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 294) took 193529 ms (client timeout 15000 ms). Trace:
I20260709 12:28:19.436596 26019 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.437619 26120 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:28:19.438401 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:19.442358 25950 rpcz_store.cc:269] 0709 12:25:05.904734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.904922 (+   188us) service_pool.cc:224] Handling call
0709 12:28:19.433876 (+193528954us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1690843,"reported_tablets":49}
I20260709 12:28:19.444994 26757 leader_election.cc:304] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.442031  4283 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:19.447937 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:19.441357  4242 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:19.452113 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac7044e09a64594b8fcd962f1d387b7" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:19.452314  4009 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.452916 26195 raft_consensus.cc:3060] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:19.452243  4256 raft_consensus.cc:2804] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:19.453455 26198 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:19.453511 26020 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.453075  4009 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.458261 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.458716  4256 raft_consensus.cc:697] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:19.459707 26124 raft_consensus.cc:2410] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 32 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 12:28:19.460196 24862 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 26079)
I20260709 12:28:19.461359  3579 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:19.464763 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.466208  4009 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:19.466814  3579 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:19.467463 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.468259 26124 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:19.468808  4224 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.468588  4240 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:19.470219  3855 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:19.470333 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.472608 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:19.473376 26756 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.473521  4216 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:19.473996  4216 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.474344  4216 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:19.476701 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.477514 26783 raft_consensus.cc:2393] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 28: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:19.471462  4009 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:19.478289  4009 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.478816  4009 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:19.456146  4218 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:19.481441 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.473277  3855 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.483947  4256 consensus_queue.cc:237] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.24, Last appended by leader: 24, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.484349  3855 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.486917 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46aa595bcb23423fb51ff268d34a3bc7" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:19.446158  3798 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:19.494776 26195 raft_consensus.cc:2468] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:28:19.500057 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:28:19.500756 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 99600.4us
I20260709 12:28:19.502523  4240 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:19.506666 26124 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:28:19.509600 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.511240  4216 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.513911  4216 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:19.515719  3855 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.516260 26785 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:19.518221 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 64781.1us
W20260709 12:28:19.519098 26168 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.520835  4240 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.521618  4240 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.527529  4224 consensus_queue.cc:1059] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:19.528738  4224 raft_consensus.cc:3055] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:19.529306  4224 raft_consensus.cc:740] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:19.530411  4224 consensus_queue.cc:260] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.531176 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:19.531716  4224 raft_consensus.cc:3060] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:19.531867 26786 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:19.532781 26784 raft_consensus.cc:1240] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:19.532980 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.533537 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.534461 26789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:19.534912 26783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:19.533627 26786 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:19.533816 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:19.540014 26167 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.540426  4265 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:19.541205  4283 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.541575 26203 raft_consensus.cc:1275] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:28:19.543363 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:19.543983 26790 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:19.548105 26167 leader_election.cc:304] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.549468  4082 raft_consensus.cc:2749] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:19.552186 26786 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 34.
I20260709 12:28:19.553601  3579 consensus_queue.cc:1059] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:19.554608  4219 raft_consensus.cc:3055] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:19.555099  4219 raft_consensus.cc:740] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:19.556520 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.557389  4219 consensus_queue.cc:260] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.559303  4219 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:19.564749 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45691.8us
I20260709 12:28:19.544685 26020 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.538074  4240 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:19.566067 26168 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.575934 26790 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
W20260709 12:28:19.577708 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 146723us
I20260709 12:28:19.582804  4240 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.004s
W20260709 12:28:19.592623 26168 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.593428 26168 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.599815 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.601135  4265 raft_consensus.cc:2749] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.596470  4283 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.603508 25943 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 11 to 12, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 12 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:19.604341  4244 raft_consensus.cc:515] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.591557  3855 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.630698  3855 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.634277  3855 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.606387 26018 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.605168  4256 consensus_queue.cc:1048] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.008s
I20260709 12:28:19.613919 26786 raft_consensus.cc:1275] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:19.640352  4240 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:19.606097 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:28:19.640769 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48189.9us
W20260709 12:28:19.648916 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 104307us
I20260709 12:28:19.659042 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.658351 26125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:19.657084 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c3d95514eb45178f2e5aa92e868e36" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:19.654110 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.660290 26125 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:28:19.663033 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.661616 26168 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.678882  4219 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:19.680853 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 74534.8us
W20260709 12:28:19.681352 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 68189.6us
W20260709 12:28:19.681951 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.682245 26094 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.682775 26094 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:19.683693  4105 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:19.687812  4283 consensus_queue.cc:1048] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:19.690317 26790 raft_consensus.cc:1240] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Rejecting Update request from peer a981c9a7972c4e99a0903924c64fbb5c for earlier term 34. Current term is 35. Ops: []
I20260709 12:28:19.690163 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:19.691051  3579 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.676657  3855 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.694893  3579 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.693562 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:19.697798 26198 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:19.699293 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.700599 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.700150  4240 consensus_queue.cc:1059] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:28:19.696384  4194 consensus_queue.cc:1059] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.007s
I20260709 12:28:19.703564  4194 raft_consensus.cc:3055] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:28:19.704416  4194 raft_consensus.cc:740] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Follower/Learner. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:19.705380  4194 consensus_queue.cc:260] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.708922  4244 leader_election.cc:290] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.709830  3579 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.730759 26203 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:19.733119 26197 raft_consensus.cc:1240] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:28:19.719907  4194 raft_consensus.cc:3060] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:19.740044 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:19.751235 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 55005us
I20260709 12:28:19.752926  4307 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:19.755177 26124 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:28:19.736634  3579 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.759063  4242 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:19.760011 26195 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:19.757097 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.764166 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.766278 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:19.768994 26203 raft_consensus.cc:2393] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:28:19.771612 26756 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.772190 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:19.770838 26094 leader_election.cc:336] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.775527 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.772635 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.777424 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:19.778054 26129 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:19.778175  4265 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:19.782104 26124 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:19.787514 26195 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:19.788425 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:19.789042  4304 log.cc:927] Time spent T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:19.792919 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.793807 26198 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 34.
I20260709 12:28:19.755977  3855 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:28:19.757668  4219 consensus_queue.cc:1059] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.016s
W20260709 12:28:19.797096 26093 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.799665 26094 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:19.804704 26128 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:19.805709 26121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:19.807749 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56956us
I20260709 12:28:19.809145 26785 raft_consensus.cc:2468] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 26.
I20260709 12:28:19.811383 26129 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:19.811827 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.813606 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.815008 26123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.817054 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.809659 26019 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:19.822834 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.811322  4218 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:19.859689  4308 raft_consensus.cc:3055] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:19.860145  4308 raft_consensus.cc:740] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:19.860983  4308 consensus_queue.cc:260] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:19.827909  4305 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:28:19.863813 26125 raft_consensus.cc:2468] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 35.
I20260709 12:28:19.865042 26785 raft_consensus.cc:1275] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:19.867208  3798 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:19.870972  4309 raft_consensus.cc:3055] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:19.871533  4309 raft_consensus.cc:740] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:19.872625  4218 consensus_queue.cc:1048] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:19.872611  4309 consensus_queue.cc:260] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.874271  4309 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:19.878890  4308 raft_consensus.cc:3060] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:19.893556  4009 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.895327 26787 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:28:19.896646  4009 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:19.897305 26756 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.898069 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:19.899068 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 74178.8us
W20260709 12:28:19.899865 26756 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.901446 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 150968us
I20260709 12:28:19.911406 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:19.912261 26120 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:19.912202 26018 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:19.829262  4310 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:19.829994  4240 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:19.917455  4240 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.826294 26019 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:19.832729 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61924.6us
I20260709 12:28:19.824827  4219 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:28:19.852540 26791 raft_consensus.cc:1240] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:19.855561 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.828617 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.909065 26757 leader_election.cc:304] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.906947  3970 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:19.772948  3579 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.936312  3579 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:19.917903  4240 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:19.930444 26094 leader_election.cc:336] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.940451 26094 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:19.941851 26094 leader_election.cc:336] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.942322 26094 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:19.946776  4218 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:19.947508  4218 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:19.932827  4244 raft_consensus.cc:2804] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:28:19.949018  4244 raft_consensus.cc:697] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:19.950037  4244 consensus_queue.cc:237] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:19.939903 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.960294 26756 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.961750 26167 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:19.962656  4283 raft_consensus.cc:2804] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:19.963171  4283 raft_consensus.cc:697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:19.963755 26120 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:19.964074  4283 consensus_queue.cc:237] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.966274 26757 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:19.967186  3970 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:19.967728  3970 raft_consensus.cc:697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:19.968930 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 142756us
I20260709 12:28:19.968637  3970 consensus_queue.cc:237] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:19.970964 26019 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.971330  4219 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:19.971825  4219 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:19.972590  4309 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:19.973120  4314 consensus_queue.cc:1059] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:19.973737  4219 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.974283  4313 raft_consensus.cc:3055] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:19.974742  4313 raft_consensus.cc:740] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:19.975591 26195 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 23. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:28:19.975714  4313 consensus_queue.cc:260] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:19.977576 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 149092us
W20260709 12:28:19.977957 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 65849.2us
I20260709 12:28:19.983801 26791 raft_consensus.cc:1275] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:19.983654  4224 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:19.979354 26018 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:19.985359 26125 raft_consensus.cc:1275] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 29 index: 23. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:28:19.987612 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:19.971036  3579 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:19.994678  4309 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:19.996021  4264 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.000s
W20260709 12:28:19.997260 26168 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.997864 26018 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.997756 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:19.988433  4310 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:20.001042  4194 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.992645  4307 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:20.012425  4194 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:20.013232  4194 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:19.998592  3855 consensus_queue.cc:1059] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:28:20.025218 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:19.990211  4105 consensus_queue.cc:1048] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.018s
I20260709 12:28:20.024883  4309 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:20.019025  3855 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.030332  4194 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:20.054334 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 67929.3us
I20260709 12:28:20.056820 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:20.057806 26196 raft_consensus.cc:2410] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:28:20.031502  4307 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:20.060590 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:20.076910 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 89395.9us
W20260709 12:28:20.097046 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 153219us
I20260709 12:28:20.103039 26020 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:20.109238 26195 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
W20260709 12:28:20.117442 26168 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:20.128554 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 50579.2us
W20260709 12:28:20.128861 26094 leader_election.cc:336] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.131469 26094 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.104173  4219 raft_consensus.cc:3055] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:28:20.132624 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 133113us
I20260709 12:28:20.131784 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:20.133083 26168 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.070380  4240 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:20.136283 26168 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:20.137899 26020 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:20.139534 26168 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.139987 26168 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:20.132869  4219 raft_consensus.cc:740] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:20.147910 26020 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.159361 26125 raft_consensus.cc:1240] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Rejecting Update request from peer 8fd5643e7b494fa79c3d0aaf1e6e8152 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:28:20.161027 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:20.137528  3579 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:20.118139  3855 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.145722  4214 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:28:20.180227 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61757.1us
I20260709 12:28:20.146646  4251 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:20.171393  4309 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:20.238752 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 67961.1us
I20260709 12:28:20.173622  4310 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:20.220069  4329 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:20.249780  4219 consensus_queue.cc:260] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.195566  4256 consensus_queue.cc:1048] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.055s
I20260709 12:28:20.250864  4219 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:20.246766  4318 log.cc:927] Time spent T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:28:20.252322  4283 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:20.260501 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 105343us
I20260709 12:28:20.246686  4331 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:20.260804  4325 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:20.296069 26094 leader_election.cc:336] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.296682 26094 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.317466  4329 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.318269 26125 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 19. (term mismatch)
I20260709 12:28:20.318768 26125 pending_rounds.cc:85] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 18
I20260709 12:28:20.318264 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:20.321127 26127 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:28:20.317497 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:20.320910 26756 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.325327 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.195468  4307 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.318167  4329 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.319036 26125 pending_rounds.cc:107] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 32.19
I20260709 12:28:20.337481 26125 raft_consensus.cc:2889] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:20.341179 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:20.353277  4330 log.cc:927] Time spent T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:28:20.339054  4336 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:20.360563  4240 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:20.360958  4323 log.cc:927] Time spent T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:28:20.262674  3579 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:20.362522  4332 log.cc:927] Time spent T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 12:28:20.369843  4324 log.cc:927] Time spent T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
W20260709 12:28:20.371543 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 101781us
W20260709 12:28:20.373713 26094 leader_election.cc:336] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.374150 26094 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.377944  4329 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:20.384794  4316 log.cc:927] Time spent T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:28:20.386574  4341 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:20.388546 26756 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.389017 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:20.390215 26756 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.390602 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:20.391723 26756 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.392495  3971 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:20.393425  3971 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:20.333709  4325 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.400417  4325 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:20.406934  4313 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:20.412741  4254 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:20.350960  3855 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:20.392086 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:20.423326 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49675.7us
W20260709 12:28:20.424145 26094 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.424576 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.425333  4254 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:20.426767 26094 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.427073  4244 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:28:20.430264 26756 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.430879  3855 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:20.430701 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.433336  4244 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:20.434410 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:20.434530 26756 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.435022 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:20.435119 26094 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.449812  4256 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:20.451089 26756 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:20.451985 26756 leader_election.cc:336] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:20.460762 24862 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 26079)
I20260709 12:28:20.470777 26127 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
I20260709 12:28:20.478082 26790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:20.478494 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:20.480093 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:20.481622 26020 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:20.482621  4340 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:20.483160  4340 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
W20260709 12:28:20.488425 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 450020us
W20260709 12:28:20.493566 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 354022us
W20260709 12:28:20.494517 26168 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.495061 26168 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:20.497229  4351 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:20.498638  4310 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:20.484140  4340 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:20.528136 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 46622.6us
I20260709 12:28:20.532323 26019 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.535902 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:20.536872  4310 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:20.537267  3579 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:20.537832  3579 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:20.538319  3579 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:20.567996 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 103783us
I20260709 12:28:20.569410 26128 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
W20260709 12:28:20.570988 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 91873.6us
I20260709 12:28:20.579105 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:20.580264  4340 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:20.581090 26167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 302253us
I20260709 12:28:20.581833 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:20.581969 26199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:20.583407 26195 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:28:20.583660 26198 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:28:20.584419 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:20.584959 26201 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:20.586395 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:20.594487  4307 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:20.585145  4219 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
W20260709 12:28:20.626408 26168 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.633507 26168 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:20.641897  3579 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:20.645126 26198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:20.645941 26198 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:20.645588  4335 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:20.669919  4254 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.688122  4254 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.702812 26201 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 33.
W20260709 12:28:20.712960 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 87032.8us
I20260709 12:28:20.738765 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.701028  4254 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:20.707602  3855 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.752002 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:20.757014 26785 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:20.757050 26094 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:20.755515 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:20.732328  4219 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:20.760383 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:20.764145 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 140136us
I20260709 12:28:20.764079 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:20.778554 26195 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 35 index: 22. (term mismatch)
I20260709 12:28:20.779171 26195 pending_rounds.cc:85] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting all ops after (but not including) 21
I20260709 12:28:20.779488 26195 pending_rounds.cc:107] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: Aborting uncommitted NO_OP operation due to leader change: 34.22
I20260709 12:28:20.779808 26195 raft_consensus.cc:2889] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:20.757666  3579 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:20.784736 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:20.761081  4307 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:20.789475 26195 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:20.784403  4105 consensus_queue.cc:1059] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:28:20.773505  4313 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:28:20.788267  4350 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.802991  4359 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.806219  4327 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:28:20.768931  4219 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.770658  3855 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:20.816037  4369 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:28:20.819373  4368 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:20.829708  4368 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.830845  3855 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:20.831980  4368 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:20.802598  4218 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:28:20.840898 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55539.4us
I20260709 12:28:20.819036  4350 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:20.820703  4359 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:20.823525  4105 raft_consensus.cc:3055] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:20.863338  4105 raft_consensus.cc:740] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:20.827244  4219 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:20.864424  4105 consensus_queue.cc:260] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:20.861987 26787 raft_consensus.cc:1275] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:20.866110  4105 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:20.861990 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:20.885505 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:20.894356 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 137365us
W20260709 12:28:20.859503  4334 log.cc:927] Time spent T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:28:20.896426 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:20.897414 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:20.899401 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:20.896348 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:20.903173  4193 consensus_queue.cc:1048] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 12:28:20.913245 26196 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:20.927151  4326 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:20.935982  4350 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:20.941361 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:20.942080 26196 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:28:20.915373  4218 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:20.943260  4218 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:20.943625  4218 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:20.956959 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 62337.4us
W20260709 12:28:20.958843 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:20.959525 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:20.960022 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:20.967571 26195 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:20.968703 26203 raft_consensus.cc:1240] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:20.974475  4359 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:20.974741 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 50390.1us
I20260709 12:28:20.990965 26785 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:28:20.991494 26784 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:28:21.001309 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.001950  4346 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:21.002748  4346 consensus_queue.cc:1243] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Peer 24e61c63097e4d3dbb149050e6fcc997 log is divergent from this leader: its last log entry 31.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:21.006911  4346 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:28:21.009563 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46421.8us
I20260709 12:28:21.011314 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
W20260709 12:28:21.011920 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.012504  4105 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:21.013089  4105 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.023917  4105 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:21.025482 26200 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
W20260709 12:28:21.014616 26756 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.034471 26785 raft_consensus.cc:1240] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Rejecting Update request from peer 9e23389325a6411c9b0634f4f6af54c3 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:21.036363 26194 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:28:21.036401  4378 consensus_queue.cc:1059] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:21.037681  4378 raft_consensus.cc:3055] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:21.038064  4378 raft_consensus.cc:740] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:21.037696 26197 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:28:21.039104  4378 consensus_queue.cc:260] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.040663  4378 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:21.056825 26019 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:21.057744 26094 leader_election.cc:336] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.061820 26787 pending_rounds.cc:85] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Aborting all ops after (but not including) 18
I20260709 12:28:21.062157 26787 pending_rounds.cc:107] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: Aborting uncommitted NO_OP operation due to leader change: 31.19
I20260709 12:28:21.062403 26787 raft_consensus.cc:2889] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:21.059894 26094 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:21.074023 26122 raft_consensus.cc:1275] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 35 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:28:21.064522  4331 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.068161  3855 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:21.083913 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.106139  4374 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:21.107831 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 93380.5us
I20260709 12:28:21.095129  4331 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.117270  4331 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.117533 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.117868  4240 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:21.118420 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:21.118479  4240 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.122844  4240 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.097626  3855 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:21.127229  3855 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:21.108975 26094 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.082593  4309 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:28:21.129215 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:21.138607 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.137068 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.139792 26195 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:21.097522  4219 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:28:21.153837  4338 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:21.154373  4338 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.154757  4338 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:21.164587 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 55648.3us
W20260709 12:28:21.165513 26094 leader_election.cc:336] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.165997 26094 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:21.167280 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.167382  4251 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:21.170845  4218 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.138622  4224 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 12:28:21.103642  4105 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.179141  4105 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.179922  4309 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.180562  4309 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.180962 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:21.181612 26785 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:21.183188  4309 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.195407  4218 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.203055  4338 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:21.204147 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 50837.8us
I20260709 12:28:21.206511 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:21.208204 26122 raft_consensus.cc:1275] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:21.207024 26785 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:21.207674  4244 consensus_queue.cc:1048] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:28:21.215487 26094 leader_election.cc:336] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.215224 26093 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:21.214887 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:21.226823 26094 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:21.230255 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.234474 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.234225  4338 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.239773 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.232321  4329 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:21.212139  4009 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:21.232713  4218 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:21.243244  4218 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:21.244632  4218 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.231988  4251 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:21.233912  4219 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:21.257961 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:21.258661 26785 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 31
W20260709 12:28:21.261953 26095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:21.265380 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49876.1us
W20260709 12:28:21.266453 26094 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.268172 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.272130  4378 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:21.283792 26094 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.290508 25947 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.292467 25947 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 18 to 21, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 21 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.294518 25947 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 21 to 24, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 24 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:21.302500 26094 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.298761 25947 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:21.304960 26785 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
I20260709 12:28:21.306838 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.307533 26197 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:21.309109 26018 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 
I20260709 12:28:21.311620 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:21.312955 26197 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:21.315618  4325 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:21.328377  4325 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:21.329694  4325 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.329249 26019 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:21.333113  4327 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:21.333564  4327 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.333985  4327 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:21.327234 26094 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.353986 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:21.362089 26168 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.365579 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:21.372946 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 45766.4us
W20260709 12:28:21.374735 26094 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.374383 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:21.381314 26197 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 31.
W20260709 12:28:21.381601 26168 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.387079 26126 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:28:21.383682 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:21.394009 26201 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:28:21.372208  4310 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.393369 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:21.394150 26168 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:21.395588 26195 raft_consensus.cc:3060] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:21.405964 26094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.407820 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:21.396052  4354 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:21.413153 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
W20260709 12:28:21.416994 26094 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:21.417685 26094 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:21.418284 26094 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:21.418824 26094 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.419050 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.421792 26195 raft_consensus.cc:2468] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:21.433126 26201 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
I20260709 12:28:21.398037  4310 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:21.437690 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet de16ce1800b64c3c94c918aa6e9cae3d: Illegal state: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:21.442058  4310 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.422015  4377 consensus_queue.cc:1059] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:28:21.422037  4216 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:21.447783 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.405438  4349 consensus_queue.cc:1048] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
W20260709 12:28:21.444142 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet 13d33ec3d2d94eeda5bc1c4bbd4a0e91: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.415920  4240 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:21.429220  4218 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:21.422151  4329 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.031s
W20260709 12:28:21.461428 24862 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 26079)
I20260709 12:28:21.471431  4251 raft_consensus.cc:3055] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:21.471951  4251 raft_consensus.cc:740] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:21.472896  4251 consensus_queue.cc:260] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.484373 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.485077 26126 raft_consensus.cc:2468] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
I20260709 12:28:21.489467 26020 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:21.494071  4327 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.494853  4331 raft_consensus.cc:2804] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:21.495247  4331 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.495617  4331 raft_consensus.cc:3060] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:21.500499  4325 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.501089  4325 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.501504 26197 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:28:21.504804  4325 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.507026 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.507455 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:21.510139 26128 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:21.508419 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.514102 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ec87f552a04ce49312a2ef65f862ce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.518466  4327 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.527700 26020 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.529065  4219 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:21.529690 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:21.530300 26200 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:21.536933 26128 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
W20260709 12:28:21.542402 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.538586 26020 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3; no voters: 
I20260709 12:28:21.551208  3855 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:21.551864  3855 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:21.552795  3855 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:21.556329 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.556871  4326 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.557432  4326 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.558282 25947 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 13 to 22, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 22 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.559576  4326 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.565629  4329 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.566205  4329 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.566499 26200 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:21.569001  4329 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.570118  4331 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:21.571389 25943 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 278) took 120855 ms (client timeout 15000 ms). Trace:
I20260709 12:28:21.570372 25947 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 20 to 21, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:21.573403 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.579576 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:21.571799 25943 rpcz_store.cc:269] 0709 12:26:20.715861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.716321 (+   460us) service_pool.cc:224] Handling call
0709 12:28:21.571306 (+120854985us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:19.751189 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:19.751941 (+   752us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:19.751994 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:28:19.752015 (+    21us) tablet.cc:662] Decoding operations
0709 12:28:19.752457 (+   442us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:19.752561 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:19.752594 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:28:19.752929 (+   335us) tablet.cc:701] Row locks acquired
0709 12:28:19.752985 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:28:19.753162 (+   177us) write_op.cc:270] Start()
0709 12:28:19.753321 (+   159us) write_op.cc:276] Timestamp: P: 1783600099753090 usec, L: 0
0709 12:28:19.753356 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:28:19.754095 (+   739us) log.cc:844] Serialized 541 byte log entry
0709 12:28:20.484228 (+730133us) op_driver.cc:464] REPLICATION: finished
0709 12:28:20.517585 (+ 33357us) write_op.cc:301] APPLY: starting
0709 12:28:20.517714 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:20.519219 (+  1505us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:20.519350 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:20.542438 (+ 23088us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:20.542891 (+   453us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:28:20.542952 (+    61us) write_op.cc:312] APPLY: finished
0709 12:28:20.543464 (+   512us) log.cc:844] Serialized 25 byte log entry
0709 12:28:20.543669 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:20.543779 (+   110us) write_op.cc:454] Released schema lock
0709 12:28:20.543839 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:20.544352 (+   513us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:20.544383 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":599177,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":32969,"apply.run_cpu_time_us":4639,"apply.run_wall_time_us":26779,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8313,"num_ops":1,"prepare.queue_time_us":25294,"prepare.run_cpu_time_us":20579,"prepare.run_wall_time_us":248717,"raft.queue_time_us":260210,"raft.run_cpu_time_us":8181,"raft.run_wall_time_us":36523,"replication_time_us":730695,"thread_start_us":372988,"threads_started":4,"wal-append.queue_time_us":297581,"wal-append.run_cpu_time_us":55839,"wal-append.run_wall_time_us":1491170}]]}
I20260709 12:28:21.588577 26201 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 28.
W20260709 12:28:21.593120 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 170320us
I20260709 12:28:21.594370 25947 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 25 to 27, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 27 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.596331 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:21.596081 25947 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 24 to 28, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 28 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.597687 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:21.597925 25947 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 20 to 23, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 23 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.600281  4370 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:21.600746  4370 raft_consensus.cc:493] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.601125  4370 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:21.601541  4251 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:21.599495 25947 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.631605 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.632869 26128 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:28:21.635927 25947 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 20 to 23, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 23 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.643579 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.593575  4331 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.654919 26785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.655171 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:28:21.657389 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 48499.7us
I20260709 12:28:21.657624 26194 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
W20260709 12:28:21.658135 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:21.654269 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.660458 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.662709 25947 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 25 to 28, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 28 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.661808 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:21.684926 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.655597  4310 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.689952 26094 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:21.690014  4329 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:21.684636  4309 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:21.696112  4377 raft_consensus.cc:2804] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:21.696693  4377 raft_consensus.cc:697] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:21.697710  4377 consensus_queue.cc:237] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.683343  4327 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:28:21.685827  4346 raft_consensus.cc:493] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.693243 25947 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:21.709262  4346 raft_consensus.cc:515] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.710528 25947 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 22 to 23, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 23 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:21.711468  4346 leader_election.cc:290] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.712986 25947 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 24 to 25, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:21.729558  4354 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.733662  4354 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.739828 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.740832 26122 raft_consensus.cc:2410] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:28:21.712054  4310 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.762458 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.704078  4370 raft_consensus.cc:515] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.775756 26120 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:28:21.779325  4224 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.742218  4350 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.747005  4354 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:21.791025 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ff2e29d14b44128ecab4f62a77317f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:21.800480 26167 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865): Couldn't send request to peer 24e61c63097e4d3dbb149050e6fcc997. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47b65a35d65c481985263a8a2546d39d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.801975  4403 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:28:21.804786  4399 log.cc:927] Time spent T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:28:21.806607 26787 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:21.807925  4370 leader_election.cc:290] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.780042  4224 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.815250  4240 raft_consensus.cc:493] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.831879  4240 raft_consensus.cc:515] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:21.832890  4310 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.833256  4224 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.834213  4240 leader_election.cc:290] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:21.814289  4375 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.021s
I20260709 12:28:21.794168  4350 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.839973 26120 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 30 index: 15. (term mismatch)
I20260709 12:28:21.840715 26120 pending_rounds.cc:85] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 14
I20260709 12:28:21.841706 26120 pending_rounds.cc:107] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 29.15
I20260709 12:28:21.841943 26120 raft_consensus.cc:2889] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:21.845528 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 52449.3us
I20260709 12:28:21.847506 26194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.848341 26194 raft_consensus.cc:2468] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 27.
I20260709 12:28:21.849924 26128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.850586 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d132ed1efbf42c38f5a384134455ca3" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.851158 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.852713 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:21.853295 26124 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:21.851749 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:21.856468 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cbe5295e8db495985835a50d39e4438" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:21.857085 26201 raft_consensus.cc:3060] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:21.855980 26019 leader_election.cc:304] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.856771 26790 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:28:21.859287 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:21.859838 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.860294 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:21.860872 26200 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 29.
I20260709 12:28:21.865504 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ddbca55e34c47f2b2d10e46e4779bd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.866223 26019 leader_election.cc:304] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:21.869388  4350 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.869553  4401 raft_consensus.cc:2749] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:21.878499  3855 raft_consensus.cc:2749] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:21.878654 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:21.879420 26786 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 28.
I20260709 12:28:21.898582 26124 raft_consensus.cc:2410] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
W20260709 12:28:21.908113 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 44573.5us
I20260709 12:28:21.902745 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:21.931192 26201 raft_consensus.cc:2468] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
W20260709 12:28:21.938666 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:21.940688 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:21.945547 26168 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.946049 26168 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:21.937105  4346 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.949971  4346 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.957963 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:21.933288  4313 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:28:21.959992 26094 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:21.961229 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.926317  4325 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
W20260709 12:28:21.968879 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.965898  4242 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:21.977716 26124 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:21.930858  4240 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:28:21.952224  4346 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:21.964556  4313 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.988863  4370 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.010067  4313 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:21.984777 26200 raft_consensus.cc:1275] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:22.010185  4370 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:21.987287  4219 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:28:22.001061  4405 raft_consensus.cc:2955] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Committing config change with OpId 31.19: config changed from index -1 to 19, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:28:22.034487  4370 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:22.019735 26018 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.023855  4251 consensus_queue.cc:1048] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:22.030977 26756 leader_election.cc:304] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.041294  4219 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:22.043030  4368 raft_consensus.cc:2804] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:22.043593  4368 raft_consensus.cc:697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:22.044534  4368 consensus_queue.cc:237] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.049300 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:22.050032 26786 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:22.057907 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.019520 26124 raft_consensus.cc:2955] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Committing config change with OpId 31.19: config changed from index -1 to 19, NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
W20260709 12:28:21.983697 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.029407 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:22.059237 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 75607.3us
I20260709 12:28:22.058605 26197 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:22.059454 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:22.060109 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.060146 26129 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 29.
I20260709 12:28:22.069217 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:22.071287  4366 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:22.071744  4366 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.072103  4366 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.072770 26201 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
W20260709 12:28:22.080571 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.093271 26168 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:22.094543 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.100850 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.101418 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:22.105821  4327 raft_consensus.cc:2749] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:22.106777  4224 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.107373  4224 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.109758  4224 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.114995  4313 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.116387 26203 raft_consensus.cc:1275] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:28:22.117795  4327 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:22.128176 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.131979 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:22.130589 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.130021  4326 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:22.136379  4326 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.136746  4326 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.139014  4313 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Requested vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.140959  4366 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.134313  4218 consensus_queue.cc:1048] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:28:22.143894  4366 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.134562 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:22.147420 26201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:22.148075 26201 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.149914 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:22.150555 26127 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.149942 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:22.150913 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:22.151526 26787 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.156337  4240 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:22.165954 26127 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:22.170892 26201 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
W20260709 12:28:22.178881 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.179737 26787 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in term 30.
W20260709 12:28:22.147557 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.181012 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.185590 26756 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:22.188014 26019 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:22.189190 26000 auto_rebalancer.cc:629] Failed to schedule move for tablet 407574b4cda94da99be372d249d21fd5: Illegal state: Replica 9e23389325a6411c9b0634f4f6af54c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:22.188105  4401 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.190068  4401 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.192042  4338 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:22.192678  4338 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Becoming Leader. State: Replica: 8fd5643e7b494fa79c3d0aaf1e6e8152, State: Running, Role: LEADER
I20260709 12:28:22.200081 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.206658 26126 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:22.207268  4401 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.215096 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:22.216816 26195 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:22.225215 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.225963 26195 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fd5643e7b494fa79c3d0aaf1e6e8152 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:22.233153 26018 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.238958  4313 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.239782  4219 raft_consensus.cc:493] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.240396  4219 raft_consensus.cc:515] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:22.242830  4219 leader_election.cc:290] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:22.216586  4401 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:22.239339 26168 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:22.245282  4401 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:22.247648  4401 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:22.252214 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.251312 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.266055 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bba37d4e722416198d4e0aa03fc3d5f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.257603  4325 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:28:22.208225  4338 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.269436 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d33ec3d2d94eeda5bc1c4bbd4a0e91" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.281263  4240 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.286113 26020 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:22.288429  4309 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:22.282054  4240 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.312489  4313 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:22.313340  4240 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.318490 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:22.319445 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.323559 26019 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:22.324965  4240 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:22.324558 26020 leader_election.cc:304] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:22.285674  4326 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.345873 26791 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
I20260709 12:28:22.336323  4401 raft_consensus.cc:2749] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:22.336122  4313 raft_consensus.cc:493] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.368324 26167 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:22.378873  4326 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:22.381896 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:22.384321 26126 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:22.385035 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:22.381964  4313 raft_consensus.cc:515] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.400125  4313 leader_election.cc:290] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.401494 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:22.384652  4365 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:22.409287 26195 raft_consensus.cc:2393] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:22.410645 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d1927cc7714e91945918d6aa1216e5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.412171 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:22.413153  4264 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:22.430581 26020 leader_election.cc:304] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.440295  4338 raft_consensus.cc:2749] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:22.440593 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 104150us
I20260709 12:28:22.460749 25943 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 26 to 31, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 31 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:22.461959 24862 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 26079)
W20260709 12:28:22.463150 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.466389 26756 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:22.482846 25943 catalog_manager.cc:5697] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 28, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.484833 25943 catalog_manager.cc:5697] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 16 to 29. New cstate: current_term: 29 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.491945 26196 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:22.493696  4423 ts_tablet_manager.cc:933] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Initiating tablet copy from peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:22.493645  4264 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:22.528074  4105 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.528716  4105 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:22.531056  4105 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:22.511575 25947 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 292) took 56764 ms (client timeout 15000 ms). Trace:
I20260709 12:28:22.486861 25943 catalog_manager.cc:5697] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 29 to 31, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 31 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.532351 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.533036 26203 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 28.
I20260709 12:28:22.532929 25943 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 26 to 29, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 29 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.534926 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:22.534641 25943 catalog_manager.cc:5697] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 34, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 34 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:22.541298 26094 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:22.542510  4105 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:22.543063  4105 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.543493  4105 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:22.550457 26124 raft_consensus.cc:1275] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:28:22.550930 26124 pending_rounds.cc:85] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Aborting all ops after (but not including) 17
I20260709 12:28:22.551149 26124 pending_rounds.cc:107] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: Aborting uncommitted NO_OP operation due to leader change: 27.18
I20260709 12:28:22.551393 26124 raft_consensus.cc:2889] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:22.551893 25943 catalog_manager.cc:5697] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 31, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 31 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.553915  4264 consensus_queue.cc:1048] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:22.554387 25943 catalog_manager.cc:5697] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 24 to 30. New cstate: current_term: 30 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.561872  4105 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:22.563184  4423 tablet_copy_client.cc:323] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.131:38261
I20260709 12:28:22.563342  4338 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.573624  4338 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:22.576202  4338 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:22.580648 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:22.532095 25947 rpcz_store.cc:269] 0709 12:27:25.815682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.191715 (+11376033us) service_pool.cc:224] Handling call
0709 12:28:22.511488 (+45319773us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:21.863815 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:21.871400 (+  7585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:21.871435 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:28:21.871452 (+    17us) tablet.cc:662] Decoding operations
0709 12:28:21.874719 (+  3267us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:21.874783 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:21.874811 (+    28us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:28:21.885708 (+ 10897us) tablet.cc:701] Row locks acquired
0709 12:28:21.885734 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:28:21.885886 (+   152us) write_op.cc:270] Start()
0709 12:28:21.886009 (+   123us) write_op.cc:276] Timestamp: P: 1783600101885838 usec, L: 0
0709 12:28:21.886035 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:28:21.886769 (+   734us) log.cc:844] Serialized 6003 byte log entry
0709 12:28:22.042140 (+155371us) op_driver.cc:464] REPLICATION: finished
0709 12:28:22.158010 (+115870us) write_op.cc:301] APPLY: starting
0709 12:28:22.158100 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:22.232118 (+ 74018us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:22.232225 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:22.363920 (+131695us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:22.364224 (+   304us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:28:22.364327 (+   103us) write_op.cc:312] APPLY: finished
0709 12:28:22.365976 (+  1649us) log.cc:844] Serialized 139 byte log entry
0709 12:28:22.366165 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:22.366322 (+   157us) write_op.cc:454] Released schema lock
0709 12:28:22.366374 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:22.367077 (+   703us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:22.367108 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":463999,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":115635,"apply.run_cpu_time_us":24139,"apply.run_wall_time_us":209026,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66917,"num_ops":15,"prepare.queue_time_us":92602,"prepare.run_cpu_time_us":15549,"prepare.run_wall_time_us":70819,"raft.queue_time_us":63200,"raft.run_cpu_time_us":18170,"raft.run_wall_time_us":83969,"replication_time_us":155936,"thread_start_us":206875,"threads_started":4,"wal-append.queue_time_us":85719}]]}
I20260709 12:28:22.588056 26213 tablet_copy_service.cc:140] P a981c9a7972c4e99a0903924c64fbb5c: Received BeginTabletCopySession request for tablet 47b65a35d65c481985263a8a2546d39d from peer 24e61c63097e4d3dbb149050e6fcc997 ({username='slave'} at 127.0.0.1:53816)
I20260709 12:28:22.588351 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:22.618742 26197 raft_consensus.cc:3060] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:22.588390 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b3e794c66a4dc19b6590272eae81f7" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.589841  4105 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:22.592140 26094 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:22.622849 26213 tablet_copy_service.cc:161] P a981c9a7972c4e99a0903924c64fbb5c: Beginning new tablet copy session on tablet 47b65a35d65c481985263a8a2546d39d from peer 24e61c63097e4d3dbb149050e6fcc997 at {username='slave'} at 127.0.0.1:53816: session id = 24e61c63097e4d3dbb149050e6fcc997-47b65a35d65c481985263a8a2546d39d
I20260709 12:28:22.633009 26019 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:22.634492  4338 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:22.634780 26197 raft_consensus.cc:2468] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:22.636085 26094 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:22.637240  4377 raft_consensus.cc:2804] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:22.637730  4377 raft_consensus.cc:697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:22.638545  4377 consensus_queue.cc:237] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:22.648715 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.650717 26213 tablet_copy_source_session.cc:215] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:28:22.615912 25943 catalog_manager.cc:5697] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 27, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 27 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.661971 25943 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 21 to 25, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 25 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.663437 25943 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 12 to 19, term changed from 22 to 27, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to <none>, VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) evicted. New cstate: current_term: 27 committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.679068 25943 catalog_manager.cc:5697] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 24 to 33. New cstate: current_term: 33 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.682685  4423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b65a35d65c481985263a8a2546d39d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:22.691901 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.680999 25943 catalog_manager.cc:5697] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 30. New cstate: current_term: 30 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.703624 25943 catalog_manager.cc:5697] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 19 to 28, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 28 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:22.707571 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.728149  4338 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.747740 25943 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 23 to 28, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.748971  4338 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.749665 25943 catalog_manager.cc:5697] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 32, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 32 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.754680 25943 catalog_manager.cc:5697] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 24 to 31, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 31 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.756817  4338 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.758358 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.758206 25943 catalog_manager.cc:5697] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 24 to 30. New cstate: current_term: 30 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.763311 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:22.765647 26018 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.768095  4338 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:22.776417 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.778445 25943 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.786659 26197 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:22.789937 25943 catalog_manager.cc:5697] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 21 to 28, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 28 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.790788  4313 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:22.792224  4423 tablet_copy_client.cc:806] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 0 data blocks...
I20260709 12:28:22.799636  4338 raft_consensus.cc:493] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.801928 25943 catalog_manager.cc:5697] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 26, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 26 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.803043  4338 raft_consensus.cc:515] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:22.804042 25943 catalog_manager.cc:5697] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.805536  4338 leader_election.cc:290] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:22.807274  4423 tablet_copy_client.cc:670] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Starting download of 1 WAL segments...
I20260709 12:28:22.808290 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:22.808501 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fee7aee965145c6b43db63c2b3ecca8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.812979  4423 tablet_copy_client.cc:538] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:28:22.814111 26020 leader_election.cc:304] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:22.816258  4338 raft_consensus.cc:2749] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:22.806021 25943 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 27, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 27 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.823040 25943 catalog_manager.cc:5697] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 29. New cstate: current_term: 29 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.826007 25943 catalog_manager.cc:5697] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 32. New cstate: current_term: 32 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:22.823925  4423 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap starting.
I20260709 12:28:22.832120 25943 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 28. New cstate: current_term: 28 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.835875 25943 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 26 to 30. New cstate: current_term: 30 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.892755 26791 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Refusing update from remote peer 8fd5643e7b494fa79c3d0aaf1e6e8152: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:22.906448  4313 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:22.966926  4313 raft_consensus.cc:493] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.967568  4313 raft_consensus.cc:515] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:22.976821 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:22.978210 26126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "407574b4cda94da99be372d249d21fd5" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:22.990272  4327 raft_consensus.cc:493] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:22.990945  4327 raft_consensus.cc:515] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:23.001410 26019 leader_election.cc:304] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:23.002447  4401 raft_consensus.cc:2749] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:23.004663  4338 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:23.005262  4338 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:23.005717 26756 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 201 similar messages]
I20260709 12:28:23.009119  4313 leader_election.cc:290] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:23.011615 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:23.015260 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585ccc5a92245cfb1bba6ff29291731" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:23.011904  4338 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:23.032699 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:23.042333 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aa4b8b33f544088007e1987c55617e" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:23.039875 26020 leader_election.cc:304] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:23.048766  4401 raft_consensus.cc:2749] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:22.995253  4327 leader_election.cc:290] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:23.071194 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:23.075299 26019 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:23.079552  4338 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:23.083513 26196 raft_consensus.cc:1275] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:23.085373  4377 consensus_queue.cc:1048] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:28:23.148190 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 70632.6us
I20260709 12:28:23.148906 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with delete_type TABLET_DATA_TOMBSTONED (TS a981c9a7972c4e99a0903924c64fbb5c not found in new config with opid_index 19) from {username='slave'} at 127.0.0.1:54828
W20260709 12:28:23.161845 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.163077 25944 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 24 to 31, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 31 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.185134 25943 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 198) took 4418 ms. Trace:
I20260709 12:28:23.190184 25944 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 25 to 28. New cstate: current_term: 28 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.191954 25944 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.193830 25944 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.196481 25944 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 23 to 24, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 24 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.197966 25944 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 24 to 27, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 27 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.199692 25944 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 26 to 27, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 27 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.202395 25944 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 23 to 25, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 25 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.204402 25944 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.206393 25944 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 21 to 25, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 25 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.190804  4438 tablet_replica.cc:333] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:23.208612 25944 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 18 to 24, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 24 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.185642 25943 rpcz_store.cc:276] 0709 12:28:18.767071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.589805 (+2822734us) service_pool.cc:224] Handling call
0709 12:28:23.185036 (+1595231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:22.898020 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:22.898714 (+   694us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:22.898752 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:28:22.898773 (+    21us) tablet.cc:662] Decoding operations
0709 12:28:22.901062 (+  2289us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:22.901155 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:22.901188 (+    33us) tablet.cc:685] Acquiring locks for 27 operations
0709 12:28:22.917375 (+ 16187us) tablet.cc:701] Row locks acquired
0709 12:28:22.917397 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:28:22.917558 (+   161us) write_op.cc:270] Start()
0709 12:28:22.917694 (+   136us) write_op.cc:276] Timestamp: P: 1783600102917504 usec, L: 0
0709 12:28:22.917721 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:28:22.945002 (+ 27281us) log.cc:844] Serialized 10587 byte log entry
0709 12:28:22.960565 (+ 15563us) op_driver.cc:464] REPLICATION: finished
0709 12:28:22.961880 (+  1315us) write_op.cc:301] APPLY: starting
0709 12:28:22.961970 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:23.026110 (+ 64140us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:23.026218 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:23.068186 (+ 41968us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:23.078382 (+ 10196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 12:28:23.078431 (+    49us) write_op.cc:312] APPLY: finished
0709 12:28:23.078900 (+   469us) log.cc:844] Serialized 236 byte log entry
0709 12:28:23.079079 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.079301 (+   222us) write_op.cc:454] Released schema lock
0709 12:28:23.079374 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:23.096045 (+ 16671us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.096087 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4591,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":1035,"apply.run_cpu_time_us":33145,"apply.run_wall_time_us":134125,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117149,"num_ops":27,"prepare.queue_time_us":17833,"prepare.run_cpu_time_us":7753,"prepare.run_wall_time_us":58814,"replication_time_us":42683,"spinlock_wait_cycles":71040,"thread_start_us":27045,"threads_started":2}]]}
I20260709 12:28:23.235812  4423 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:23.236908  4423 tablet_bootstrap.cc:492] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Bootstrap complete.
I20260709 12:28:23.237702  4423 ts_tablet_manager.cc:1403] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Time spent bootstrapping tablet: real 0.414s	user 0.067s	sys 0.004s
I20260709 12:28:23.264477  4423 raft_consensus.cc:359] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:28:23.265470  4423 raft_consensus.cc:740] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEARNER]: Becoming Follower/Learner. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Initialized, Role: LEARNER
I20260709 12:28:23.397010  4438 raft_consensus.cc:2243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:23.462437 24862 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 26079)
I20260709 12:28:23.436914  4423 consensus_queue.cc:260] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } }
I20260709 12:28:23.458123  4438 raft_consensus.cc:2272] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.492650  4327 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.530687  4423 ts_tablet_manager.cc:1434] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Time spent starting tablet: real 0.293s	user 0.023s	sys 0.005s
W20260709 12:28:23.550467 25944 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 293) took 23491 ms (client timeout 15000 ms). Trace:
I20260709 12:28:23.537299  4327 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:23.561635  4327 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:23.565534 25944 rpcz_store.cc:269] 0709 12:28:00.059382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.603122 (+8543740us) service_pool.cc:224] Handling call
0709 12:28:23.550253 (+14947131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:23.214057 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:23.214807 (+   750us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:23.214851 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:28:23.214873 (+    22us) tablet.cc:662] Decoding operations
0709 12:28:23.215912 (+  1039us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:23.215964 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:23.216013 (+    49us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:28:23.217083 (+  1070us) tablet.cc:701] Row locks acquired
0709 12:28:23.217102 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:28:23.217271 (+   169us) write_op.cc:270] Start()
0709 12:28:23.217372 (+   101us) write_op.cc:276] Timestamp: P: 1783600103217215 usec, L: 0
0709 12:28:23.217390 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:28:23.217990 (+   600us) log.cc:844] Serialized 4399 byte log entry
0709 12:28:23.225607 (+  7617us) op_driver.cc:464] REPLICATION: finished
0709 12:28:23.225981 (+   374us) write_op.cc:301] APPLY: starting
0709 12:28:23.226066 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:23.231757 (+  5691us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:23.231855 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:23.255019 (+ 23164us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:23.255282 (+   263us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:28:23.255338 (+    56us) write_op.cc:312] APPLY: finished
0709 12:28:23.282575 (+ 27237us) log.cc:844] Serialized 105 byte log entry
0709 12:28:23.310350 (+ 27775us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.398913 (+ 88563us) write_op.cc:454] Released schema lock
0709 12:28:23.398976 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:23.399354 (+   378us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.399387 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26015,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":135,"apply.run_cpu_time_us":22130,"apply.run_wall_time_us":173331,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46046,"num_ops":11,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":4208,"prepare.run_wall_time_us":5425,"replication_time_us":8078}]]}
I20260709 12:28:23.565301  4327 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.572484  4438 ts_tablet_manager.cc:1916] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:28:23.572147  4327 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:23.575201  4327 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:23.581633 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 60140.5us
I20260709 12:28:23.536463 25947 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 23 to 33. New cstate: current_term: 33 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.583654 25947 catalog_manager.cc:5697] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 31 to 34, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.585893 25947 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 24 to 31, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131), now has a pending config: VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129), VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130), VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131), NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 31 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: NON_VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: true } } }
I20260709 12:28:23.611159 25947 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 22 to 31, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 31 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.613916 25947 catalog_manager.cc:5697] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.615851 25947 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 33. New cstate: current_term: 33 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:23.617975 25947 catalog_manager.cc:5697] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 27 to 28, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.622174 25947 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 26 to 33, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 33 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.626044 25947 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 21 to 34, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 34 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:23.627903 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:23.629189 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:23.625311  4401 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.633224 25947 catalog_manager.cc:5697] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 27 to 33, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 33 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.648509  4458 raft_consensus.cc:493] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.656342 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3c86db3d4e4643a77b0fe0b0e687e1" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:23.662794  4459 raft_consensus.cc:493] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.663674  4455 raft_consensus.cc:493] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.667304 25947 catalog_manager.cc:5697] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 34, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.673573 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891138e164dc40a7af7618323d52397f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:23.649171  4458 raft_consensus.cc:515] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:23.663427  4459 raft_consensus.cc:515] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:23.678884  4458 leader_election.cc:290] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:23.664206  4455 raft_consensus.cc:515] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:23.683621  4455 leader_election.cc:290] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:23.691427 26788 raft_consensus.cc:1217] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEARNER]: Deduplicated request from leader. Original: 31.18->[31.19-31.19]   Dedup: 31.19->[]
I20260709 12:28:23.712296 25947 catalog_manager.cc:5697] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 35, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 35 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:23.714807 25947 catalog_manager.cc:5697] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 33, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 33 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.717608 25947 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 25 to 29, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 29 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.719518 25947 catalog_manager.cc:5697] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 26 to 33. New cstate: current_term: 33 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:23.722101 25947 catalog_manager.cc:5697] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.725183 25947 catalog_manager.cc:5697] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 25 to 26, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 26 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.727372 25947 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 24 to 28, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 28 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.680323  4459 leader_election.cc:290] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:23.651077  4401 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:23.738145 25947 catalog_manager.cc:5697] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 25 to 34, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.750355 25947 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 17 to 27, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 27 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.752183 25947 catalog_manager.cc:5697] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 24 to 31, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 31 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.754547 25947 catalog_manager.cc:5697] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 17 to 26, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 26 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.757789 25947 catalog_manager.cc:5697] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 16 to 25. New cstate: current_term: 25 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.759896 25947 catalog_manager.cc:5697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 12 to 33, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 33 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.762388 25947 catalog_manager.cc:5697] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 23 to 34, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.777311 25947 catalog_manager.cc:5697] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:23.785799 26213 tablet_copy_service.cc:342] P a981c9a7972c4e99a0903924c64fbb5c: Request end of tablet copy session 24e61c63097e4d3dbb149050e6fcc997-47b65a35d65c481985263a8a2546d39d received from {username='slave'} at 127.0.0.1:53816
I20260709 12:28:23.786382 26213 tablet_copy_service.cc:434] P a981c9a7972c4e99a0903924c64fbb5c: ending tablet copy session 24e61c63097e4d3dbb149050e6fcc997-47b65a35d65c481985263a8a2546d39d on tablet 47b65a35d65c481985263a8a2546d39d with peer 24e61c63097e4d3dbb149050e6fcc997
W20260709 12:28:23.788597 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 143918us
I20260709 12:28:23.792824 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:23.795251 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:23.781111  4327 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.817165 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:23.808889  4327 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:23.827746  4327 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:23.839669 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 84807.1us
I20260709 12:28:23.848138 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0673bc676154f04907934c1d3a60bce" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:23.850816 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de16ce1800b64c3c94c918aa6e9cae3d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:23.810396  4401 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:23.860957 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:23.901988 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 43694.3us
I20260709 12:28:23.859930  4338 raft_consensus.cc:493] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:23.913551 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 56860.4us
W20260709 12:28:23.913731 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 52117.5us
I20260709 12:28:23.918393 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:23.909355  4438 ts_tablet_manager.cc:1929] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 26.18
W20260709 12:28:23.921471 26018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 284644us
I20260709 12:28:23.925225 26018 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:23.929176 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe81c3162f24fc0b8716b2b42f0676f" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:23.929610  4450 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:23.925854  4438 log.cc:1199] T 8ddbca55e34c47f2b2d10e46e4779bd0 P a981c9a7972c4e99a0903924c64fbb5c: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/wals/8ddbca55e34c47f2b2d10e46e4779bd0
I20260709 12:28:23.946177 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
W20260709 12:28:23.951337 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 85180.8us
I20260709 12:28:23.953598 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:23.920593  4338 raft_consensus.cc:515] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:23.981721  4338 leader_election.cc:290] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:23.998919 26020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 83848.4us
I20260709 12:28:24.000649 26020 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:24.001600  4453 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:24.007718 26020 leader_election.cc:304] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:24.016623  4453 raft_consensus.cc:2749] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:24.022939 26020 leader_election.cc:304] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:24.028072  4338 raft_consensus.cc:2749] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:24.033684 26127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b65a35d65c481985263a8a2546d39d" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:24.040534 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:24.041129 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 277727us
W20260709 12:28:24.077981 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.085704 26020 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:24.086663  4338 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:24.090457 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:24.104175 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:24.105737 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.101958 26018 leader_election.cc:304] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:24.127239  4338 raft_consensus.cc:2749] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:24.130748  4242 raft_consensus.cc:1064] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: attempting to promote NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 to VOTER
I20260709 12:28:24.133687  4242 consensus_queue.cc:237] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 17, Current term: 31, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:24.141556 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:24.154194  4469 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:24.171765 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86fd5767abab4e45ad1c8bd5f6d7a9c8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:24.189466  4338 raft_consensus.cc:493] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.190197  4338 raft_consensus.cc:515] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:24.192734  4338 leader_election.cc:290] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:24.195480 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:24.197836 26129 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:28:24.217880 26784 raft_consensus.cc:1275] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEARNER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:28:24.223803  4446 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:24.225173  4464 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.225783  4464 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:24.226862 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49f4a1a801de461182f8315a884b9889" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 34 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:24.229907  4464 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:24.232999 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:24.233938 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:24.242010  4242 consensus_queue.cc:1048] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:24.247756 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:24.248485 26129 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:24.250614 26018 leader_election.cc:304] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:24.251408  4453 raft_consensus.cc:493] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:24.251925  4453 raft_consensus.cc:515] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:24.254047  4469 raft_consensus.cc:2749] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:24.259029  4453 leader_election.cc:290] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:24.261371 26018 leader_election.cc:304] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:24.262533 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:24.264816 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:24.266041 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e8b57c1b80240d483c7ca4e95d74673" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:24.270314  4453 raft_consensus.cc:2749] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:24.276021 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.278373 26020 leader_election.cc:304] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:24.279605  4453 raft_consensus.cc:2749] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:24.287024 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:24.291141 26168 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:24.289541  4446 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:24.301204  4446 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.314021  4446 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:24.371048 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.374848  4477 log.cc:927] Time spent T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:28:24.369436  4338 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:24.392201  4468 raft_consensus.cc:2955] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Committing config change with OpId 31.20: config changed from index 19 to 20, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.392508 26122 raft_consensus.cc:2955] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Committing config change with OpId 31.20: config changed from index 19 to 20, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.398311  4338 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:24.412480 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.416863  4446 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:24.425629  4338 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:24.426494 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:24.427567 26168 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:24.436832 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.443459 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824b6757fe414215b1cf6aa5ba238004" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:24.445640 26788 raft_consensus.cc:2955] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Committing config change with OpId 31.20: config changed from index 19 to 20, 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.447983  4446 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:24.449862 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:24.450537 26122 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:24.455009 26018 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:24.455983  4338 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:24.457079 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:24.457687 26784 raft_consensus.cc:3060] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:24.462942 24862 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 26079)
I20260709 12:28:24.471738 26784 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:24.474359 26122 raft_consensus.cc:2468] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:24.475620 26169 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:24.480178  4446 raft_consensus.cc:2804] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:28:24.503728 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.490603  4446 raft_consensus.cc:697] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:24.520154 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 12 (current committed config index is 19)) from {username='slave'} at 127.0.0.1:54828
W20260709 12:28:24.531574 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.537842 25947 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 283) took 2550 ms. Trace:
I20260709 12:28:24.560506  4446 consensus_queue.cc:237] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.14, Last appended by leader: 14, Current term: 31, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:24.539059 25947 rpcz_store.cc:276] 0709 12:28:21.987633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.631192 (+643559us) service_pool.cc:224] Handling call
0709 12:28:24.537778 (+1906586us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:24.043004 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:24.043557 (+   553us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:24.043597 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:28:24.043614 (+    17us) tablet.cc:662] Decoding operations
0709 12:28:24.045442 (+  1828us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:24.045526 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:24.045557 (+    31us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:28:24.060333 (+ 14776us) tablet.cc:701] Row locks acquired
0709 12:28:24.060354 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:28:24.060501 (+   147us) write_op.cc:270] Start()
0709 12:28:24.060657 (+   156us) write_op.cc:276] Timestamp: P: 1783600104060456 usec, L: 0
0709 12:28:24.060679 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:28:24.062254 (+  1575us) log.cc:844] Serialized 10454 byte log entry
0709 12:28:24.080825 (+ 18571us) op_driver.cc:464] REPLICATION: finished
0709 12:28:24.159747 (+ 78922us) write_op.cc:301] APPLY: starting
0709 12:28:24.159841 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:24.265133 (+105292us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:24.265284 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:24.459824 (+194540us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:24.460081 (+   257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=0,mrs_lookups=0
0709 12:28:24.460111 (+    30us) write_op.cc:312] APPLY: finished
0709 12:28:24.460782 (+   671us) log.cc:844] Serialized 228 byte log entry
0709 12:28:24.460964 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.461227 (+   263us) write_op.cc:454] Released schema lock
0709 12:28:24.461298 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:24.484059 (+ 22761us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.484303 (+   244us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30772,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":78656,"apply.run_cpu_time_us":35318,"apply.run_wall_time_us":318059,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112963,"num_ops":26,"prepare.queue_time_us":112192,"prepare.run_cpu_time_us":6299,"prepare.run_wall_time_us":19585,"replication_time_us":19979,"spinlock_wait_cycles":1894272,"thread_start_us":186797,"threads_started":2}]]}
W20260709 12:28:24.593840 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.594612 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
W20260709 12:28:24.704612 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.722347  4338 raft_consensus.cc:493] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.722998  4338 raft_consensus.cc:515] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:24.728955 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:24.735132 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:24.736907 26020 leader_election.cc:304] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:24.737846  4453 raft_consensus.cc:2749] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:24.737331 25945 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.738983  4338 leader_election.cc:290] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:24.744822 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c5f2ff7a584f80b9a347f6adb3c2f8" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:24.753831 25946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 282) took 18823 ms (client timeout 15000 ms). Trace:
W20260709 12:28:24.754287 25946 rpcz_store.cc:269] 0709 12:28:06.069027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.191405 (+11122378us) service_pool.cc:224] Handling call
0709 12:28:24.753621 (+7562216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7953,"reported_tablets":49}
I20260709 12:28:24.745085 25945 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 reported cstate change: term changed from 21 to 28, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129). New cstate: current_term: 28 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.762089 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 12 (current committed config index is 19)) from {username='slave'} at 127.0.0.1:54828
I20260709 12:28:24.767297 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:28:24.785009 26200 consensus_queue.cc:237] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:24.805662 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.809911 26788 raft_consensus.cc:1275] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:28:24.810884 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:24.811630  4446 consensus_queue.cc:1048] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:28:24.823597 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.832705  4468 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Committing config change with OpId 28.20: config changed from index 10 to 20, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.836719 25943 catalog_manager.cc:5697] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 22 to 29, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 29 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.838523 25943 catalog_manager.cc:5697] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 27 to 32, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 32 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.840080 25943 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.851760 25943 catalog_manager.cc:5697] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 27 to 28, leader changed from <none> to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 28 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.857453 26784 raft_consensus.cc:2955] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 10 to 20, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
W20260709 12:28:24.859797 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.860452 25943 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 30 to 32, leader changed from 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 32 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.861934 25943 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 25 to 30. New cstate: current_term: 30 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:24.864100 25943 catalog_manager.cc:5697] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.872545 25945 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 294) took 7300 ms. Trace:
I20260709 12:28:24.873332 25945 rpcz_store.cc:276] 0709 12:28:17.572671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.081591 (+1508920us) service_pool.cc:224] Handling call
0709 12:28:24.872471 (+5790880us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:24.796100 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:24.796657 (+   557us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:24.796694 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:28:24.796721 (+    27us) tablet.cc:662] Decoding operations
0709 12:28:24.797134 (+   413us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:24.797212 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:24.797246 (+    34us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:28:24.797538 (+   292us) tablet.cc:701] Row locks acquired
0709 12:28:24.797556 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:28:24.797693 (+   137us) write_op.cc:270] Start()
0709 12:28:24.797801 (+   108us) write_op.cc:276] Timestamp: P: 1783600104797645 usec, L: 0
0709 12:28:24.797821 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:28:24.798507 (+   686us) log.cc:844] Serialized 929 byte log entry
0709 12:28:24.811887 (+ 13380us) op_driver.cc:464] REPLICATION: finished
0709 12:28:24.812475 (+   588us) write_op.cc:301] APPLY: starting
0709 12:28:24.812574 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:24.813961 (+  1387us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:24.814209 (+   248us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:24.817429 (+  3220us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:24.817604 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:28:24.817657 (+    53us) write_op.cc:312] APPLY: finished
0709 12:28:24.817870 (+   213us) log.cc:844] Serialized 33 byte log entry
0709 12:28:24.818029 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.818141 (+   112us) write_op.cc:454] Released schema lock
0709 12:28:24.818185 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:24.818557 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.818588 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30939,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":4353,"apply.run_wall_time_us":6259,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8372,"num_ops":2,"prepare.queue_time_us":36692,"prepare.run_cpu_time_us":2740,"prepare.run_wall_time_us":2740,"replication_time_us":13903,"thread_start_us":39602,"threads_started":1}]]}
I20260709 12:28:24.884634 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:24.888875 24862 tablet_replica.cc:333] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:24.890306 24862 raft_consensus.cc:2243] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:24.890954 24862 raft_consensus.cc:2272] T 0bba37d4e722416198d4e0aa03fc3d5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:24.892580 25938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 93ec87f552a04ce49312a2ef65f862ce with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:28:24.897380 24862 tablet_replica.cc:333] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:24.899232 24862 raft_consensus.cc:2243] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:24.905985 24862 raft_consensus.cc:2272] T 49f4a1a801de461182f8315a884b9889 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:24.909171 24862 tablet_replica.cc:333] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:24.927244 24862 raft_consensus.cc:2243] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:24.928114 24862 pending_rounds.cc:70] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:24.928435 24862 pending_rounds.cc:77] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 31 index: 20 } timestamp: 7305626001081589760 op_type: NO_OP noop_request { }
I20260709 12:28:24.928890 24862 raft_consensus.cc:2889] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:24.929309 24862 raft_consensus.cc:2272] T 891138e164dc40a7af7618323d52397f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:24.929672 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.950359 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.970815 25943 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39330 (request call id 199) took 1434 ms. Trace:
I20260709 12:28:24.971351 25943 rpcz_store.cc:276] 0709 12:28:23.535957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.539415 (+  3458us) service_pool.cc:224] Handling call
0709 12:28:24.970749 (+1431334us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:24.886681 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:24.887224 (+   543us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:24.887258 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:28:24.887276 (+    18us) tablet.cc:662] Decoding operations
0709 12:28:24.891291 (+  4015us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:24.891354 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:24.891381 (+    27us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:28:24.891993 (+   612us) tablet.cc:701] Row locks acquired
0709 12:28:24.892012 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:28:24.892186 (+   174us) write_op.cc:270] Start()
0709 12:28:24.892351 (+   165us) write_op.cc:276] Timestamp: P: 1783600104892134 usec, L: 0
0709 12:28:24.892377 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:28:24.893109 (+   732us) log.cc:844] Serialized 2831 byte log entry
0709 12:28:24.901819 (+  8710us) op_driver.cc:464] REPLICATION: finished
0709 12:28:24.902238 (+   419us) write_op.cc:301] APPLY: starting
0709 12:28:24.902315 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:24.905967 (+  3652us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:24.906056 (+    89us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:24.911850 (+  5794us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:24.912066 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:28:24.912118 (+    52us) write_op.cc:312] APPLY: finished
0709 12:28:24.920763 (+  8645us) log.cc:844] Serialized 73 byte log entry
0709 12:28:24.936418 (+ 15655us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.936550 (+   132us) write_op.cc:454] Released schema lock
0709 12:28:24.936623 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:24.937023 (+   400us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.937052 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9233,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":237,"apply.run_cpu_time_us":9643,"apply.run_wall_time_us":34713,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29302,"num_ops":7,"prepare.queue_time_us":11959,"prepare.run_cpu_time_us":3595,"prepare.run_wall_time_us":6715,"replication_time_us":9332}]]}
W20260709 12:28:24.980520 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.987562 24862 tablet_replica.cc:333] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:24.988855 24862 raft_consensus.cc:2243] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.016010 24862 raft_consensus.cc:2272] T d2b3e794c66a4dc19b6590272eae81f7 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:25.023298 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:25.017176 25950 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:25.030480 24862 tablet_replica.cc:333] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.031684 24862 raft_consensus.cc:2243] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Raft consensus shutting down.
W20260709 12:28:25.038545 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:25.057770 25943 catalog_manager.cc:5697] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index 10 to 20, VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) evicted. New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:25.061653 24862 raft_consensus.cc:2272] T 6389d613e36a42958d24b00099843b93 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.059607 25943 catalog_manager.cc:5697] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: config changed from index -1 to 20, VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132) added, no longer has a pending config: VOTER 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129), VOTER 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130), VOTER a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131), NON_VOTER 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 31 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } } }
I20260709 12:28:25.070416 26788 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:28:25.072072  4446 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:28:25.097932 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:25.097606 25947 catalog_manager.cc:5697] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 23 to 32, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 32 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:25.099601 25947 catalog_manager.cc:5697] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 23 to 31, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 31 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:25.101890 25947 catalog_manager.cc:5697] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 26 to 29, leader changed from a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 29 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:25.103464 25947 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:25.104849 25947 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } }
I20260709 12:28:25.106467 24862 tablet_replica.cc:333] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.217265 24862 raft_consensus.cc:2243] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.228621 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 93ec87f552a04ce49312a2ef65f862ce with delete_type TABLET_DATA_TOMBSTONED (TS 9e23389325a6411c9b0634f4f6af54c3 not found in new config with opid_index 20) from {username='slave'} at 127.0.0.1:44004
I20260709 12:28:25.254208  4492 raft_consensus.cc:493] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.254820  4492 raft_consensus.cc:515] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:25.228595 24862 raft_consensus.cc:2272] T 47b65a35d65c481985263a8a2546d39d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.262806 24862 tablet_replica.cc:333] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.326344 24862 raft_consensus.cc:2243] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.280400 26122 raft_consensus.cc:1275] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
W20260709 12:28:25.287213 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:25.293143  4492 leader_election.cc:290] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:25.305563  4401 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.320935 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
W20260709 12:28:25.336360 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49212.3us
W20260709 12:28:25.337785 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:25.322335  4500 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.338474  4500 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:25.341212  4500 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:25.337549  4401 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:25.343629  4401 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:25.270900  4502 tablet_replica.cc:333] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:25.352692  4446 consensus_queue.cc:1048] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:25.356513  4502 raft_consensus.cc:2243] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.357558  4502 raft_consensus.cc:2272] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.361930  4502 ts_tablet_manager.cc:1916] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:28:25.334373 25950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 300) took 7486 ms. Trace:
I20260709 12:28:25.369678 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:25.370851 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:25.362675 24862 pending_rounds.cc:70] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 3 pending ops.
I20260709 12:28:25.372344 24862 pending_rounds.cc:77] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 29 index: 21 } timestamp: 7305625987494658048 op_type: NO_OP noop_request { }
I20260709 12:28:25.372793 24862 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:25.373095 24862 pending_rounds.cc:77] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 28 index: 20 } timestamp: 7305625931148312576 op_type: NO_OP noop_request { }
I20260709 12:28:25.373392 24862 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:25.373780 24862 pending_rounds.cc:77] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 26 index: 19 } timestamp: 7305625873758916608 op_type: NO_OP noop_request { }
I20260709 12:28:25.374117 24862 raft_consensus.cc:2889] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:25.374457 24862 raft_consensus.cc:2272] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:25.363019 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 46618.6us
I20260709 12:28:25.520452 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3b95127d51d4e5db31f5dbda1a935c0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:25.526768 26019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 138514us
I20260709 12:28:25.533908  4455 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.534693  4455 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:25.538461 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526fbe6496c445dc95df29efcbeb8cd0" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:25.543953 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "549b49961cbb430d9ece6acd81aede77" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:25.368749 25950 rpcz_store.cc:276] 0709 12:28:17.848368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.494808 (+1646440us) service_pool.cc:224] Handling call
0709 12:28:20.058151 (+563343us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b743adf3a 00007f5b7a163e9c 00007f5b7a26204e 00007f5b787232a4 00007f5b78723232 00007f5b78723161 00007f5b787230dc 00007f5b78722392 00007f5b726e048c 00007f5b726df8c6
0709 12:28:25.334296 (+5276145us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:25.035396 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:25.035937 (+   541us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:25.035973 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:28:25.035989 (+    16us) tablet.cc:662] Decoding operations
0709 12:28:25.036359 (+   370us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:25.036417 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:25.036441 (+    24us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:28:25.036728 (+   287us) tablet.cc:701] Row locks acquired
0709 12:28:25.036742 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:28:25.036881 (+   139us) write_op.cc:270] Start()
0709 12:28:25.036988 (+   107us) write_op.cc:276] Timestamp: P: 1783600105036834 usec, L: 0
0709 12:28:25.037009 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:28:25.037621 (+   612us) log.cc:844] Serialized 541 byte log entry
0709 12:28:25.043427 (+  5806us) op_driver.cc:464] REPLICATION: finished
0709 12:28:25.044473 (+  1046us) write_op.cc:301] APPLY: starting
0709 12:28:25.044638 (+   165us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:25.046368 (+  1730us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:25.046518 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:25.047390 (+   872us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:25.047562 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:28:25.047617 (+    55us) write_op.cc:312] APPLY: finished
0709 12:28:25.047831 (+   214us) log.cc:844] Serialized 25 byte log entry
0709 12:28:25.047989 (+   158us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:25.048162 (+   173us) write_op.cc:454] Released schema lock
0709 12:28:25.048203 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:25.053222 (+  5019us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:25.053261 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":49,"spinlock_wait_cycles":458982272,"child_traces":[["op",{"apply.queue_time_us":573,"apply.run_cpu_time_us":3372,"apply.run_wall_time_us":4128,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":6499,"prepare.run_cpu_time_us":2511,"prepare.run_wall_time_us":2512,"replication_time_us":6193}]]}
I20260709 12:28:25.593034 24862 tablet_replica.cc:333] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.599146 24862 raft_consensus.cc:2243] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.537437  4455 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:25.648689 24862 raft_consensus.cc:2272] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.655041 26020 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:25.658639 24862 tablet_replica.cc:333] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.669018  4500 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:25.672681 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:25.660086 24862 raft_consensus.cc:2243] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.686385 24862 raft_consensus.cc:2272] T 2d132ed1efbf42c38f5a384134455ca3 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.694252 26191 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ddbca55e34c47f2b2d10e46e4779bd0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 19)) from {username='slave'} at 127.0.0.1:54828
W20260709 12:28:25.697417 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 61025.7us
I20260709 12:28:25.706856  4401 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.720392 26018 leader_election.cc:304] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:25.724125  4401 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:25.738544  4401 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:25.744903 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3998a28d0e4c4da61419183dc7ffb4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 33 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:25.747560 26018 raft_consensus.cc:2686] T 526fbe6496c445dc95df29efcbeb8cd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:28:25.735222 25938 catalog_manager.cc:5028] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): tablet 8ddbca55e34c47f2b2d10e46e4779bd0 (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:28:25.753513 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:25.775671 26018 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:25.777905  4508 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:25.791208 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:25.792308 25951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52656 (request call id 277) took 151293 ms (client timeout 15000 ms). Trace:
W20260709 12:28:25.792801 25951 rpcz_store.cc:269] 0709 12:25:54.418508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.418772 (+   264us) service_pool.cc:224] Handling call
0709 12:25:54.600875 (+182103us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b54000528d4. stack: 00007f5b71286343 00007f5b71286240 00007f5b70c151d4 00007f5b7a953304 00007f5b7a953299 00007f5b7430f090 00007f5b743adf3a 00007f5b7a163e9c 00007f5b7a26204e 00007f5b787232a4 00007f5b78723232 00007f5b78723161 00007f5b787230dc 00007f5b78722392 00007f5b726e048c 00007f5b726df8c6
0709 12:28:25.708181 (+151107306us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":49,"spinlock_wait_cycles":13560064}
I20260709 12:28:25.804955 24862 tablet_replica.cc:333] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.806176 24862 raft_consensus.cc:2243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:25.807057 24862 raft_consensus.cc:2272] T ecbcf3556b554f8a9c1c00b01dce7f5f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.818864 26019 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:25.822371  4497 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:25.832345 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 87100.4us
I20260709 12:28:25.842041 24862 tablet_replica.cc:333] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.843111 24862 raft_consensus.cc:2243] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:25.853549 24862 raft_consensus.cc:2272] T 689b9b7393b349afbf302f0f99710830 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.857740 24862 tablet_replica.cc:333] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.858692 24862 raft_consensus.cc:2243] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.861068 24862 pending_rounds.cc:70] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:25.861404 24862 pending_rounds.cc:77] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 29 index: 20 } timestamp: 7305626001080156160 op_type: NO_OP noop_request { }
I20260709 12:28:25.861837 24862 raft_consensus.cc:2889] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:25.861720 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f720b2e4986f496694c7578beef67cd4" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:25.864528 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.873529 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.874195 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:25.879769 24862 raft_consensus.cc:2272] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.881897 26018 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:25.830336  4497 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.894094  4508 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:25.920599 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.922595 25948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 297) took 114922 ms (client timeout 15000 ms). Trace:
W20260709 12:28:25.923125 25948 rpcz_store.cc:269] 0709 12:26:30.985017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.988508 (+  3491us) service_pool.cc:224] Handling call
0709 12:28:25.900778 (+114912270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5380,"reported_tablets":49}
I20260709 12:28:25.927420 24862 tablet_replica.cc:333] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.928475 24862 raft_consensus.cc:2243] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 LEADER]: Raft consensus shutting down.
W20260709 12:28:25.929968 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:25.930300  4502 ts_tablet_manager.cc:1929] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 28.19
I20260709 12:28:25.930650 24862 raft_consensus.cc:2272] T ec9017ccb0d849c49a75c9b1dce1547e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.930763  4502 log.cc:1199] T 93ec87f552a04ce49312a2ef65f862ce P 9e23389325a6411c9b0634f4f6af54c3: Deleting WAL directory at /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/wals/93ec87f552a04ce49312a2ef65f862ce
I20260709 12:28:25.937253 24862 tablet_replica.cc:333] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.938287 24862 raft_consensus.cc:2243] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.941360 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 93ec87f552a04ce49312a2ef65f862ce (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:28:25.903478  4497 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:25.956737 24862 pending_rounds.cc:70] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:25.957052 24862 pending_rounds.cc:77] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 33 index: 21 } timestamp: 7305625968591360000 op_type: NO_OP noop_request { }
I20260709 12:28:25.958370 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:25.960078 24862 raft_consensus.cc:2889] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:25.965840 24862 raft_consensus.cc:2272] T 407574b4cda94da99be372d249d21fd5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.969061 24862 tablet_replica.cc:333] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:25.969785 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86798289f90549ed8084d260f08cb204" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:25.955976  4497 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:25.983500 26018 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:25.984766 24862 raft_consensus.cc:2243] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.987783 24862 raft_consensus.cc:2272] T d3b95127d51d4e5db31f5dbda1a935c0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.986975  4497 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:25.997514 24862 tablet_replica.cc:333] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.001673 24862 raft_consensus.cc:2243] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.003023 24862 raft_consensus.cc:2272] T 0d096e6da5114d91b7c4dd46412d2a6a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.032913 24862 tablet_replica.cc:333] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.037211 24862 raft_consensus.cc:2243] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.038617 24862 raft_consensus.cc:2272] T c47092de97434584b0810190b58e7623 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:26.045400 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:26.050464 24862 tablet_replica.cc:333] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.056279 24862 raft_consensus.cc:2243] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.057508 24862 raft_consensus.cc:2272] T 323fa2c24ef84fd39b5fe37060615814 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:26.063720 25952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52646 (request call id 289) took 133618 ms (client timeout 15000 ms). Trace:
W20260709 12:28:26.068552 25952 rpcz_store.cc:269] 0709 12:26:12.446487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.448792 (+  2305us) service_pool.cc:224] Handling call
0709 12:28:26.063500 (+133614708us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":250456,"reported_tablets":47,"spinlock_wait_cycles":5760}
I20260709 12:28:26.071599 24862 tablet_replica.cc:333] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.072726 24862 raft_consensus.cc:2243] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.073419 24862 raft_consensus.cc:2272] T 1e8b57c1b80240d483c7ca4e95d74673 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.076758 24862 tablet_replica.cc:333] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.077670 24862 raft_consensus.cc:2243] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.078850 24862 raft_consensus.cc:2272] T 0f85110b9e6d45578bf2f39e2e9ad47c P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.081879 24862 tablet_replica.cc:333] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.091900 24862 raft_consensus.cc:2243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.092694 24862 raft_consensus.cc:2272] T 9e3998a28d0e4c4da61419183dc7ffb4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.106585  4497 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.107136  4497 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:26.108613 24862 tablet_replica.cc:333] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.111394 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:26.111737 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef606226bb434b95911632eec3ede435" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 32 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:26.112134 24862 raft_consensus.cc:2243] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.113293 24862 pending_rounds.cc:70] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:26.113580 24862 pending_rounds.cc:77] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 28 index: 19 } timestamp: 7305625966173593600 op_type: NO_OP noop_request { }
I20260709 12:28:26.113969 24862 raft_consensus.cc:2889] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.114387 24862 raft_consensus.cc:2272] T 93ec87f552a04ce49312a2ef65f862ce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.114607 26020 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:26.115617  4508 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:26.118930 24862 tablet_replica.cc:333] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.121369 24862 raft_consensus.cc:2243] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.126514 24862 pending_rounds.cc:70] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:26.127838 26118 tablet_service.cc:1558] Processing DeleteTablet for tablet 93ec87f552a04ce49312a2ef65f862ce with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 20)) from {username='slave'} at 127.0.0.1:44004
I20260709 12:28:26.130326  4497 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:26.131847 24862 pending_rounds.cc:77] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 24 index: 18 } timestamp: 7305625860021993472 op_type: NO_OP noop_request { }
I20260709 12:28:26.132401 24862 raft_consensus.cc:2889] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.132818 24862 raft_consensus.cc:2272] T 2fee7aee965145c6b43db63c2b3ecca8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:26.140491 25949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52652 (request call id 295) took 180920 ms (client timeout 15000 ms). Trace:
W20260709 12:28:26.140954 25949 rpcz_store.cc:269] 0709 12:25:25.220338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.220572 (+   234us) service_pool.cc:224] Handling call
0709 12:28:26.140257 (+180919685us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3640249,"reported_tablets":49}
I20260709 12:28:26.147799 24862 tablet_replica.cc:333] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.130290 25939 catalog_manager.cc:5028] TS 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): tablet 93ec87f552a04ce49312a2ef65f862ce (table test-workload [id=2ef52bacbccb4a5fa40b3e37523f2ad9]) successfully deleted
I20260709 12:28:26.149044 24862 raft_consensus.cc:2243] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.149883 24862 raft_consensus.cc:2272] T 3fe81c3162f24fc0b8716b2b42f0676f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.153352 24862 tablet_replica.cc:333] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.160175 24862 raft_consensus.cc:2243] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.160876 24862 raft_consensus.cc:2272] T ef606226bb434b95911632eec3ede435 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.165318 24862 tablet_replica.cc:333] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.168998 24862 raft_consensus.cc:2243] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.170259 24862 raft_consensus.cc:2272] T f5a510044c5b459abfd55896cc2d21eb P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.176851 24862 tablet_replica.cc:333] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.184969 24862 raft_consensus.cc:2243] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.186084 24862 raft_consensus.cc:2272] T 549b49961cbb430d9ece6acd81aede77 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.189350 24862 tablet_replica.cc:333] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.191874 24862 raft_consensus.cc:2243] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.192649 24862 pending_rounds.cc:70] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:26.192952 24862 pending_rounds.cc:77] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 26 index: 21 } timestamp: 7305625860865589248 op_type: NO_OP noop_request { }
I20260709 12:28:26.193352 24862 raft_consensus.cc:2889] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.197255 24862 raft_consensus.cc:2272] T d0d1927cc7714e91945918d6aa1216e5 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.205798 24862 tablet_replica.cc:333] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.206796 24862 raft_consensus.cc:2243] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.207345 24862 raft_consensus.cc:2272] T 86798289f90549ed8084d260f08cb204 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.210304 24862 tablet_replica.cc:333] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.211081 24862 raft_consensus.cc:2243] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.220772 24862 pending_rounds.cc:70] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 3 pending ops.
I20260709 12:28:26.221062 24862 pending_rounds.cc:77] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 27 index: 14 } timestamp: 7305625945511616512 op_type: NO_OP noop_request { }
I20260709 12:28:26.221406 24862 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.222245 24862 pending_rounds.cc:77] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 25 index: 13 } timestamp: 7305625873819926528 op_type: NO_OP noop_request { }
I20260709 12:28:26.222569 24862 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.222937 24862 pending_rounds.cc:77] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 23 index: 12 } timestamp: 7305625824268021760 op_type: NO_OP noop_request { }
I20260709 12:28:26.223240 24862 raft_consensus.cc:2889] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.227137  4497 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.227715  4497 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:26.231942 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:26.230720  4497 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:26.234524 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec84548f93742a096e72c1bd887809a" candidate_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:26.234733 24862 raft_consensus.cc:2272] T f720b2e4986f496694c7578beef67cd4 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.236191 26019 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fd5643e7b494fa79c3d0aaf1e6e8152; no voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:26.237494  4508 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:26.279277 24862 tablet_replica.cc:333] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.281445 24862 raft_consensus.cc:2243] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.282156 24862 raft_consensus.cc:2272] T e0673bc676154f04907934c1d3a60bce P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:26.314541 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:26.315610  4489 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:26.316275  4489 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:26.318502  4489 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:26.320075 24862 tablet_replica.cc:333] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.328040 24862 raft_consensus.cc:2243] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.328779 24862 pending_rounds.cc:70] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:26.329128 24862 pending_rounds.cc:77] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 27 index: 19 } timestamp: 7305625958982991872 op_type: NO_OP noop_request { }
I20260709 12:28:26.330718 24862 raft_consensus.cc:2889] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.333388 24862 raft_consensus.cc:2272] T bec84548f93742a096e72c1bd887809a P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.337925  4493 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:26.338670  4493 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:26.339838 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:26.345616 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:26.346199  4493 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:26.346747 24862 tablet_replica.cc:333] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.348559 24862 raft_consensus.cc:2243] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.349726 24862 raft_consensus.cc:2272] T 772287743e884fe8a6dc6c516c3ad901 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.354027 24862 tablet_replica.cc:333] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.355670 24862 raft_consensus.cc:2243] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.356411 24862 pending_rounds.cc:70] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 1 pending ops.
I20260709 12:28:26.356956 24862 pending_rounds.cc:77] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 30 index: 17 } timestamp: 7305625966089916416 op_type: NO_OP noop_request { }
I20260709 12:28:26.357390 24862 raft_consensus.cc:2889] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.357827 24862 raft_consensus.cc:2272] T c0ff2e29d14b44128ecab4f62a77317f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.359913 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:26.360635 26120 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:26.360958 24862 tablet_replica.cc:333] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.359977 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:26.361954 26197 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 29.
I20260709 12:28:26.361954 24862 raft_consensus.cc:2243] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.365079 24862 raft_consensus.cc:2272] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.367287 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:26.368181  4489 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:26.368647  4489 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:26.369019  4489 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:26.372291 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:26.378553  4489 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:26.368010 24862 tablet_replica.cc:333] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.381073 24862 raft_consensus.cc:2243] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.382278 24862 raft_consensus.cc:2272] T 40485ee0f2b44c1290c368cf529eade1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.385852  4503 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:28:26.391374 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:26.391433  4503 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:26.392508  4503 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:26.394995 24862 tablet_replica.cc:333] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.395896 24862 raft_consensus.cc:2243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.396735 24862 pending_rounds.cc:70] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 2 pending ops.
I20260709 12:28:26.396979 24862 pending_rounds.cc:77] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 26 index: 18 } timestamp: 7305625926891233280 op_type: NO_OP noop_request { }
I20260709 12:28:26.397336 24862 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.397644 24862 pending_rounds.cc:77] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 24 index: 17 } timestamp: 7305625847265947648 op_type: NO_OP noop_request { }
I20260709 12:28:26.397962 24862 raft_consensus.cc:2889] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.398293 24862 raft_consensus.cc:2272] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.403774 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:26.405203  4503 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:26.406574 26200 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 30: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:26.410604 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:26.411612  4495 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:26.402635  4489 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:26.414041 24862 tablet_replica.cc:333] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.415189 24862 raft_consensus.cc:2243] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.417092  4503 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:26.417840 24862 pending_rounds.cc:70] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Trying to abort 2 pending ops.
I20260709 12:28:26.418197 24862 pending_rounds.cc:77] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 33 index: 17 } timestamp: 7305625958973124608 op_type: NO_OP noop_request { }
I20260709 12:28:26.418520 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:26.418857 24862 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.419423 26122 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 30: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:26.419413 24862 pending_rounds.cc:77] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152: Aborting op as it isn't in flight: id { term: 30 index: 16 } timestamp: 7305625860783882240 op_type: NO_OP noop_request { }
I20260709 12:28:26.420660 24862 raft_consensus.cc:2889] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:26.421198 24862 raft_consensus.cc:2272] T 7585ccc5a92245cfb1bba6ff29291731 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.424194 24862 tablet_replica.cc:333] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
W20260709 12:28:26.424572 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:26.425071 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:26.429005  4493 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:26.431150 24862 raft_consensus.cc:2243] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.433851 24862 raft_consensus.cc:2272] T be17917578e1415f98071364583240c1 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.439802 24862 tablet_replica.cc:333] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.440737 24862 raft_consensus.cc:2243] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.445917 24862 raft_consensus.cc:2272] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:26.455982 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:26.460057 24862 tablet_replica.cc:333] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.461032 24862 raft_consensus.cc:2243] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.462212 24862 raft_consensus.cc:2272] T 2ee8b7d54fee44329721d2da7c8f62ad P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.465481 24862 tablet_replica.cc:333] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.466277 24862 raft_consensus.cc:2243] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.470968 24862 raft_consensus.cc:2272] T c49cc98db5354047a91f3ec2f90daa28 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.482395 24862 tablet_replica.cc:333] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.483227 24862 raft_consensus.cc:2243] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.484449 24862 raft_consensus.cc:2272] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.494685 24862 tablet_replica.cc:333] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.495774 24862 raft_consensus.cc:2243] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.497854 24862 raft_consensus.cc:2272] T de16ce1800b64c3c94c918aa6e9cae3d P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.500839 24862 tablet_replica.cc:333] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.501627 24862 raft_consensus.cc:2243] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.502367 24862 raft_consensus.cc:2272] T 55aa4b8b33f544088007e1987c55617e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.505681 24862 tablet_replica.cc:333] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.506474 24862 raft_consensus.cc:2243] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:26.507470 24862 raft_consensus.cc:2272] T 824b6757fe414215b1cf6aa5ba238004 P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.510571 24862 tablet_replica.cc:333] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152: stopping tablet replica
I20260709 12:28:26.511272 24862 raft_consensus.cc:2243] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.512502 24862 raft_consensus.cc:2272] T b4a62528d9404c66b98e87e76ac0f21e P 8fd5643e7b494fa79c3d0aaf1e6e8152 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:26.538112 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:26.538878 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.693064 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:26.869024  4495 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:26.869678  4495 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:26.872100  4495 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:26.895043 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:26.899971 26094 leader_election.cc:336] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:26.900480 26094 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:26.901252  4495 raft_consensus.cc:2749] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:26.932950 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:26.958706 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.044071 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.095348 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:27.178314  4516 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.178953  4516 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.182332  4516 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.188449 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.190651  4489 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.191228  4489 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:27.192752 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:27.193476  4489 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.195230 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:27.200860 26094 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.201452 26094 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:27.202294  4489 raft_consensus.cc:2749] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:27.203346 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:27.203879 26168 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.204326 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:27.206152  4516 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:27.208056  4516 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.208642  4516 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:27.210320 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 49395.4us
I20260709 12:28:27.214618  4446 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.215250  4446 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.217460  4446 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.236917 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.237821 26122 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:27.239259 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:27.240332  4446 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:27.240773  4446 raft_consensus.cc:493] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.241168  4446 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:27.244086  4489 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.244721  4489 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.246977  4489 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.251983 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:27.256314 26094 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.256923 26094 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:27.258245  4489 raft_consensus.cc:2749] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:27.301527  4516 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.320453 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
W20260709 12:28:27.342232 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.356616 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:27.356768  4446 raft_consensus.cc:515] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.364136  4494 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.364814  4494 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.367051  4494 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.367564  4494 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.368050  4494 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.370137  4494 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.375130  4520 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.375778  4520 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.378019  4520 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.384737  4520 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.385304  4520 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.386245  4521 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.386822  4521 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.387491  4446 leader_election.cc:290] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.387763  4520 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.389225  4489 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.389776  4489 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.392138  4489 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.392315  4516 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.392936  4516 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.393550 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.394181 26197 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:28:27.395071  4516 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:27.397087 26168 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.397867 26168 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.389556  4521 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.398705  4521 raft_consensus.cc:2749] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:27.396330 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:27.399883  4489 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:27.399933 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.400467  4489 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:27.398783 26168 leader_election.cc:336] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.400882  4489 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:27.436599  4489 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.439612  4489 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.439703  4467 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.440428  4467 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.422268  4495 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:27.442940 26168 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.448431 26168 leader_election.cc:336] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.449136 26168 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.442816  4467 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:27.444032 26756 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.452710 26756 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.450111 26168 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.455086 26168 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.447739  4446 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.461290 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:27.462108 26197 raft_consensus.cc:3060] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:27.456461  4489 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.463671  4489 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.465898  4489 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.452080  4495 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.468045  4495 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.471975 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.472702 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
I20260709 12:28:27.474363 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689b9b7393b349afbf302f0f99710830" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:27.474928 26122 raft_consensus.cc:3060] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:27.459817  4521 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.476317  4521 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.470496  4446 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.478439  4521 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.479056  4446 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.490633 26197 raft_consensus.cc:2468] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:27.494671 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.495048 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47092de97434584b0810190b58e7623" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.495399 26120 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:27.495761 26124 raft_consensus.cc:2393] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:27.497051 26784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:27.497754 26784 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
I20260709 12:28:27.499913 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.500535 26124 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
W20260709 12:28:27.501220 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:27.501778 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:27.502355 26787 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
I20260709 12:28:27.503652 26122 raft_consensus.cc:2468] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 34.
W20260709 12:28:27.504619 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.506439 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:27.507652 26168 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.508496 26168 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.509222 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
W20260709 12:28:27.509243 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:27.511445  4523 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:27.512036  4523 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.512559  4523 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:27.523634 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.524415 26197 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:27.525815  4523 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.526460 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.526777  4524 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:27.527954 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.530803 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.532011  4523 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.532423  4521 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:27.534289  4521 raft_consensus.cc:493] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.534703  4521 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:27.535097 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a510044c5b459abfd55896cc2d21eb" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:27.535694 26787 raft_consensus.cc:3060] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:27.532866  4446 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:27.532759 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.540570  4446 raft_consensus.cc:493] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.540782 26200 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 33.
I20260709 12:28:27.541047  4446 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:27.540081 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.542039  4467 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:27.542414  4467 raft_consensus.cc:493] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.542761  4467 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:27.547271 26787 raft_consensus.cc:2468] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 35.
I20260709 12:28:27.548830 26167 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:27.531927 26169 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:27.553170 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.554034  4525 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:27.554337 26169 leader_election.cc:304] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.554704  4525 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:27.555119  4525 raft_consensus.cc:493] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.555145  4520 raft_consensus.cc:2804] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:27.555673  4525 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:27.555830  4520 raft_consensus.cc:697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:27.556854  4520 consensus_queue.cc:237] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.560305  4524 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.561373  4524 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:27.566114  4521 raft_consensus.cc:515] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.530822 26197 raft_consensus.cc:2393] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 35: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:27.568131  4516 raft_consensus.cc:2804] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:28:27.568676  4516 raft_consensus.cc:697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 35 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:27.569276  4521 leader_election.cc:290] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.569566  4516 consensus_queue.cc:237] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 34.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.558641  4446 raft_consensus.cc:515] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.574612 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbcf3556b554f8a9c1c00b01dce7f5f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:27.575189 26122 raft_consensus.cc:3060] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:27.583359 26122 raft_consensus.cc:2468] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:27.593020 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.621902  4533 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:27.622414  4533 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.622790  4533 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:27.630507  4533 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.633265  4533 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.596181  4520 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.665891 25950 catalog_manager.cc:5697] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 33 to 34, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 34 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:27.682317 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.683171 26197 raft_consensus.cc:2393] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 28: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:27.656783  4446 leader_election.cc:290] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.606408  4467 raft_consensus.cc:515] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.680809  4528 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.709067 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f85110b9e6d45578bf2f39e2e9ad47c" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:27.714555 26122 raft_consensus.cc:3060] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:27.714418  4528 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.715770 26094 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:27.675071  4520 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.710480 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.717998 26197 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 25.
I20260709 12:28:27.711853  4525 raft_consensus.cc:515] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.713863  4467 leader_election.cc:290] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.722600 26122 raft_consensus.cc:2468] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:27.688599  4523 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:27.735687 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 142778us
I20260709 12:28:27.727885  4535 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.737919  4535 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.735474  4527 raft_consensus.cc:2804] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:27.732613  4523 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.742657  4523 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.743783  4523 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.751986 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:27.740134  4535 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.750223  4527 raft_consensus.cc:697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:27.749823  4523 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.768543  4528 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.802773  4525 leader_election.cc:290] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:27.807245 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 91540.9us
I20260709 12:28:27.809039 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:27.809528 26168 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.816320  4498 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.817035  4498 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.819505  4498 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.789428  4535 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.820216  4535 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.822837  4535 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.825253  4523 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.826665  4520 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:27.827426  4527 consensus_queue.cc:237] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:27.830742 24862 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 26075)
I20260709 12:28:27.810680  4528 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:27.778721  4533 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:27.792881  4522 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.853056 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
W20260709 12:28:27.852340 26094 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.854063 26200 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
W20260709 12:28:27.863108 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.863646 26168 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.864276 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.864320 26094 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.864881 26168 leader_election.cc:336] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.865438 26168 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.866178 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.866597 26168 leader_election.cc:336] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.867100 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.867550 26168 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.855836  4522 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:27.868111 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:27.868649 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.869068 26168 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.869098 26094 leader_election.cc:336] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.869849 26094 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:27.870178  4522 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:27.845292  4528 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.871104  4528 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:27.876415 26168 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:27.877066 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:27.877485 26168 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.879896 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d096e6da5114d91b7c4dd46412d2a6a" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:27.886415 26203 raft_consensus.cc:3060] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:27.889671 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:27.892576  4532 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:27.893080  4532 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.893517  4532 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:27.897925  4524 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:27.846038  4547 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.902352  4547 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:27.903014 26168 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.882391 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.909343 26195 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 30.
I20260709 12:28:27.880003 26200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:27.917506 26200 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 34: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:27.917923 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:27.920606 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:27.918711 26788 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:27.884749  4526 raft_consensus.cc:2749] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:27.922075 26786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:27.922791 26786 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 27.
I20260709 12:28:27.925211  4547 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:27.925261 26791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:27.926322 26791 raft_consensus.cc:2393] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 34: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:27.928840 26094 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.938515 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:27.939463 26195 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
I20260709 12:28:27.949901 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:27.954994 26203 raft_consensus.cc:2468] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:28:27.961354 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.878106  4550 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.921015  4524 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
W20260709 12:28:27.959841 26168 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:27.967994 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.907660  4532 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.978533 26167 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.983877 26167 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.964506  4547 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:27.991158 26169 leader_election.cc:304] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.978152  4520 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:27.982718  4446 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:27.985308  4524 raft_consensus.cc:2749] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:28.008281 26169 leader_election.cc:304] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:27.988795  4547 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:27.988133  4521 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:28:28.013934 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:28.017189 26203 raft_consensus.cc:3060] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:28.033756 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:28.024065 26203 raft_consensus.cc:2468] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 34.
I20260709 12:28:28.035250  4549 raft_consensus.cc:2804] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:28.035808  4549 raft_consensus.cc:697] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:28.015969  4547 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:28.036731  4549 consensus_queue.cc:237] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.038759 25949 catalog_manager.cc:5697] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 30 to 31, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 31 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.042213  4528 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:27.993940  4520 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:28.044826  4520 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:28.044816 26094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111) [suppressed 305 similar messages]
W20260709 12:28:28.046917 26094 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.048925 26094 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:28.050781  4529 raft_consensus.cc:2804] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:28.051290  4529 raft_consensus.cc:697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:28.052148  4529 consensus_queue.cc:237] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.007790  4522 raft_consensus.cc:2804] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:28.053395  4522 raft_consensus.cc:697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:28.054265  4522 consensus_queue.cc:237] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.058629 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49cc98db5354047a91f3ec2f90daa28" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:28.060784 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:27.968643  4550 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.061080 26124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323fa2c24ef84fd39b5fe37060615814" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:28.061688 26124 raft_consensus.cc:3060] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:28.061776 26120 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 31.
I20260709 12:28:28.062911  4550 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.005062  4532 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.008539  4446 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:28.064281  4446 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:28.064738 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:28.077788 26124 raft_consensus.cc:2468] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 29.
W20260709 12:28:28.079454 26094 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.084046  4528 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:28.086609 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:28.091358 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:28.091980 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
W20260709 12:28:28.095422 26756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 84197.1us
I20260709 12:28:28.090720 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:28.096436 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9017ccb0d849c49a75c9b1dce1547e" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:28.097047 26203 raft_consensus.cc:3060] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:28.096596 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:28.103220 26203 raft_consensus.cc:2468] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 26.
I20260709 12:28:28.101414 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:28.106148 26094 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
W20260709 12:28:28.101857 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 108525us
I20260709 12:28:28.104878  4534 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:28.109776 26756 leader_election.cc:304] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:28.112090  4467 raft_consensus.cc:2804] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:28.112632  4467 raft_consensus.cc:697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:28.113576  4467 consensus_queue.cc:237] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:28.110004  4550 raft_consensus.cc:2804] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:28.115640  4550 raft_consensus.cc:697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:28.116658  4550 consensus_queue.cc:237] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:28.118930 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.118824 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.119510 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:28.103797  4524 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:28.125886 26756 leader_election.cc:336] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.109891  4529 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:28.129304  4541 raft_consensus.cc:2749] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:28:28.129312  4544 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:28.133812  4547 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.144884  4544 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:28.145334  4544 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:28.145385 26122 raft_consensus.cc:2393] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 34: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:28.148375  4547 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.149039  4529 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:28.110544  4534 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
W20260709 12:28:28.143936 26094 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.149870  4534 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:28.149462  4529 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:28.157171 26169 leader_election.cc:304] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.157070 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40485ee0f2b44c1290c368cf529eade1" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:28.130442 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:28.158326 26203 raft_consensus.cc:3060] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:28.158797 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.159713 26756 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:28.165247 26094 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:28.168923 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:28.175344 26203 raft_consensus.cc:2468] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 34.
I20260709 12:28:28.185218 26784 raft_consensus.cc:1275] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 34 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:28:28.184180 26094 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:28.198218  4446 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:28.204595 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:28.170007  4559 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:28.204730 25950 catalog_manager.cc:5697] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 31 to 32, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 32 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.206054  4559 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:28.207955 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:28.216603  4523 raft_consensus.cc:2804] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:28.227311  4522 consensus_queue.cc:1048] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:28.227932  4520 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:28.230494 26169 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:28.231190  4523 raft_consensus.cc:697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:28.216358  4550 raft_consensus.cc:2804] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:28.232542  4524 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:28.232843  4550 raft_consensus.cc:697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:28.232441  4523 consensus_queue.cc:237] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.233749  4550 consensus_queue.cc:237] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.238066 26122 raft_consensus.cc:1275] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
W20260709 12:28:28.240566 26094 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.238196 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:28.241910 26196 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
I20260709 12:28:28.245941 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:28.236541  4559 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.252393  4529 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.242604  4526 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.272634  4526 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.274032  4529 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.275336  4526 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.247092  4521 consensus_queue.cc:1048] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:28:28.253340  4551 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:28.281746  4551 raft_consensus.cc:493] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 8fd5643e7b494fa79c3d0aaf1e6e8152)
I20260709 12:28:28.282153  4551 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:28.292593 26196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
W20260709 12:28:28.296139 26756 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.318228 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:28.325533  4446 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:28.331023 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee8b7d54fee44329721d2da7c8f62ad" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:28.331990 26787 raft_consensus.cc:3060] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:28.314220  4535 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.308826  4544 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.349675 26196 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 32: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:28.351666 26787 raft_consensus.cc:2468] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 28.
I20260709 12:28:28.365330  4534 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:28.379012 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:28.414786 26094 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:28.415271 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.343890  4535 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.418455 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:28.390200  4544 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:28.391776 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:28.395183  4520 leader_election.cc:290] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:28.397132 26196 raft_consensus.cc:1275] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 30 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:28:28.402371 26788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
W20260709 12:28:28.402699 26168 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.413295 25950 catalog_manager.cc:5697] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 34 to 35, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 35 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.549183 26788 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 24e61c63097e4d3dbb149050e6fcc997 in this term.
W20260709 12:28:28.549256 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 146577us
I20260709 12:28:28.549870 25950 catalog_manager.cc:5697] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 27 to 28, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:28.550855 26168 leader_election.cc:336] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.388811  4551 raft_consensus.cc:515] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:28.420991 26129 raft_consensus.cc:2393] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 32: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
I20260709 12:28:28.552629  4530 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:28.553877 26167 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.557855  4536 raft_consensus.cc:2804] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:28.558013  4551 leader_election.cc:290] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:28.561280 26168 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:28.566000 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:28.566527 26756 leader_election.cc:336] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.571377 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a62528d9404c66b98e87e76ac0f21e" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:28.572001 26195 raft_consensus.cc:3060] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:28.573635  4536 raft_consensus.cc:697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
I20260709 12:28:28.574662  4536 consensus_queue.cc:237] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:28.579457 26167 leader_election.cc:304] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 24e61c63097e4d3dbb149050e6fcc997, 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.580979 26197 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 30.
I20260709 12:28:28.583359  4544 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:28.592152 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:28.605065 26195 raft_consensus.cc:2468] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
I20260709 12:28:28.607285 26756 leader_election.cc:304] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.614200  4534 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.614977  4526 consensus_queue.cc:1048] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:28.605490  4521 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:28.622452 25952 catalog_manager.cc:5697] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 33 to 34, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 34 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.627238  4535 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:28.630563 25952 catalog_manager.cc:5697] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 25 to 26, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 26 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.631933  4540 raft_consensus.cc:2804] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:28.632375 25952 catalog_manager.cc:5697] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 33 to 34, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 34 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.632635  4540 raft_consensus.cc:697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:28.634862 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.635493  4540 consensus_queue.cc:237] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:28.632412 25950 catalog_manager.cc:5697] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 28 to 29, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 29 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.649040 25950 catalog_manager.cc:5697] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 29 to 30, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 30 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.652666 25950 catalog_manager.cc:5697] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 27 to 28, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 28 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:28.668502 26093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:39865, user_credentials={real_user=slave}} blocked reactor thread for 48205.6us
W20260709 12:28:28.675287 26756 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.676602 26129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3" is_pre_election: true
I20260709 12:28:28.677316 26129 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 30.
I20260709 12:28:28.684553 26122 raft_consensus.cc:1275] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:28:28.692116 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 77325.3us
I20260709 12:28:28.693513 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.701488 26120 raft_consensus.cc:1275] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:28.701685 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:28.701535  4547 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:28.704145 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:28.705102 26197 raft_consensus.cc:2393] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 24e61c63097e4d3dbb149050e6fcc997 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
I20260709 12:28:28.712199 26200 raft_consensus.cc:1275] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:28:28.716948 26195 raft_consensus.cc:1275] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:28:28.719170 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:28.720981  4551 raft_consensus.cc:2749] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:28.724114  4543 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:28.724256  4527 consensus_queue.cc:1048] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:28.724808  4543 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.725240  4543 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:28.703265  4547 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.727465  4547 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:28.730115  4549 consensus_queue.cc:1048] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:28.736883  4520 consensus_queue.cc:1048] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:28.770341 25949 catalog_manager.cc:5697] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 31 to 32, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 32 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.780153 26195 raft_consensus.cc:1275] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:28.799264  4547 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.775113  4530 consensus_queue.cc:1048] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 12:28:28.806605 26120 raft_consensus.cc:1275] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:28.811200  4446 consensus_queue.cc:1048] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:28.810029  4543 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:28.816879  4543 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:28.818508 26120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "9e23389325a6411c9b0634f4f6af54c3"
I20260709 12:28:28.819052  4547 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:28.819370 26120 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a981c9a7972c4e99a0903924c64fbb5c in current term 31: Already voted for candidate 9e23389325a6411c9b0634f4f6af54c3 in this term.
W20260709 12:28:28.820478 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.832841 24862 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:28.835181 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:28.837365 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:28.838558 26195 raft_consensus.cc:2393] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e23389325a6411c9b0634f4f6af54c3 in current term 31: Already voted for candidate a981c9a7972c4e99a0903924c64fbb5c in this term.
W20260709 12:28:28.842711 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.843592 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:28.856567  4527 consensus_queue.cc:1048] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:28.861665 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:28.866782 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:28.871997 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:28.871991 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:28.873523 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:28.877853 26197 raft_consensus.cc:1275] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:28:28.877123 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:28.884451  4547 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:28:28.874243 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.899616 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:28.925289  4540 consensus_queue.cc:1048] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.014s
W20260709 12:28:28.954767 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:28.955327 26168 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:28.973261  4520 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:28:29.017211 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:29.175858 26203 raft_consensus.cc:1275] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:28:29.188705 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 61142.2us
I20260709 12:28:29.201068 26787 raft_consensus.cc:1275] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Refusing update from remote peer a981c9a7972c4e99a0903924c64fbb5c: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:29.218212  4533 consensus_queue.cc:1048] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:28:29.243177 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.243919 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:29.252499  4520 consensus_queue.cc:1048] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:28:29.253482 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:29.377978 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.489727 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:29.494186 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.620359 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:29.775324 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.815569 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:29.833488 24862 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 26075)
I20260709 12:28:29.855924  4494 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.859308  4494 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:29.861685  4494 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:29.867686 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.889165 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:29.926045 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:29.927294 26197 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 34.
W20260709 12:28:29.935988 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:29.944711 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:29.946265  4494 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:29.946820  4494 raft_consensus.cc:493] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:29.947804 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:29.947312  4494 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:30.018357  4494 raft_consensus.cc:515] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:30.025072 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772287743e884fe8a6dc6c516c3ad901" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:30.025753 26195 raft_consensus.cc:3060] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:30.020649  4494 leader_election.cc:290] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:30.034171 26756 leader_election.cc:336] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:30.051786 26195 raft_consensus.cc:2468] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 35.
I20260709 12:28:30.054947 26756 leader_election.cc:304] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:30.065405  4599 raft_consensus.cc:2804] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:28:30.066175  4599 raft_consensus.cc:697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
I20260709 12:28:30.067924  4599 consensus_queue.cc:237] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:30.085695 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.103055 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.123880 25950 catalog_manager.cc:5697] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 33 to 35, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 35 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:30.181325 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.200709 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:30.220419 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:30.269852  4599 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.270568  4599 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:30.290555  4599 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:30.294483 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:30.295827 26195 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 31.
W20260709 12:28:30.300722 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.302055 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:30.312760  4599 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:30.313246  4599 raft_consensus.cc:493] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.313601  4599 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:30.338642 26756 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:30.361610 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:30.469252 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:30.473634 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.457700  4599 raft_consensus.cc:515] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:30.479693  4599 leader_election.cc:290] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:30.483238 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17917578e1415f98071364583240c1" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:30.483985 26195 raft_consensus.cc:3060] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:30.487372 26756 leader_election.cc:336] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:30.473335 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:30.495414  4598 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.495946  4598 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:30.498381  4598 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:30.503567 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:30.504343 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
W20260709 12:28:30.503624 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:30.505795 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.509642 26203 raft_consensus.cc:1275] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:28:30.511698  4494 consensus_queue.cc:1048] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:30.518512 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:30.521898 26195 raft_consensus.cc:2468] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e61c63097e4d3dbb149050e6fcc997 in term 32.
W20260709 12:28:30.522445 26094 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:30.524518  4598 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:30.525007  4598 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.525449  4598 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:30.537840  4598 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:30.539803 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.550925 26756 leader_election.cc:304] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:30.556500  4494 raft_consensus.cc:2804] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:30.557148  4494 raft_consensus.cc:697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Becoming Leader. State: Replica: 24e61c63097e4d3dbb149050e6fcc997, State: Running, Role: LEADER
W20260709 12:28:30.558480 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:30.558364  4494 consensus_queue.cc:237] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:30.580957 26094 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:30.596788  4598 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:30.606719 26203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8d039bdedc4e1db6d66b58d4b56c4f" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:30.607462 26203 raft_consensus.cc:3060] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:30.618333 25950 catalog_manager.cc:5697] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 reported cstate change: term changed from 30 to 32, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132). New cstate: current_term: 32 leader_uuid: "24e61c63097e4d3dbb149050e6fcc997" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:30.620433 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:30.627317 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:30.636451 26203 raft_consensus.cc:2468] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 33.
I20260709 12:28:30.638589 26094 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:30.644022  4598 raft_consensus.cc:2804] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:30.645015  4592 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.645635  4592 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:30.648566  4592 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:30.651726 26195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:30.652627 26195 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 31.
I20260709 12:28:30.680896 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:30.685882  4610 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:28:30.686362 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.686333  4610 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.687119  4610 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:30.695816  4610 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:30.701967  4598 raft_consensus.cc:697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
W20260709 12:28:30.697674 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:30.703167  4598 consensus_queue.cc:237] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:30.706943  4610 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Requested vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:30.710664 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6389d613e36a42958d24b00099843b93" candidate_uuid: "9e23389325a6411c9b0634f4f6af54c3" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c"
I20260709 12:28:30.740099 26197 raft_consensus.cc:3060] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:30.738324 26094 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:30.817250 26197 raft_consensus.cc:2468] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e23389325a6411c9b0634f4f6af54c3 in term 32.
W20260709 12:28:30.834007 24862 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:30.839380 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:30.847635 26094 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152
I20260709 12:28:30.872090  4598 raft_consensus.cc:2804] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:30.872658  4598 raft_consensus.cc:697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Becoming Leader. State: Replica: 9e23389325a6411c9b0634f4f6af54c3, State: Running, Role: LEADER
I20260709 12:28:30.873623  4598 consensus_queue.cc:237] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:30.891407 25950 catalog_manager.cc:5697] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 31 to 33, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 33 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:30.910668 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:30.929360 25950 catalog_manager.cc:5697] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 reported cstate change: term changed from 29 to 32, leader changed from 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129) to 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130). New cstate: current_term: 32 leader_uuid: "9e23389325a6411c9b0634f4f6af54c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:30.946166 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:30.957144 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:30.966688 26203 raft_consensus.cc:1275] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 24e61c63097e4d3dbb149050e6fcc997: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:30.968436  4599 consensus_queue.cc:1048] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:28:30.979759 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.035207 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.053237 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:31.089902 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.131398 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.158111 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.208333 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.237699 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.302258 26094 consensus_peers.cc:597] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.304605 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.307572 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:31.353749 26203 raft_consensus.cc:1275] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 32 index: 16. (index mismatch)
I20260709 12:28:31.357852  4598 consensus_queue.cc:1048] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:31.377031 26197 raft_consensus.cc:1275] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Refusing update from remote peer 9e23389325a6411c9b0634f4f6af54c3: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:28:31.404865  4616 consensus_queue.cc:1048] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:28:31.438500 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.479036 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 47822.2us
W20260709 12:28:31.553493 26094 consensus_peers.cc:597] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.607404 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.613318 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.624058 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.662329 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.810351 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:31.838804 24862 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:31.851244 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:31.860450 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.992451 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.192682 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:38261, user_credentials={real_user=slave}} blocked reactor thread for 56539.6us
W20260709 12:28:32.219883 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.284088 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.349514 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.429081 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.505607 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.641913 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.694260 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:32.726274 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.739668 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.813397 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.839521 24862 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:32.850844 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:32.916548 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:32.942539 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:32.993556 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:33.034264 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.044948 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.050915 26168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111) [suppressed 443 similar messages]
W20260709 12:28:33.051707 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.070891 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.114195 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:33.127960 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.309412 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.419261 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:33.470407 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:33.495092 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:33.530844 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.561853 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:33.610884 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:33.640144 26094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 66993.9us
W20260709 12:28:33.707816 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:33.714182 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.722182 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:33.724574 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:33.832796 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.840200 24862 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:33.941953 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:34.148836 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.180694 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:34.263754 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:34.330492 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.378347 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:34.458824 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:34.513037 26094 consensus_peers.cc:597] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.542996 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:34.565876 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:34.630689 26094 consensus_peers.cc:597] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.702291 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:34.770543 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:34.840714 24862 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:35.008011 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:35.071369 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.069139 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.124691 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.142976 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.146533 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.180022 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:35.228791 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.363217 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.409425 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.412441 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.511430 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.685519 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.777422 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.801641 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.805577 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.841230 24862 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:36.033145 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:36.066104 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:36.249866 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:36.315253 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:36.369666 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:36.422722 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:36.466993 26168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44989, user_credentials={real_user=slave}} blocked reactor thread for 46148.9us
W20260709 12:28:36.533034 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800175a60 after lost signal to thread 24865
W20260709 12:28:36.534235 25983 debug-util.cc:398] Leaking SignalData structure 0x7b0800333600 after lost signal to thread 25986
W20260709 12:28:36.535620 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08003812c0 after lost signal to thread 26153
W20260709 12:28:36.536835 25983 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1fe0 after lost signal to thread 26227
W20260709 12:28:36.538313 25983 debug-util.cc:398] Leaking SignalData structure 0x7b080032b680 after lost signal to thread 26815
W20260709 12:28:36.762356 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:36.842206 24862 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:36.898566 26094 consensus_peers.cc:597] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:36.920876 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:36.964278 26094 consensus_peers.cc:597] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.333938 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.354817 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.390493 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.406991 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:37.500872 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.509204 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.510058 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:37.510547 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.511070 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.511808 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.516561 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.517140 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.524914 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.563882 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.564800 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:37.751207 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.766593 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.842757 24862 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:37.856114 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 53349us
W20260709 12:28:37.876106 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.953928 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:37.988858 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.996287 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:38.030381 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:38.031100 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:38.061902 26756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111) [suppressed 386 similar messages]
W20260709 12:28:38.066421 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.068782 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:38.176522 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.210014 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.250579 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:38.268633 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:38.312431 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.369978 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.502269 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.546651 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:38.618852 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:38.672017 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:38.843328 24862 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:38.844812 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.846493 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:39.016949 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:39.097114 26094 consensus_peers.cc:597] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.172382 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:39.196563 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.287169 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:39.308251 26169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 62333us
W20260709 12:28:39.409857 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:39.418712 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.474725 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.496541 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.504556 26094 consensus_peers.cc:597] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.521656 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:39.523912 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:39.529856 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.556320 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:39.561045 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.614692 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:39.770292 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:39.799348 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:39.823748 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.844087 24862 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:40.063079 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.083731 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.097290 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.104743 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:40.115458 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.140472 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.180660 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.298116 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:40.333434 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.336644 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:40.341305 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:40.396564 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.534286 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:40.564158 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.677629 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:40.717538 26757 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:40611, user_credentials={real_user=slave}} blocked reactor thread for 60114us
W20260709 12:28:40.743887 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.748945 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.793740 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:40.794566 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.795823 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.828136 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.844619 24862 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:40.860554 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:40.891788 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:40.935098 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:41.117143 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:41.137866 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:41.336301 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:41.521102 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:41.523891 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:41.539604 26094 consensus_peers.cc:597] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:41.543298 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:41.615888 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:41.637812 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:41.649888 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:41.741339 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:41.812497 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:41.845186 24862 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:41.868958 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:41.887732 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:41.898767 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:41.924396 26094 consensus_peers.cc:597] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:41.945230 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:42.025192 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.038612 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.236515 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.237149 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.249537 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:42.330180 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:42.486403 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.506160 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:42.542733 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.548058 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:42.690644 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:42.711018 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.751080 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.815519 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:42.837977 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:42.846218 24862 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:42.853968 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:42.897521 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:42.897855 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.046869 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:43.073086 26168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111) [suppressed 473 similar messages]
W20260709 12:28:43.076326 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:43.123256 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.186775 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:43.229431 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.293236 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.294727 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.314150 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.391412 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.459247 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:43.493340 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.591500 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.717826 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.757107 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.765244 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:43.823153 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.846699 24862 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 26075)
W20260709 12:28:43.880720 26094 consensus_peers.cc:597] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.889528 26094 consensus_peers.cc:597] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:43.943612 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:43.973098 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:44.052700 26094 consensus_peers.cc:597] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:44.105037 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:44.159854 26094 consensus_peers.cc:597] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:44.255645 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:44.271100 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:44.405476 26094 consensus_peers.cc:597] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:44.426466 26094 consensus_peers.cc:597] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:44.539500 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:44.559340 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:44.725934 26094 consensus_peers.cc:597] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:44.744829 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:44.787521 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:44.836913 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:44.907902 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:44.987507 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:45.011368 26094 consensus_peers.cc:597] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:45.116835 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
W20260709 12:28:45.152287 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.170038 26094 consensus_peers.cc:597] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.244601 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:45.273458 26094 consensus_peers.cc:597] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:45.298362 26094 consensus_peers.cc:597] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.307188 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:45.351565 26168 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.402395 26094 consensus_peers.cc:597] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.456609 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:45.489699 26168 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:45.507524 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
W20260709 12:28:45.521148 26094 consensus_peers.cc:597] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.538854 26757 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.564227 26757 consensus_peers.cc:597] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.574746 26757 consensus_peers.cc:597] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.580624 26169 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.583794 26168 consensus_peers.cc:597] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:45.584455 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:45.606494 26757 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.612872 26757 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.622265 26169 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.649722 26757 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.653041 26757 consensus_peers.cc:597] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.656256 26169 consensus_peers.cc:597] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.657969 26757 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.666664 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:45.683168 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:45.685088 26094 consensus_peers.cc:597] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.687397 26169 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.692188 26169 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.698252 26169 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.692348 24862 tablet_replica.cc:333] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.699712 24862 raft_consensus.cc:2243] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.702222 24862 raft_consensus.cc:2272] T 0bba37d4e722416198d4e0aa03fc3d5f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.703389 26757 consensus_peers.cc:597] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.703521 26168 consensus_peers.cc:597] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:45.704052 26169 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.721602 26757 consensus_peers.cc:597] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.722210 24862 tablet_replica.cc:333] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
W20260709 12:28:45.724813 26168 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:45.728492 26757 consensus_peers.cc:597] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.731348 24862 raft_consensus.cc:2243] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Raft consensus shutting down.
W20260709 12:28:45.733752 26169 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.735420 26757 consensus_peers.cc:597] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.733721 24862 raft_consensus.cc:2272] T d2b3e794c66a4dc19b6590272eae81f7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.743685 24862 tablet_replica.cc:333] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.744827 24862 raft_consensus.cc:2243] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.761363 24862 raft_consensus.cc:2272] T 6389d613e36a42958d24b00099843b93 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.764745 24862 tablet_replica.cc:333] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.765905 24862 raft_consensus.cc:2243] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.767387 24862 raft_consensus.cc:2272] T 891138e164dc40a7af7618323d52397f P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.770704 24862 tablet_replica.cc:333] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.771756 24862 raft_consensus.cc:2243] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.772696 24862 raft_consensus.cc:2272] T c7ad94af29be4656bf3ce8a023d19f7d P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.777434 26169 consensus_peers.cc:597] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.780763 24862 tablet_replica.cc:333] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.781920 24862 raft_consensus.cc:2243] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.782783 24862 raft_consensus.cc:2272] T 2d132ed1efbf42c38f5a384134455ca3 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.791687 26757 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.794131 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:45.794723 24862 tablet_replica.cc:333] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.795881 24862 raft_consensus.cc:2243] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.797465 24862 raft_consensus.cc:2272] T 47b65a35d65c481985263a8a2546d39d P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.800778 24862 tablet_replica.cc:333] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.813055 24862 raft_consensus.cc:2243] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.814406 24862 raft_consensus.cc:2272] T 824b6757fe414215b1cf6aa5ba238004 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.814503 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:45.817770 24862 tablet_replica.cc:333] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.819157 24862 raft_consensus.cc:2243] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.820276 24862 raft_consensus.cc:2272] T 526fbe6496c445dc95df29efcbeb8cd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.824194 24862 tablet_replica.cc:333] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.836966 24862 raft_consensus.cc:2243] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.837819 24862 raft_consensus.cc:2272] T 49f4a1a801de461182f8315a884b9889 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.840746 24862 tablet_replica.cc:333] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.841688 24862 raft_consensus.cc:2243] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.842399 24862 raft_consensus.cc:2272] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.844591 26757 consensus_peers.cc:597] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.844985 26169 consensus_peers.cc:597] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.845367 24862 tablet_replica.cc:333] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.846453 24862 raft_consensus.cc:2243] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.847400 24862 raft_consensus.cc:2272] T ecbcf3556b554f8a9c1c00b01dce7f5f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.859282 26169 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.863353 24862 tablet_replica.cc:333] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.864342 24862 raft_consensus.cc:2243] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.865515 24862 raft_consensus.cc:2272] T 689b9b7393b349afbf302f0f99710830 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.868768 24862 tablet_replica.cc:333] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.869789 24862 raft_consensus.cc:2243] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.871133 24862 raft_consensus.cc:2272] T ec9017ccb0d849c49a75c9b1dce1547e P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.872027 26757 consensus_peers.cc:597] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.874133 24862 tablet_replica.cc:333] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.885058 24862 raft_consensus.cc:2243] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.885913 24862 raft_consensus.cc:2272] T ddc5e875aa744c50b603a5072bc7305b P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.886026 26757 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.889680 24862 tablet_replica.cc:333] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.890733 24862 raft_consensus.cc:2243] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 LEADER]: Raft consensus shutting down.
W20260709 12:28:45.892356 26169 consensus_peers.cc:597] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.892360 24862 raft_consensus.cc:2272] T c47092de97434584b0810190b58e7623 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.896108 24862 tablet_replica.cc:333] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.905009 24862 raft_consensus.cc:2243] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Raft consensus shutting down.
W20260709 12:28:45.915930 26168 consensus_peers.cc:597] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:45.917219 24862 raft_consensus.cc:2272] T 4db91b54f05348e6855f89c508d0be01 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.920142 24862 tablet_replica.cc:333] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.921028 24862 raft_consensus.cc:2243] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.921583 24862 raft_consensus.cc:2272] T 46aa595bcb23423fb51ff268d34a3bc7 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.924352 24862 tablet_replica.cc:333] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.925216 24862 raft_consensus.cc:2243] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.926003 24862 raft_consensus.cc:2272] T 407574b4cda94da99be372d249d21fd5 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.929035 24862 tablet_replica.cc:333] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.930006 24862 raft_consensus.cc:2243] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.931586 24862 raft_consensus.cc:2272] T d3b95127d51d4e5db31f5dbda1a935c0 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.931892 26169 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.935415 24862 tablet_replica.cc:333] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.936586 24862 raft_consensus.cc:2243] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.937294 24862 raft_consensus.cc:2272] T 323fa2c24ef84fd39b5fe37060615814 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.940100 24862 tablet_replica.cc:333] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.940964 24862 raft_consensus.cc:2243] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.941573 24862 raft_consensus.cc:2272] T 1e8b57c1b80240d483c7ca4e95d74673 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.944456 24862 tablet_replica.cc:333] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.951395 24862 raft_consensus.cc:2243] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.953063 24862 raft_consensus.cc:2272] T 0f85110b9e6d45578bf2f39e2e9ad47c P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.957115 24862 tablet_replica.cc:333] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.961292 24862 raft_consensus.cc:2243] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.962971 24862 raft_consensus.cc:2272] T adcc76aa423a40eb95d31d98a8c8776f P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.966714 24862 tablet_replica.cc:333] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:45.970688 24862 raft_consensus.cc:2243] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:45.971467 26169 consensus_peers.cc:597] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:45.971947 26168 consensus_peers.cc:597] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:45.972924 24862 raft_consensus.cc:2272] T 2fee7aee965145c6b43db63c2b3ecca8 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.976962 24862 tablet_replica.cc:333] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
W20260709 12:28:45.977855 26169 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.995107 24862 raft_consensus.cc:2243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.001392 24862 raft_consensus.cc:2272] T 9e3998a28d0e4c4da61419183dc7ffb4 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.006102 24862 tablet_replica.cc:333] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.010159 24862 raft_consensus.cc:2243] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.011749 24862 raft_consensus.cc:2272] T 955755ce63f743b9a01d5871aff94f6c P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.015228 24862 tablet_replica.cc:333] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.019035 24862 raft_consensus.cc:2243] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.021064 24862 raft_consensus.cc:2272] T 3fe81c3162f24fc0b8716b2b42f0676f P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.025722 24862 tablet_replica.cc:333] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.026907 24862 raft_consensus.cc:2243] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.027778 24862 raft_consensus.cc:2272] T eac7044e09a64594b8fcd962f1d387b7 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.030903 24862 tablet_replica.cc:333] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.031790 24862 raft_consensus.cc:2243] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.032527 24862 raft_consensus.cc:2272] T d0d1927cc7714e91945918d6aa1216e5 P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.035723 24862 tablet_replica.cc:333] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.036582 24862 raft_consensus.cc:2243] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.037302 24862 raft_consensus.cc:2272] T f720b2e4986f496694c7578beef67cd4 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.040047 24862 tablet_replica.cc:333] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.041011 24862 raft_consensus.cc:2243] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.041811 24862 raft_consensus.cc:2272] T e0673bc676154f04907934c1d3a60bce P 9e23389325a6411c9b0634f4f6af54c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.044991 24862 tablet_replica.cc:333] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.045820 24862 raft_consensus.cc:2243] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.046967 24862 raft_consensus.cc:2272] T c0ff2e29d14b44128ecab4f62a77317f P 9e23389325a6411c9b0634f4f6af54c3 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.049978 24862 tablet_replica.cc:333] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.050899 24862 raft_consensus.cc:2243] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.051795 24862 raft_consensus.cc:2272] T 94c3d95514eb45178f2e5aa92e868e36 P 9e23389325a6411c9b0634f4f6af54c3 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.063055 24862 tablet_replica.cc:333] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
W20260709 12:28:46.065135 26169 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:46.068910 24862 raft_consensus.cc:2243] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.069746 24862 raft_consensus.cc:2272] T a34019ceec5a4e4e877bed7351d1d91d P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.084095 24862 tablet_replica.cc:333] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.085049 24862 raft_consensus.cc:2243] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.085800 24862 raft_consensus.cc:2272] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P 9e23389325a6411c9b0634f4f6af54c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.088722 24862 tablet_replica.cc:333] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.089841 24862 raft_consensus.cc:2243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.090639 24862 raft_consensus.cc:2272] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 9e23389325a6411c9b0634f4f6af54c3 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.105696 24862 tablet_replica.cc:333] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.106644 24862 raft_consensus.cc:2243] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.108700 24862 raft_consensus.cc:2272] T 40485ee0f2b44c1290c368cf529eade1 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.114733 24862 tablet_replica.cc:333] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.115554 24862 raft_consensus.cc:2243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.117000 24862 raft_consensus.cc:2272] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.119890 24862 tablet_replica.cc:333] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.120697 24862 raft_consensus.cc:2243] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.121832 24862 raft_consensus.cc:2272] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 9e23389325a6411c9b0634f4f6af54c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.124895 24862 tablet_replica.cc:333] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.125872 24862 raft_consensus.cc:2243] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.127345 24862 raft_consensus.cc:2272] T c49cc98db5354047a91f3ec2f90daa28 P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.130316 24862 tablet_replica.cc:333] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.131109 24862 raft_consensus.cc:2243] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.132889 24862 raft_consensus.cc:2272] T 8c8d039bdedc4e1db6d66b58d4b56c4f P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:46.134464 26094 consensus_peers.cc:597] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:46.135959 24862 tablet_replica.cc:333] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.136842 24862 raft_consensus.cc:2243] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.137550 24862 raft_consensus.cc:2272] T e9b765338c61436481fe416f17637ab2 P 9e23389325a6411c9b0634f4f6af54c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.140528 24862 tablet_replica.cc:333] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.141853 24862 raft_consensus.cc:2243] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.148212 24862 raft_consensus.cc:2272] T 7585ccc5a92245cfb1bba6ff29291731 P 9e23389325a6411c9b0634f4f6af54c3 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.151578 24862 tablet_replica.cc:333] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.152483 24862 raft_consensus.cc:2243] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.153232 24862 raft_consensus.cc:2272] T 0cbe5295e8db495985835a50d39e4438 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:46.160290 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:46.162591 24862 tablet_replica.cc:333] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.163509 24862 raft_consensus.cc:2243] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:46.165154 24862 raft_consensus.cc:2272] T ef606226bb434b95911632eec3ede435 P 9e23389325a6411c9b0634f4f6af54c3 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.168408 24862 tablet_replica.cc:333] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.169267 24862 raft_consensus.cc:2243] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.170152 24862 raft_consensus.cc:2272] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 9e23389325a6411c9b0634f4f6af54c3 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:46.170254 26168 consensus_peers.cc:597] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:46.181951 24862 tablet_replica.cc:333] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3: stopping tablet replica
I20260709 12:28:46.182834 24862 raft_consensus.cc:2243] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:46.185168 24862 raft_consensus.cc:2272] T de16ce1800b64c3c94c918aa6e9cae3d P 9e23389325a6411c9b0634f4f6af54c3 [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:46.370059 26168 consensus_peers.cc:597] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:46.388438 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:46.457794 26168 consensus_peers.cc:597] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:46.591830 26168 consensus_peers.cc:597] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:46.592463 26168 consensus_peers.cc:597] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:46.596870 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:46.714462  4652 raft_consensus.cc:493] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:46.715247  4652 raft_consensus.cc:515] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:46.716667  4659 raft_consensus.cc:493] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:46.717240  4659 raft_consensus.cc:515] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:46.719440  4659 leader_election.cc:290] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:46.727277 26168 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:46.734510 26169 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:46.743074 26168 leader_election.cc:336] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:46.747095 26168 leader_election.cc:304] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:46.748167  4661 raft_consensus.cc:2749] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:46.739178  4652 leader_election.cc:290] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:46.762645 26169 leader_election.cc:336] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:46.763862 26169 leader_election.cc:304] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:46.763317 26168 consensus_peers.cc:597] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:46.765405  4652 raft_consensus.cc:2749] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:46.813112  4652 raft_consensus.cc:493] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:46.813671  4652 raft_consensus.cc:515] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:46.830736 26168 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:46.834277  4652 leader_election.cc:290] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:46.865568 26169 leader_election.cc:336] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:46.869565 26169 leader_election.cc:304] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:46.877343  4652 raft_consensus.cc:2749] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:46.927222  4658 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:46.927800  4658 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:46.930286  4658 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:46.937136 26756 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:46.943651 26168 consensus_peers.cc:597] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:46.947700  4652 raft_consensus.cc:493] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:46.948378  4652 raft_consensus.cc:515] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:46.950326 26757 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:46.960974 26169 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:46.961305  4658 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:46.962129  4658 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:46.963183 26757 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:46.964558  4658 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:46.966707 26197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "24e61c63097e4d3dbb149050e6fcc997" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a981c9a7972c4e99a0903924c64fbb5c" is_pre_election: true
I20260709 12:28:46.970521  4648 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:46.982491  4652 leader_election.cc:290] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:46.986019 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:47.001394 26168 leader_election.cc:336] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.002008  4652 raft_consensus.cc:493] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.005712  4652 raft_consensus.cc:515] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:47.013047 26169 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:47.003432 26757 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.014628 26757 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:47.016142  4658 raft_consensus.cc:2749] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:47.015187  4652 leader_election.cc:290] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:47.003968 26168 leader_election.cc:304] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.017995  4652 raft_consensus.cc:2749] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:47.037071  4658 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.037750  4658 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:47.039880  4658 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:47.045244 26756 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:47.046861 26168 leader_election.cc:336] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.052382 26168 leader_election.cc:304] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.053392  4652 raft_consensus.cc:2749] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:47.063761 26168 consensus_peers.cc:597] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:47.066231 26757 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.066860 26757 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.067680  4658 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:47.109771 26168 consensus_peers.cc:597] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:47.176144 26168 consensus_peers.cc:597] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:47.216161  4664 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.216753  4664 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:47.219002  4664 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:47.226413 26756 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:47.233093 26757 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.236922 26757 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.238157  4664 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:47.261126  4661 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.261837  4661 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:47.266498 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997" is_pre_election: true
I20260709 12:28:47.267186 26787 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 32.
I20260709 12:28:47.269071 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:47.270105  4659 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:47.270483  4659 raft_consensus.cc:493] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting leader election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.270829  4659 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:47.278858  4652 raft_consensus.cc:493] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.279464  4652 raft_consensus.cc:515] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:47.281538  4652 leader_election.cc:290] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:47.291342 26168 consensus_peers.cc:597] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:47.298792  4661 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:47.300302  4661 raft_consensus.cc:493] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.301790  4661 raft_consensus.cc:515] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:47.305099  4661 leader_election.cc:290] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:47.327862 26169 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:47.311062 26168 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:47.332022 26169 leader_election.cc:336] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:47.333837 26168 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.337574 26169 leader_election.cc:304] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.342804  4661 raft_consensus.cc:2749] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:47.341094  4659 raft_consensus.cc:515] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:47.346535 26787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db91b54f05348e6855f89c508d0be01" candidate_uuid: "a981c9a7972c4e99a0903924c64fbb5c" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "24e61c63097e4d3dbb149050e6fcc997"
I20260709 12:28:47.346939  4661 raft_consensus.cc:493] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.347123 26787 raft_consensus.cc:3060] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:47.347606  4661 raft_consensus.cc:515] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:47.350037  4661 leader_election.cc:290] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:47.356452  4659 leader_election.cc:290] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: Requested vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), 24e61c63097e4d3dbb149050e6fcc997 (127.24.71.132:39865)
I20260709 12:28:47.368456 26787 raft_consensus.cc:2468] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a981c9a7972c4e99a0903924c64fbb5c in term 33.
W20260709 12:28:47.368456 26168 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.369966 26167 leader_election.cc:304] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997, a981c9a7972c4e99a0903924c64fbb5c; no voters: 
I20260709 12:28:47.373831  4659 raft_consensus.cc:2804] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:47.374444  4659 raft_consensus.cc:697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 33 LEADER]: Becoming Leader. State: Replica: a981c9a7972c4e99a0903924c64fbb5c, State: Running, Role: LEADER
W20260709 12:28:47.370815 26169 leader_election.cc:336] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.377069 26169 leader_election.cc:304] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.378516  4668 raft_consensus.cc:2749] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:47.375890  4659 consensus_queue.cc:237] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:47.395450 26169 leader_election.cc:336] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:47.396430 26169 leader_election.cc:336] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.396904 26169 leader_election.cc:304] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.397781  4668 raft_consensus.cc:2749] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:47.433981 25949 catalog_manager.cc:5697] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c reported cstate change: term changed from 32 to 33, leader changed from 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130) to a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131). New cstate: current_term: 33 leader_uuid: "a981c9a7972c4e99a0903924c64fbb5c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.449231  4668 raft_consensus.cc:493] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.449820  4668 raft_consensus.cc:515] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
I20260709 12:28:47.452119  4668 leader_election.cc:290] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:47.462121 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
W20260709 12:28:47.462592 26168 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.436376  4658 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.463469  4658 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:47.469336  4664 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.469861  4664 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:47.472530 26757 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.472018  4664 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:47.476884 26756 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.478672 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:47.477810 26757 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.477916  4658 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:47.482285  4664 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:47.489720 26169 leader_election.cc:336] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:47.490360 26169 leader_election.cc:304] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.491127  4652 raft_consensus.cc:2749] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:47.492796 26756 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.493336 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.494058  4658 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:47.511586 26168 consensus_peers.cc:597] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:47.520617 26168 consensus_peers.cc:597] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:47.528985  4659 raft_consensus.cc:493] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:47.529721  4659 raft_consensus.cc:515] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } }
W20260709 12:28:47.544363 26169 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:47.545969 26168 leader_election.cc:336] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:47.548725 26168 leader_election.cc:304] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a981c9a7972c4e99a0903924c64fbb5c; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:47.532171  4659 leader_election.cc:290] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:47.551087  4652 raft_consensus.cc:2749] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:47.578189 26168 consensus_peers.cc:597] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:47.603401 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:28:47.623006 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
W20260709 12:28:47.645103 26756 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:28:47.667500 26756 consensus_peers.cc:597] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.710065 26756 consensus_peers.cc:597] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.726804 26756 consensus_peers.cc:597] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.730379 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:47.731041 26756 consensus_peers.cc:597] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.731868 24862 tablet_replica.cc:333] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.732956 24862 raft_consensus.cc:2243] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.733587 24862 raft_consensus.cc:2272] T 0bba37d4e722416198d4e0aa03fc3d5f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.736800 24862 tablet_replica.cc:333] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.737738 24862 raft_consensus.cc:2243] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.738381 24862 raft_consensus.cc:2272] T d2b3e794c66a4dc19b6590272eae81f7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.741189 24862 tablet_replica.cc:333] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.742749 24862 raft_consensus.cc:2243] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.743505 24862 raft_consensus.cc:2272] T 6389d613e36a42958d24b00099843b93 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.746333 24862 tablet_replica.cc:333] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.747208 24862 raft_consensus.cc:2243] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.748925 24862 raft_consensus.cc:2272] T 47b65a35d65c481985263a8a2546d39d P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.751765 24862 tablet_replica.cc:333] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.752599 24862 raft_consensus.cc:2243] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.753218 24862 raft_consensus.cc:2272] T c7ad94af29be4656bf3ce8a023d19f7d P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.755914 24862 tablet_replica.cc:333] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.756806 24862 raft_consensus.cc:2243] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.757896 24862 raft_consensus.cc:2272] T 2d132ed1efbf42c38f5a384134455ca3 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.760600 24862 tablet_replica.cc:333] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.761479 24862 raft_consensus.cc:2243] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.762152 24862 raft_consensus.cc:2272] T ddc5e875aa744c50b603a5072bc7305b P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.765074 24862 tablet_replica.cc:333] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.765883 24862 raft_consensus.cc:2243] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.771165 24862 raft_consensus.cc:2272] T ecbcf3556b554f8a9c1c00b01dce7f5f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.774191 24862 tablet_replica.cc:333] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.775107 24862 raft_consensus.cc:2243] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.776391 24862 raft_consensus.cc:2272] T 689b9b7393b349afbf302f0f99710830 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.779454 24862 tablet_replica.cc:333] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.780323 24862 raft_consensus.cc:2243] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.781025 24862 raft_consensus.cc:2272] T 46aa595bcb23423fb51ff268d34a3bc7 P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.783852 24862 tablet_replica.cc:333] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.784883 24862 raft_consensus.cc:2243] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.785780 24862 raft_consensus.cc:2272] T ec9017ccb0d849c49a75c9b1dce1547e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.788729 24862 tablet_replica.cc:333] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.789608 24862 raft_consensus.cc:2243] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.790235 24862 raft_consensus.cc:2272] T 0d096e6da5114d91b7c4dd46412d2a6a P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.793581 24862 tablet_replica.cc:333] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.794407 24862 raft_consensus.cc:2243] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.795706 24862 raft_consensus.cc:2272] T 407574b4cda94da99be372d249d21fd5 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.798480 24862 tablet_replica.cc:333] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.799345 24862 raft_consensus.cc:2243] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.800017 24862 raft_consensus.cc:2272] T d3b95127d51d4e5db31f5dbda1a935c0 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.802970 24862 tablet_replica.cc:333] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.803793 24862 raft_consensus.cc:2243] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.804507 24862 raft_consensus.cc:2272] T c47092de97434584b0810190b58e7623 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.806648 26756 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.807380 24862 tablet_replica.cc:333] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.808314 24862 raft_consensus.cc:2243] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.809305 24862 pending_rounds.cc:70] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Trying to abort 1 pending ops.
I20260709 12:28:47.809610 24862 pending_rounds.cc:77] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c: Aborting op as it isn't in flight: id { term: 33 index: 23 } timestamp: 7305626121804517376 op_type: NO_OP noop_request { }
I20260709 12:28:47.810093 24862 raft_consensus.cc:2889] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 33 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:47.810470 24862 raft_consensus.cc:2272] T 4db91b54f05348e6855f89c508d0be01 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.813709 24862 tablet_replica.cc:333] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.814729 24862 raft_consensus.cc:2243] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.820035 24862 raft_consensus.cc:2272] T 323fa2c24ef84fd39b5fe37060615814 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.822841 24862 tablet_replica.cc:333] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
W20260709 12:28:47.823354 26756 consensus_peers.cc:597] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.826105 24862 raft_consensus.cc:2243] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.827322 24862 raft_consensus.cc:2272] T 1e8b57c1b80240d483c7ca4e95d74673 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.830247 24862 tablet_replica.cc:333] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.831132 24862 raft_consensus.cc:2243] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.834477 24862 raft_consensus.cc:2272] T 0f85110b9e6d45578bf2f39e2e9ad47c P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.838182 24862 tablet_replica.cc:333] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.839437 24862 raft_consensus.cc:2243] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.840978 24862 raft_consensus.cc:2272] T adcc76aa423a40eb95d31d98a8c8776f P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.844424 24862 tablet_replica.cc:333] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.845431 24862 raft_consensus.cc:2243] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.846103 24862 raft_consensus.cc:2272] T 955755ce63f743b9a01d5871aff94f6c P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.849327 24862 tablet_replica.cc:333] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.850244 24862 raft_consensus.cc:2243] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.851063 24862 raft_consensus.cc:2272] T eac7044e09a64594b8fcd962f1d387b7 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.854622 24862 tablet_replica.cc:333] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.855829 24862 raft_consensus.cc:2243] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.857240 24862 raft_consensus.cc:2272] T 93ec87f552a04ce49312a2ef65f862ce P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.860110 24862 tablet_replica.cc:333] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.860996 24862 raft_consensus.cc:2243] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.861761 24862 raft_consensus.cc:2272] T ef606226bb434b95911632eec3ede435 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.864884 24862 tablet_replica.cc:333] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.865911 24862 raft_consensus.cc:2243] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.866699 24862 raft_consensus.cc:2272] T 0cbe5295e8db495985835a50d39e4438 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.869926 24862 tablet_replica.cc:333] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.870733 24862 raft_consensus.cc:2243] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.871942 24862 raft_consensus.cc:2272] T f5a510044c5b459abfd55896cc2d21eb P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.874926 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.874919 24862 tablet_replica.cc:333] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.877022 24862 raft_consensus.cc:2243] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.878307 24862 raft_consensus.cc:2272] T 549b49961cbb430d9ece6acd81aede77 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.881150 24862 tablet_replica.cc:333] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.882054 24862 raft_consensus.cc:2243] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.882834 24862 raft_consensus.cc:2272] T 772287743e884fe8a6dc6c516c3ad901 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.884447 26757 consensus_peers.cc:597] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:47.885890 24862 tablet_replica.cc:333] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.886855 24862 raft_consensus.cc:2243] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.888093 24862 raft_consensus.cc:2272] T c0ff2e29d14b44128ecab4f62a77317f P a981c9a7972c4e99a0903924c64fbb5c [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.891201 24862 tablet_replica.cc:333] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.892043 24862 raft_consensus.cc:2243] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.894114 24862 raft_consensus.cc:2272] T f720b2e4986f496694c7578beef67cd4 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.897253 24862 tablet_replica.cc:333] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.898075 24862 raft_consensus.cc:2243] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.899334 24862 raft_consensus.cc:2272] T 86798289f90549ed8084d260f08cb204 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.902220 24862 tablet_replica.cc:333] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.903112 24862 raft_consensus.cc:2243] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.904271 24862 raft_consensus.cc:2272] T e0673bc676154f04907934c1d3a60bce P a981c9a7972c4e99a0903924c64fbb5c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.907034 24862 tablet_replica.cc:333] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.907837 24862 raft_consensus.cc:2243] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:28:47.908304 26168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.908821 24862 raft_consensus.cc:2272] T bec84548f93742a096e72c1bd887809a P a981c9a7972c4e99a0903924c64fbb5c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.911756 24862 tablet_replica.cc:333] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.912639 24862 raft_consensus.cc:2243] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.913802 24862 raft_consensus.cc:2272] T de16ce1800b64c3c94c918aa6e9cae3d P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.916908 24862 tablet_replica.cc:333] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.917801 24862 raft_consensus.cc:2243] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:47.916981 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.918650 24862 raft_consensus.cc:2272] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P a981c9a7972c4e99a0903924c64fbb5c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.921624 24862 tablet_replica.cc:333] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.922557 24862 raft_consensus.cc:2243] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.923187 24862 raft_consensus.cc:2272] T 40485ee0f2b44c1290c368cf529eade1 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.926263 24862 tablet_replica.cc:333] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.927055 24862 raft_consensus.cc:2243] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.928269 24862 raft_consensus.cc:2272] T 13d33ec3d2d94eeda5bc1c4bbd4a0e91 P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.931315 24862 tablet_replica.cc:333] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.932157 24862 raft_consensus.cc:2243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.933358 24862 raft_consensus.cc:2272] T 8bede0bf927e45a7b18a1bf2ad0bb594 P a981c9a7972c4e99a0903924c64fbb5c [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.935429 24862 tablet_replica.cc:333] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.935982 24862 raft_consensus.cc:2243] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.936618 24862 raft_consensus.cc:2272] T 94c3d95514eb45178f2e5aa92e868e36 P a981c9a7972c4e99a0903924c64fbb5c [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.938992 24862 tablet_replica.cc:333] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.939628 24862 raft_consensus.cc:2243] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.940194 24862 raft_consensus.cc:2272] T a34019ceec5a4e4e877bed7351d1d91d P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.942458 24862 tablet_replica.cc:333] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.943389 24862 raft_consensus.cc:2243] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.944017 24862 raft_consensus.cc:2272] T be17917578e1415f98071364583240c1 P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.946322 24862 tablet_replica.cc:333] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.947022 24862 raft_consensus.cc:2243] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.947825 24862 raft_consensus.cc:2272] T 7585ccc5a92245cfb1bba6ff29291731 P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.949944 24862 tablet_replica.cc:333] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.950441 24862 raft_consensus.cc:2243] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.951260 24862 raft_consensus.cc:2272] T 2ee8b7d54fee44329721d2da7c8f62ad P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.953269 24862 tablet_replica.cc:333] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.953766 24862 raft_consensus.cc:2243] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.954437 24862 raft_consensus.cc:2272] T c49cc98db5354047a91f3ec2f90daa28 P a981c9a7972c4e99a0903924c64fbb5c [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.957243 24862 tablet_replica.cc:333] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.958456 24862 raft_consensus.cc:2243] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.959018 24862 raft_consensus.cc:2272] T 8c8d039bdedc4e1db6d66b58d4b56c4f P a981c9a7972c4e99a0903924c64fbb5c [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.959227 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.961311 24862 tablet_replica.cc:333] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.962036 24862 raft_consensus.cc:2243] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.963156 24862 raft_consensus.cc:2272] T 55aa4b8b33f544088007e1987c55617e P a981c9a7972c4e99a0903924c64fbb5c [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.965943 26756 consensus_peers.cc:597] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.966952 24862 tablet_replica.cc:333] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.967569 24862 raft_consensus.cc:2243] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.968461 24862 raft_consensus.cc:2272] T e9b765338c61436481fe416f17637ab2 P a981c9a7972c4e99a0903924c64fbb5c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.970639 24862 tablet_replica.cc:333] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c: stopping tablet replica
I20260709 12:28:47.971148 24862 raft_consensus.cc:2243] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.971745 24862 raft_consensus.cc:2272] T b4a62528d9404c66b98e87e76ac0f21e P a981c9a7972c4e99a0903924c64fbb5c [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.989163 26757 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:47.993348 26756 consensus_peers.cc:597] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.034139 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.037474 26756 consensus_peers.cc:597] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.057905 26757 consensus_peers.cc:597] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.104719 26757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111) [suppressed 595 similar messages]
W20260709 12:28:48.107120 26757 consensus_peers.cc:597] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.118266 26757 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.121425 26756 consensus_peers.cc:597] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:48.127974 26756 consensus_peers.cc:597] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Couldn't send request to peer a981c9a7972c4e99a0903924c64fbb5c. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.138259 26757 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.138842 26757 consensus_peers.cc:597] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.156461 26757 consensus_peers.cc:597] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.186115 26757 consensus_peers.cc:597] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.197317 26756 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:28:48.218087 26757 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.230419 26757 consensus_peers.cc:597] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.270877 26757 consensus_peers.cc:597] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.280879 26757 consensus_peers.cc:597] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.288003 26757 consensus_peers.cc:597] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.289446 26756 consensus_peers.cc:597] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:28:48.349611 26757 consensus_peers.cc:597] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.389891 26756 consensus_peers.cc:597] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:28:48.440474 26757 consensus_peers.cc:597] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Couldn't send request to peer 9e23389325a6411c9b0634f4f6af54c3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.473944 26756 consensus_peers.cc:597] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:48.478425  4674 raft_consensus.cc:493] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:48.478960  4674 raft_consensus.cc:515] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:48.481159  4674 leader_election.cc:290] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:48.481249 26756 consensus_peers.cc:597] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:28:48.487564 26757 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:48.493249 26756 leader_election.cc:336] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:48.493701 26756 leader_election.cc:304] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:48.494396  4674 raft_consensus.cc:2749] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:48.831724  4679 raft_consensus.cc:493] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:48.832340  4679 raft_consensus.cc:515] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:48.834493  4679 leader_election.cc:290] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:48.839282 26757 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:48.844487 26756 leader_election.cc:336] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:48.844970 26756 leader_election.cc:304] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:48.846215  4679 raft_consensus.cc:2749] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:48.942271  4679 raft_consensus.cc:493] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:48.942786  4679 raft_consensus.cc:515] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:48.944682  4679 leader_election.cc:290] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:48.949678 26756 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:48.951360 26756 leader_election.cc:336] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:48.951727 26756 leader_election.cc:304] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:48.952345  4679 raft_consensus.cc:2749] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:48.958662  4679 raft_consensus.cc:493] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:48.959196  4679 raft_consensus.cc:515] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } attrs { replace: true } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:48.962512  4679 leader_election.cc:290] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:48.967901 26757 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:48.971290 26756 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:48.971733 26756 leader_election.cc:304] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:48.972849  4679 raft_consensus.cc:2749] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:48.973759 26756 leader_election.cc:336] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:48.996595  4679 raft_consensus.cc:493] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:48.997728  4679 raft_consensus.cc:515] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:48.999884  4679 leader_election.cc:290] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.009855 26756 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:49.010798 26756 leader_election.cc:336] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:49.011303 26756 leader_election.cc:304] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.016044  4679 raft_consensus.cc:2749] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:49.036919  4679 raft_consensus.cc:493] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:49.037503  4679 raft_consensus.cc:515] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.041617  4679 leader_election.cc:290] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:49.044584 26757 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:49.045095 26756 leader_election.cc:336] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:49.045502 26756 leader_election.cc:304] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:49.046147  4679 raft_consensus.cc:2749] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:49.061941  4676 raft_consensus.cc:493] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:49.062537  4676 raft_consensus.cc:515] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
W20260709 12:28:49.063089 24862 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 26224)
W20260709 12:28:49.065971 26756 consensus_peers.cc:597] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:28:49.064751  4676 leader_election.cc:290] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
I20260709 12:28:49.068696  4676 raft_consensus.cc:493] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.069288  4676 raft_consensus.cc:515] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } attrs { replace: true } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.073345  4676 leader_election.cc:290] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.075855 26757 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:49.077777 26756 leader_election.cc:336] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:49.078194 26756 leader_election.cc:304] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:49.080062  4676 raft_consensus.cc:2749] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:49.079378 26757 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:49.083709 26756 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:49.084347 26756 leader_election.cc:304] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
W20260709 12:28:49.085270 26756 leader_election.cc:336] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:49.085741  4679 raft_consensus.cc:2749] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:49.096168  4679 raft_consensus.cc:493] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
W20260709 12:28:49.096700 26756 consensus_peers.cc:597] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:49.096911  4679 raft_consensus.cc:515] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.099082  4679 leader_election.cc:290] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.105417 26756 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:49.105988 26756 leader_election.cc:336] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:49.106344 26756 leader_election.cc:304] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.107003  4679 raft_consensus.cc:2749] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:49.168877  4679 raft_consensus.cc:493] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.169469  4679 raft_consensus.cc:515] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.171729  4679 leader_election.cc:290] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.177030 26756 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:49.177893 26756 leader_election.cc:336] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:49.178282 26756 leader_election.cc:304] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.179003  4679 raft_consensus.cc:2749] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:49.198467  4676 raft_consensus.cc:493] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.199035  4676 raft_consensus.cc:515] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.202365  4676 leader_election.cc:290] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.210116 26756 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
W20260709 12:28:49.210865 26756 leader_election.cc:336] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:49.211323 26756 leader_election.cc:304] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.212204  4679 raft_consensus.cc:2749] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:49.227797  4679 raft_consensus.cc:493] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.228425  4679 raft_consensus.cc:515] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.230907  4679 leader_election.cc:290] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.236639 26756 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
W20260709 12:28:49.238039 26757 leader_election.cc:336] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
I20260709 12:28:49.239167 26757 leader_election.cc:304] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.240577  4679 raft_consensus.cc:2749] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:49.269939  4676 raft_consensus.cc:493] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.270506  4676 raft_consensus.cc:515] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.272750  4676 leader_election.cc:290] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.278663 26757 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:49.281248 26756 leader_election.cc:336] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:49.281807 26756 leader_election.cc:304] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 9e23389325a6411c9b0634f4f6af54c3, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.282534  4676 raft_consensus.cc:2749] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:49.335850  4678 raft_consensus.cc:493] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.337433  4678 raft_consensus.cc:515] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.342604  4678 leader_election.cc:290] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
I20260709 12:28:49.347548  4678 raft_consensus.cc:493] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a981c9a7972c4e99a0903924c64fbb5c)
I20260709 12:28:49.348487  4678 raft_consensus.cc:515] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "a981c9a7972c4e99a0903924c64fbb5c" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38261 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.351478  4678 leader_election.cc:290] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261)
W20260709 12:28:49.358085 26756 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
W20260709 12:28:49.359445 26756 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
W20260709 12:28:49.359942 26756 leader_election.cc:336] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:49.361631 26756 leader_election.cc:304] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.362429  4678 raft_consensus.cc:2749] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:49.365597 26756 leader_election.cc:336] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:28:49.366137 26756 leader_election.cc:304] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, a981c9a7972c4e99a0903924c64fbb5c
I20260709 12:28:49.367082  4682 raft_consensus.cc:2749] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:49.415231 26756 consensus_peers.cc:597] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 -> Peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Couldn't send request to peer 8fd5643e7b494fa79c3d0aaf1e6e8152. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:49.426838  4682 raft_consensus.cc:493] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9e23389325a6411c9b0634f4f6af54c3)
I20260709 12:28:49.427769  4682 raft_consensus.cc:515] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "8fd5643e7b494fa79c3d0aaf1e6e8152" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44989 } } peers { permanent_uuid: "9e23389325a6411c9b0634f4f6af54c3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 40611 } } peers { permanent_uuid: "24e61c63097e4d3dbb149050e6fcc997" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 39865 } attrs { promote: false } }
I20260709 12:28:49.429877  4682 leader_election.cc:290] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989), 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611)
W20260709 12:28:49.434762 26757 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9e23389325a6411c9b0634f4f6af54c3 (127.24.71.130:40611): Network error: Client connection negotiation failed: client connection to 127.24.71.130:40611: connect: Connection refused (error 111)
W20260709 12:28:49.435743 26756 leader_election.cc:336] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8fd5643e7b494fa79c3d0aaf1e6e8152 (127.24.71.129:44989): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44989: connect: Connection refused (error 111)
I20260709 12:28:49.436138 26756 leader_election.cc:304] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24e61c63097e4d3dbb149050e6fcc997; no voters: 8fd5643e7b494fa79c3d0aaf1e6e8152, 9e23389325a6411c9b0634f4f6af54c3
I20260709 12:28:49.436823  4682 raft_consensus.cc:2749] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:49.561605 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:28:49.756455 24862 tablet_server.cc:179] TabletServer@127.24.71.132:0 shutting down...
I20260709 12:28:49.832847 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:49.833875 24862 tablet_replica.cc:333] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.835433 24862 raft_consensus.cc:2243] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.838311 24862 raft_consensus.cc:2272] T 8ddbca55e34c47f2b2d10e46e4779bd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.841252 24862 tablet_replica.cc:333] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.842123 24862 raft_consensus.cc:2243] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.843056 24862 raft_consensus.cc:2272] T 47b65a35d65c481985263a8a2546d39d P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.847483 24862 tablet_replica.cc:333] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.850546 24862 raft_consensus.cc:2243] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.851919 24862 raft_consensus.cc:2272] T 69c5f2ff7a584f80b9a347f6adb3c2f8 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.855351 24862 tablet_replica.cc:333] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.856063 24862 raft_consensus.cc:2243] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.857352 24862 raft_consensus.cc:2272] T 9e3998a28d0e4c4da61419183dc7ffb4 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.859949 24862 tablet_replica.cc:333] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.860805 24862 raft_consensus.cc:2243] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.862270 24862 raft_consensus.cc:2272] T ddc5e875aa744c50b603a5072bc7305b P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.865069 24862 tablet_replica.cc:333] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.865890 24862 raft_consensus.cc:2243] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.867311 24862 raft_consensus.cc:2272] T be17917578e1415f98071364583240c1 P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.869948 24862 tablet_replica.cc:333] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.870550 24862 raft_consensus.cc:2243] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.871881 24862 raft_consensus.cc:2272] T 891138e164dc40a7af7618323d52397f P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.874835 24862 tablet_replica.cc:333] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.875777 24862 raft_consensus.cc:2243] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.876683 24862 raft_consensus.cc:2272] T e9b765338c61436481fe416f17637ab2 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.878942 24862 tablet_replica.cc:333] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.879599 24862 raft_consensus.cc:2243] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.880887 24862 raft_consensus.cc:2272] T adcc76aa423a40eb95d31d98a8c8776f P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.883337 24862 tablet_replica.cc:333] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.883996 24862 raft_consensus.cc:2243] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.884961 24862 raft_consensus.cc:2272] T f720b2e4986f496694c7578beef67cd4 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.887658 24862 tablet_replica.cc:333] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.888624 24862 raft_consensus.cc:2243] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.889946 24862 raft_consensus.cc:2272] T 86798289f90549ed8084d260f08cb204 P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.893366 24862 tablet_replica.cc:333] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.894261 24862 raft_consensus.cc:2243] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.896087 24862 raft_consensus.cc:2272] T d0d1927cc7714e91945918d6aa1216e5 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.899031 24862 tablet_replica.cc:333] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.899760 24862 raft_consensus.cc:2243] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.900681 24862 raft_consensus.cc:2272] T bec84548f93742a096e72c1bd887809a P 24e61c63097e4d3dbb149050e6fcc997 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.903404 24862 tablet_replica.cc:333] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.904089 24862 raft_consensus.cc:2243] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.905767 24862 raft_consensus.cc:2272] T 549b49961cbb430d9ece6acd81aede77 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.908390 24862 tablet_replica.cc:333] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.909158 24862 raft_consensus.cc:2243] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.910672 24862 raft_consensus.cc:2272] T 0cbe5295e8db495985835a50d39e4438 P 24e61c63097e4d3dbb149050e6fcc997 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.913928 24862 tablet_replica.cc:333] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.914791 24862 raft_consensus.cc:2243] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.916181 24862 raft_consensus.cc:2272] T f5a510044c5b459abfd55896cc2d21eb P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.919481 24862 tablet_replica.cc:333] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.920217 24862 raft_consensus.cc:2243] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.921020 24862 raft_consensus.cc:2272] T 55aa4b8b33f544088007e1987c55617e P 24e61c63097e4d3dbb149050e6fcc997 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.923789 24862 tablet_replica.cc:333] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.924479 24862 raft_consensus.cc:2243] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.925660 24862 raft_consensus.cc:2272] T c7ad94af29be4656bf3ce8a023d19f7d P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.928606 24862 tablet_replica.cc:333] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.929417 24862 raft_consensus.cc:2243] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.931097 24862 raft_consensus.cc:2272] T 772287743e884fe8a6dc6c516c3ad901 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.934644 24862 tablet_replica.cc:333] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.935395 24862 raft_consensus.cc:2243] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.936962 24862 raft_consensus.cc:2272] T b4a62528d9404c66b98e87e76ac0f21e P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.939739 24862 tablet_replica.cc:333] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.940418 24862 raft_consensus.cc:2243] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.941263 24862 raft_consensus.cc:2272] T 824b6757fe414215b1cf6aa5ba238004 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.944473 24862 tablet_replica.cc:333] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.945292 24862 raft_consensus.cc:2243] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.946095 24862 raft_consensus.cc:2272] T 8bede0bf927e45a7b18a1bf2ad0bb594 P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.948419 24862 tablet_replica.cc:333] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.948967 24862 raft_consensus.cc:2243] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.949796 24862 raft_consensus.cc:2272] T 2ee8b7d54fee44329721d2da7c8f62ad P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.952137 24862 tablet_replica.cc:333] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.952766 24862 raft_consensus.cc:2243] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.953959 24862 raft_consensus.cc:2272] T 94c3d95514eb45178f2e5aa92e868e36 P 24e61c63097e4d3dbb149050e6fcc997 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.956220 24862 tablet_replica.cc:333] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.956863 24862 raft_consensus.cc:2243] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.957988 24862 raft_consensus.cc:2272] T a34019ceec5a4e4e877bed7351d1d91d P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.960352 24862 tablet_replica.cc:333] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.960938 24862 raft_consensus.cc:2243] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.962095 24862 raft_consensus.cc:2272] T 49f4a1a801de461182f8315a884b9889 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.964499 24862 tablet_replica.cc:333] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.965081 24862 raft_consensus.cc:2243] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.966311 24862 raft_consensus.cc:2272] T 46aa595bcb23423fb51ff268d34a3bc7 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.968549 24862 tablet_replica.cc:333] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.969118 24862 raft_consensus.cc:2243] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.969764 24862 raft_consensus.cc:2272] T dc3c86db3d4e4643a77b0fe0b0e687e1 P 24e61c63097e4d3dbb149050e6fcc997 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.972132 24862 tablet_replica.cc:333] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.972828 24862 raft_consensus.cc:2243] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.974030 24862 raft_consensus.cc:2272] T 3fe81c3162f24fc0b8716b2b42f0676f P 24e61c63097e4d3dbb149050e6fcc997 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.976485 24862 tablet_replica.cc:333] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.977108 24862 raft_consensus.cc:2243] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.978135 24862 raft_consensus.cc:2272] T 955755ce63f743b9a01d5871aff94f6c P 24e61c63097e4d3dbb149050e6fcc997 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.980473 24862 tablet_replica.cc:333] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.981256 24862 raft_consensus.cc:2243] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.982764 24862 raft_consensus.cc:2272] T eac7044e09a64594b8fcd962f1d387b7 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.985646 24862 tablet_replica.cc:333] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.986294 24862 raft_consensus.cc:2243] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.987793 24862 raft_consensus.cc:2272] T 86fd5767abab4e45ad1c8bd5f6d7a9c8 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.990310 24862 tablet_replica.cc:333] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.990912 24862 raft_consensus.cc:2243] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.992712 24862 raft_consensus.cc:2272] T 93ec87f552a04ce49312a2ef65f862ce P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.995038 24862 tablet_replica.cc:333] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.995600 24862 raft_consensus.cc:2243] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.996994 24862 raft_consensus.cc:2272] T 2fee7aee965145c6b43db63c2b3ecca8 P 24e61c63097e4d3dbb149050e6fcc997 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.999135 24862 tablet_replica.cc:333] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:49.999632 24862 raft_consensus.cc:2243] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:50.000854 24862 raft_consensus.cc:2272] T 0d096e6da5114d91b7c4dd46412d2a6a P 24e61c63097e4d3dbb149050e6fcc997 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:50.003170 24862 tablet_replica.cc:333] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:50.003754 24862 raft_consensus.cc:2243] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:50.004372 24862 raft_consensus.cc:2272] T 4db91b54f05348e6855f89c508d0be01 P 24e61c63097e4d3dbb149050e6fcc997 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:50.006721 24862 tablet_replica.cc:333] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997: stopping tablet replica
I20260709 12:28:50.007272 24862 raft_consensus.cc:2243] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:50.008911 24862 raft_consensus.cc:2272] T 526fbe6496c445dc95df29efcbeb8cd0 P 24e61c63097e4d3dbb149050e6fcc997 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:50.695484 24862 tablet_server.cc:196] TabletServer@127.24.71.132:0 shutdown complete.
I20260709 12:28:50.777261 24862 master.cc:562] Master@127.24.71.190:42395 shutting down...
W20260709 12:28:50.821898 26000 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:28:50.822340 26000 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:28:50.822567 26000 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:29:10.310938 25938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111) [suppressed 190 similar messages]
W20260709 12:29:10.317413 25938 catalog_manager.cc:4755] TS a981c9a7972c4e99a0903924c64fbb5c (127.24.71.131:38261): ChangeConfig:REMOVE_PEER RPC failed for tablet f720b2e4986f496694c7578beef67cd4: Network error: Client connection negotiation failed: client connection to 127.24.71.131:38261: connect: Connection refused (error 111)
I20260709 12:29:12.076994 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:12.078496 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:12.079020 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5eede3841e1242afbeab30840148b558: stopping tablet replica
I20260709 12:29:12.313311 24862 master.cc:584] Master@127.24.71.190:42395 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (358721 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:29:12.444003 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:43079
I20260709 12:29:12.445232 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:12.451398  4688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:12.451856  4689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:12.454005  4691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:12.455204 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:12.459120 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:12.459367 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:12.459583 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600152459566 us; error 0 us; skew 500 ppm
I20260709 12:29:12.460302 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:12.463418 24862 webserver.cc:533] Webserver started at http://127.24.71.190:41777/ using document root <none> and password file <none>
I20260709 12:29:12.463967 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:12.464185 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:12.464551 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:12.465906 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "a45175607d2c4b6c8f4c87e11ac28642"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.471202 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:29:12.475633  4696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.476717 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:12.477109 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "a45175607d2c4b6c8f4c87e11ac28642"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.477509 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:12.525005 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:12.526259 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:12.571725 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:43079
I20260709 12:29:12.571846  4747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:43079 every 8 connection(s)
I20260709 12:29:12.576375  4748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:12.589237  4748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642: Bootstrap starting.
I20260709 12:29:12.594239  4748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:12.598791  4748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642: No bootstrap required, opened a new log
I20260709 12:29:12.601039  4748 raft_consensus.cc:359] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45175607d2c4b6c8f4c87e11ac28642" member_type: VOTER }
I20260709 12:29:12.601491  4748 raft_consensus.cc:385] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:12.601673  4748 raft_consensus.cc:740] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45175607d2c4b6c8f4c87e11ac28642, State: Initialized, Role: FOLLOWER
I20260709 12:29:12.602173  4748 consensus_queue.cc:260] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45175607d2c4b6c8f4c87e11ac28642" member_type: VOTER }
I20260709 12:29:12.602623  4748 raft_consensus.cc:399] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:12.602829  4748 raft_consensus.cc:493] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:12.603164  4748 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:12.607972  4748 raft_consensus.cc:515] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45175607d2c4b6c8f4c87e11ac28642" member_type: VOTER }
I20260709 12:29:12.608484  4748 leader_election.cc:304] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a45175607d2c4b6c8f4c87e11ac28642; no voters: 
I20260709 12:29:12.609589  4748 leader_election.cc:290] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:12.609990  4751 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:12.611626  4751 raft_consensus.cc:697] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 1 LEADER]: Becoming Leader. State: Replica: a45175607d2c4b6c8f4c87e11ac28642, State: Running, Role: LEADER
I20260709 12:29:12.612447  4751 consensus_queue.cc:237] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45175607d2c4b6c8f4c87e11ac28642" member_type: VOTER }
I20260709 12:29:12.612751  4748 sys_catalog.cc:565] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:12.620512  4753 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a45175607d2c4b6c8f4c87e11ac28642. Latest consensus state: current_term: 1 leader_uuid: "a45175607d2c4b6c8f4c87e11ac28642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45175607d2c4b6c8f4c87e11ac28642" member_type: VOTER } }
I20260709 12:29:12.620738  4752 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a45175607d2c4b6c8f4c87e11ac28642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a45175607d2c4b6c8f4c87e11ac28642" member_type: VOTER } }
I20260709 12:29:12.621517  4753 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:12.621666  4752 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:12.623431  4761 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:12.629998  4761 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:12.631122 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:12.639052  4761 catalog_manager.cc:1383] Generated new cluster ID: 5ab29bc5172749108cbe880ec51083e9
I20260709 12:29:12.639367  4761 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:12.677358  4761 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:12.678812  4761 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:12.692452  4761 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642: Generated new TSK 0
I20260709 12:29:12.693207  4761 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:12.698493 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:12.705335  4769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:12.706455  4770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:12.707952  4772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:12.709834 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:12.713217 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:12.713436 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:12.713657 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600152713623 us; error 0 us; skew 500 ppm
I20260709 12:29:12.714344 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:12.717036 24862 webserver.cc:533] Webserver started at http://127.24.71.129:34921/ using document root <none> and password file <none>
I20260709 12:29:12.717531 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:12.717742 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:12.718024 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:12.719259 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "0ec932ee892248cda1f4e99e4ba7887e"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.724422 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:29:12.728009  4777 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.729010 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:12.729331 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "0ec932ee892248cda1f4e99e4ba7887e"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.729631 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:12.744344 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:12.745672 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:12.747211 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:12.749708 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:12.749940 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.750191 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:12.750397 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.796555 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:44881
I20260709 12:29:12.796716  4839 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:44881 every 8 connection(s)
I20260709 12:29:12.801905 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:12.810357  4844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:12.812340  4845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:12.815105 24862 server_base.cc:1061] running on GCE node
W20260709 12:29:12.815586  4847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:12.819269 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:12.819631 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:12.819846 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600152819830 us; error 0 us; skew 500 ppm
I20260709 12:29:12.820823 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:12.823105  4840 heartbeater.cc:344] Connected to a master server at 127.24.71.190:43079
I20260709 12:29:12.823455  4840 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:12.824265 24862 webserver.cc:533] Webserver started at http://127.24.71.130:37969/ using document root <none> and password file <none>
I20260709 12:29:12.824565  4840 heartbeater.cc:507] Master 127.24.71.190:43079 requested a full tablet report, sending...
I20260709 12:29:12.824935 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:12.825224 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:12.825574 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:12.827420  4713 ts_manager.cc:194] Registered new tserver with Master: 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:12.827525 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.830655  4713 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56002
I20260709 12:29:12.834926 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:12.839445  4852 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.840457 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:29:12.840796 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.841133 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:12.866658 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:12.868206 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:12.869926 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:12.872668 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:12.872924 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.873193 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:12.873390 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.920420 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:33187
I20260709 12:29:12.920544  4914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:33187 every 8 connection(s)
I20260709 12:29:12.926261 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:12.935004  4918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:12.939091  4919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:12.941618 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:12.942039  4915 heartbeater.cc:344] Connected to a master server at 127.24.71.190:43079
I20260709 12:29:12.942529  4915 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:12.942638  4922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:12.943696  4915 heartbeater.cc:507] Master 127.24.71.190:43079 requested a full tablet report, sending...
I20260709 12:29:12.943976 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:12.944324 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:12.944555 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600152944535 us; error 0 us; skew 500 ppm
I20260709 12:29:12.945391 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:12.946310  4713 ts_manager.cc:194] Registered new tserver with Master: 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:12.948330  4713 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56014
I20260709 12:29:12.948333 24862 webserver.cc:533] Webserver started at http://127.24.71.131:35151/ using document root <none> and password file <none>
I20260709 12:29:12.949097 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:12.949368 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:12.949720 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:12.951469 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "0fa1ec70394d466ab14173b48b67b8ed"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.956445 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:29:12.960059  4926 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.960985 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:12.961424 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "0fa1ec70394d466ab14173b48b67b8ed"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:12.961721 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:12.972049 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:12.973453 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:12.975262 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:12.977612 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:12.977819 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:12.978137 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:12.978291 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:13.024658 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:41655
I20260709 12:29:13.024760  4988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:41655 every 8 connection(s)
I20260709 12:29:13.030179 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:13.038544  4992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:13.040807  4993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:13.042022 24862 server_base.cc:1061] running on GCE node
W20260709 12:29:13.042649  4995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:13.043788 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:13.044035 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:13.044293 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600153044272 us; error 0 us; skew 500 ppm
I20260709 12:29:13.044852  4989 heartbeater.cc:344] Connected to a master server at 127.24.71.190:43079
I20260709 12:29:13.044977 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:13.045392  4989 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:13.046551  4989 heartbeater.cc:507] Master 127.24.71.190:43079 requested a full tablet report, sending...
I20260709 12:29:13.048236 24862 webserver.cc:533] Webserver started at http://127.24.71.132:43949/ using document root <none> and password file <none>
I20260709 12:29:13.048915 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:13.049117 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:13.049154  4713 ts_manager.cc:194] Registered new tserver with Master: 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:13.049512 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:13.050762 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-3-root/instance:
uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
format_stamp: "Formatted at 2026-07-09 12:29:13 on dist-test-slave-3zgk"
I20260709 12:29:13.050901  4713 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56022
I20260709 12:29:13.055737 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:29:13.060164  5000 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:13.061232 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:13.061595 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-3-root
uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
format_stamp: "Formatted at 2026-07-09 12:29:13 on dist-test-slave-3zgk"
I20260709 12:29:13.061896 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:13.086282 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:13.087749 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:13.089424 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:13.091832 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:13.092092 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:13.092360 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:13.092551 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:13.139935 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.132:42487
I20260709 12:29:13.140015  5062 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.132:42487 every 8 connection(s)
I20260709 12:29:13.155089  5063 heartbeater.cc:344] Connected to a master server at 127.24.71.190:43079
I20260709 12:29:13.155555  5063 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:13.156611  5063 heartbeater.cc:507] Master 127.24.71.190:43079 requested a full tablet report, sending...
I20260709 12:29:13.158674  4713 ts_manager.cc:194] Registered new tserver with Master: 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:13.159235 24862 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015411975s
I20260709 12:29:13.160221  4713 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56032
I20260709 12:29:13.189932  4713 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56042:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:29:13.446842  5028 tablet_service.cc:1511] Processing CreateTablet for tablet 07fab4a18b1e4d2faa9f58d62f7ecd89 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:29:13.448379  5028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fab4a18b1e4d2faa9f58d62f7ecd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.457914  5027 tablet_service.cc:1511] Processing CreateTablet for tablet e99cb592cbcf46d3bbf20936ff9ec873 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:29:13.458796  4880 tablet_service.cc:1511] Processing CreateTablet for tablet 7abd4abb7c334f02ba9c4672289410f4 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:29:13.459589  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e99cb592cbcf46d3bbf20936ff9ec873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.462291  4876 tablet_service.cc:1511] Processing CreateTablet for tablet a0d5e87b68e14a5580328547bf386316 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:29:13.462625  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 30e68af9cb68410b9539fea5e2191450 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:29:13.463493  4879 tablet_service.cc:1511] Processing CreateTablet for tablet e99cb592cbcf46d3bbf20936ff9ec873 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:29:13.462319  5026 tablet_service.cc:1511] Processing CreateTablet for tablet 999b4d2234f74d80ad85ddf37ec99b01 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:29:13.462620  4878 tablet_service.cc:1511] Processing CreateTablet for tablet 999b4d2234f74d80ad85ddf37ec99b01 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:29:13.465173  4880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd4abb7c334f02ba9c4672289410f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.465411  5026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 999b4d2234f74d80ad85ddf37ec99b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.465917  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30e68af9cb68410b9539fea5e2191450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.468667  4879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e99cb592cbcf46d3bbf20936ff9ec873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.469384  4876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0d5e87b68e14a5580328547bf386316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.470184  4878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 999b4d2234f74d80ad85ddf37ec99b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.491927  5025 tablet_service.cc:1511] Processing CreateTablet for tablet a0d5e87b68e14a5580328547bf386316 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:29:13.494036  5025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0d5e87b68e14a5580328547bf386316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.495075  5082 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:13.501585  4875 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc20ff0e7f54f64a936b32f513913fc (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:29:13.503158  5082 tablet_bootstrap.cc:654] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:13.503211  4875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc20ff0e7f54f64a936b32f513913fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.505761  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce6101a3bd546fe9ae5ba5767af6277 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:29:13.507246  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce6101a3bd546fe9ae5ba5767af6277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.530926  5026 tablet_service.cc:1511] Processing CreateTablet for tablet 72db5336d073433fa404fdf28bdfa919 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:29:13.531311  4954 tablet_service.cc:1511] Processing CreateTablet for tablet 07fab4a18b1e4d2faa9f58d62f7ecd89 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:29:13.532503  5026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72db5336d073433fa404fdf28bdfa919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.532593  4954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fab4a18b1e4d2faa9f58d62f7ecd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.533118  5082 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:13.533711  5082 ts_tablet_manager.cc:1403] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.009s
I20260709 12:29:13.535498  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 351e55d393df4e159f251e3253097ed7 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:29:13.536845  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351e55d393df4e159f251e3253097ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.537259  4875 tablet_service.cc:1511] Processing CreateTablet for tablet cc4a1cd832b5450f8efa34e455eeead0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:29:13.538568  4875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4a1cd832b5450f8efa34e455eeead0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.541827  4876 tablet_service.cc:1511] Processing CreateTablet for tablet b790753ff8784fb4821657345fa7353e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:29:13.542048  4953 tablet_service.cc:1511] Processing CreateTablet for tablet 7abd4abb7c334f02ba9c4672289410f4 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:29:13.542450  4874 tablet_service.cc:1511] Processing CreateTablet for tablet c079e12498924bdaa8ba38c6c2c4dd6c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:29:13.543354  4953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd4abb7c334f02ba9c4672289410f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.543345  4876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b790753ff8784fb4821657345fa7353e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.546635  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c079e12498924bdaa8ba38c6c2c4dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.548069  4952 tablet_service.cc:1511] Processing CreateTablet for tablet 999b4d2234f74d80ad85ddf37ec99b01 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:29:13.549527  4952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 999b4d2234f74d80ad85ddf37ec99b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.553588  4951 tablet_service.cc:1511] Processing CreateTablet for tablet 30e68af9cb68410b9539fea5e2191450 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:29:13.555866  4951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30e68af9cb68410b9539fea5e2191450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.556140  5083 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:13.557793  4873 tablet_service.cc:1511] Processing CreateTablet for tablet 54153631246c4cc68f62a6d2724ab306 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:29:13.543975  5082 raft_consensus.cc:359] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:13.559011  5082 raft_consensus.cc:385] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:13.561425  4950 tablet_service.cc:1511] Processing CreateTablet for tablet 72db5336d073433fa404fdf28bdfa919 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:29:13.563681  5082 raft_consensus.cc:740] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:13.560105  4873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54153631246c4cc68f62a6d2724ab306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.564812  4950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72db5336d073433fa404fdf28bdfa919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.564743  5082 consensus_queue.cc:260] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:13.567924  4872 tablet_service.cc:1511] Processing CreateTablet for tablet 5f743d73272948abbd7fc320e050742c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:29:13.569530  4872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f743d73272948abbd7fc320e050742c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.570922  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc20ff0e7f54f64a936b32f513913fc (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:29:13.571049  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc20ff0e7f54f64a936b32f513913fc (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:29:13.572559  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc20ff0e7f54f64a936b32f513913fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.572564  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc20ff0e7f54f64a936b32f513913fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.578117  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 31a7ec1dbc9448afaffb4f4291fbabed (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:29:13.579672  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a7ec1dbc9448afaffb4f4291fbabed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.581516  4948 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce6101a3bd546fe9ae5ba5767af6277 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:29:13.583024  4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce6101a3bd546fe9ae5ba5767af6277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.594522  5024 tablet_service.cc:1511] Processing CreateTablet for tablet cc4a1cd832b5450f8efa34e455eeead0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:29:13.596093  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4a1cd832b5450f8efa34e455eeead0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.606655  4947 tablet_service.cc:1511] Processing CreateTablet for tablet 3ef746273b574779b4dcb59a0d239aa2 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:29:13.608182  4947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ef746273b574779b4dcb59a0d239aa2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:13.625248  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:13.634481  4876 tablet_service.cc:1511] Processing CreateTablet for tablet 64b05cbdc6f14118b098b8dd06d859c3 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:29:13.635221  5083 tablet_bootstrap.cc:654] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:13.635962  4876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b05cbdc6f14118b098b8dd06d859c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.640733  4874 tablet_service.cc:1511] Processing CreateTablet for tablet 0165ba3321bd42768963b30a87b715e5 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:29:13.666826  4915 heartbeater.cc:499] Master 127.24.71.190:43079 was elected leader, sending a full tablet report...
I20260709 12:29:13.667419  5025 tablet_service.cc:1511] Processing CreateTablet for tablet 3ef746273b574779b4dcb59a0d239aa2 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:29:13.668139  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0165ba3321bd42768963b30a87b715e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.669045  5025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ef746273b574779b4dcb59a0d239aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.682389  5086 tablet_bootstrap.cc:492] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:13.682668  5082 ts_tablet_manager.cc:1434] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.149s	user 0.016s	sys 0.016s
I20260709 12:29:13.683755  5082 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:13.689764  5028 tablet_service.cc:1511] Processing CreateTablet for tablet 57c5a10b9463451aaf68ce652335418b (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:29:13.690397  5086 tablet_bootstrap.cc:654] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:13.691154  5028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c5a10b9463451aaf68ce652335418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.691527  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 351e55d393df4e159f251e3253097ed7 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:29:13.693027  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351e55d393df4e159f251e3253097ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.698616  4947 tablet_service.cc:1511] Processing CreateTablet for tablet 54153631246c4cc68f62a6d2724ab306 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:29:13.699223  4953 tablet_service.cc:1511] Processing CreateTablet for tablet 57c5a10b9463451aaf68ce652335418b (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:29:13.700052  4947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54153631246c4cc68f62a6d2724ab306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.704448  4953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c5a10b9463451aaf68ce652335418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.705339  4950 tablet_service.cc:1511] Processing CreateTablet for tablet 351e55d393df4e159f251e3253097ed7 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:29:13.709403  4950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351e55d393df4e159f251e3253097ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.711076  5026 tablet_service.cc:1511] Processing CreateTablet for tablet c079e12498924bdaa8ba38c6c2c4dd6c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:29:13.723896  4954 tablet_service.cc:1511] Processing CreateTablet for tablet 07913583dcd7461b82e4125d7bf9d9f0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:29:13.725380  4954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07913583dcd7461b82e4125d7bf9d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.725493  5026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c079e12498924bdaa8ba38c6c2c4dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.729943  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 07fab4a18b1e4d2faa9f58d62f7ecd89 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:29:13.731055  4803 tablet_service.cc:1511] Processing CreateTablet for tablet e99cb592cbcf46d3bbf20936ff9ec873 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:29:13.721140  5023 tablet_service.cc:1511] Processing CreateTablet for tablet 54153631246c4cc68f62a6d2724ab306 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:29:13.733414  5023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54153631246c4cc68f62a6d2724ab306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.733198  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 7abd4abb7c334f02ba9c4672289410f4 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:29:13.734052  4879 tablet_service.cc:1511] Processing CreateTablet for tablet 95c3d8ad3cb14b11aef810d11bc59682 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:29:13.735908  4879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95c3d8ad3cb14b11aef810d11bc59682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.731489  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fab4a18b1e4d2faa9f58d62f7ecd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.736824  4803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e99cb592cbcf46d3bbf20936ff9ec873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.721608  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 31a7ec1dbc9448afaffb4f4291fbabed (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:29:13.737452  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd4abb7c334f02ba9c4672289410f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.738629  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a7ec1dbc9448afaffb4f4291fbabed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.742895  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 30e68af9cb68410b9539fea5e2191450 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:29:13.744493  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30e68af9cb68410b9539fea5e2191450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.746133  4801 tablet_service.cc:1511] Processing CreateTablet for tablet a0d5e87b68e14a5580328547bf386316 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:29:13.747599  4801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0d5e87b68e14a5580328547bf386316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.749624  4800 tablet_service.cc:1511] Processing CreateTablet for tablet 72db5336d073433fa404fdf28bdfa919 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:29:13.751080  4800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72db5336d073433fa404fdf28bdfa919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.753477  4799 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce6101a3bd546fe9ae5ba5767af6277 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:29:13.754927  4799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce6101a3bd546fe9ae5ba5767af6277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.718613  5083 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:13.760869  5083 ts_tablet_manager.cc:1403] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.205s	user 0.014s	sys 0.006s
I20260709 12:29:13.766196  4798 tablet_service.cc:1511] Processing CreateTablet for tablet cc4a1cd832b5450f8efa34e455eeead0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:29:13.767743  4798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4a1cd832b5450f8efa34e455eeead0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.778223  4797 tablet_service.cc:1511] Processing CreateTablet for tablet 3ef746273b574779b4dcb59a0d239aa2 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:29:13.779882  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ef746273b574779b4dcb59a0d239aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.790222  4796 tablet_service.cc:1511] Processing CreateTablet for tablet b790753ff8784fb4821657345fa7353e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:29:13.791767  4796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b790753ff8784fb4821657345fa7353e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.797850  5083 raft_consensus.cc:359] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:13.799124  5083 raft_consensus.cc:385] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:13.799599  5083 raft_consensus.cc:740] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:13.804672  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 5f743d73272948abbd7fc320e050742c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:29:13.806135  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f743d73272948abbd7fc320e050742c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.817310  4951 tablet_service.cc:1511] Processing CreateTablet for tablet b790753ff8784fb4821657345fa7353e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:29:13.818727  4951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b790753ff8784fb4821657345fa7353e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.823237  5021 tablet_service.cc:1511] Processing CreateTablet for tablet 07913583dcd7461b82e4125d7bf9d9f0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:29:13.824736  5021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07913583dcd7461b82e4125d7bf9d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.834663  4840 heartbeater.cc:499] Master 127.24.71.190:43079 was elected leader, sending a full tablet report...
I20260709 12:29:13.800990  5083 consensus_queue.cc:260] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:13.843293  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 7909fe384cf84b089c40d28a15fd8869 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:29:13.844679  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7909fe384cf84b089c40d28a15fd8869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.861459  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 1c15fed052da4a3392865a529df5eda0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:29:13.862980  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c15fed052da4a3392865a529df5eda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.881067  5082 tablet_bootstrap.cc:654] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:13.893110  5083 ts_tablet_manager.cc:1434] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.132s	user 0.016s	sys 0.022s
I20260709 12:29:13.892733  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 64b05cbdc6f14118b098b8dd06d859c3 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:29:13.893903  5063 heartbeater.cc:499] Master 127.24.71.190:43079 was elected leader, sending a full tablet report...
I20260709 12:29:13.894240  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b05cbdc6f14118b098b8dd06d859c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.898520  5083 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
W20260709 12:29:13.901784  5064 tablet.cc:2406] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:13.905591  5083 tablet_bootstrap.cc:654] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:13.911538  5089 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:13.916769  5086 tablet_bootstrap.cc:492] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:13.917351  5086 ts_tablet_manager.cc:1403] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.239s	user 0.073s	sys 0.101s
I20260709 12:29:13.912150  5089 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:13.928915  4948 tablet_service.cc:1511] Processing CreateTablet for tablet 0165ba3321bd42768963b30a87b715e5 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:29:13.941844  4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0165ba3321bd42768963b30a87b715e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.933405  4952 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc367942ca4feb9a1fd830092d860f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:29:13.929288  4798 tablet_service.cc:1511] Processing CreateTablet for tablet c079e12498924bdaa8ba38c6c2c4dd6c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:29:13.944772  4952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc367942ca4feb9a1fd830092d860f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.947263  4798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c079e12498924bdaa8ba38c6c2c4dd6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.936854  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 31a7ec1dbc9448afaffb4f4291fbabed (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:29:13.941687  5089 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:13.924027  5086 raft_consensus.cc:359] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:13.957523  5086 raft_consensus.cc:385] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:13.958279  5086 raft_consensus.cc:740] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:13.959766  4946 tablet_service.cc:1511] Processing CreateTablet for tablet 7909fe384cf84b089c40d28a15fd8869 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:29:13.961241  4946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7909fe384cf84b089c40d28a15fd8869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.960124  5086 consensus_queue.cc:260] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:13.930807  4801 tablet_service.cc:1511] Processing CreateTablet for tablet 5f743d73272948abbd7fc320e050742c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:29:13.954821  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a7ec1dbc9448afaffb4f4291fbabed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.966778  4801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f743d73272948abbd7fc320e050742c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.921010  4953 tablet_service.cc:1511] Processing CreateTablet for tablet 64b05cbdc6f14118b098b8dd06d859c3 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:29:13.969609  4953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b05cbdc6f14118b098b8dd06d859c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:13.925678  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 57c5a10b9463451aaf68ce652335418b (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:29:14.005870  5085 raft_consensus.cc:493] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.006692  4876 tablet_service.cc:1511] Processing CreateTablet for tablet 747d7984e025401681cfe15c121b8a64 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:29:14.006902  4799 tablet_service.cc:1511] Processing CreateTablet for tablet 95c3d8ad3cb14b11aef810d11bc59682 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:29:13.938982  4803 tablet_service.cc:1511] Processing CreateTablet for tablet 07913583dcd7461b82e4125d7bf9d9f0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:29:14.012451  5085 raft_consensus.cc:515] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.017928  4872 tablet_service.cc:1511] Processing CreateTablet for tablet 0799f16a2c4d4001826b14d3b2f5f42d (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:29:14.019717  4876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747d7984e025401681cfe15c121b8a64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.023546  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c5a10b9463451aaf68ce652335418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.026497  4797 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc367942ca4feb9a1fd830092d860f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:29:14.028997  4947 tablet_service.cc:1511] Processing CreateTablet for tablet 26d588e01b9943119c0c0427f2bc7b6f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:29:14.030858  4945 tablet_service.cc:1511] Processing CreateTablet for tablet 0799f16a2c4d4001826b14d3b2f5f42d (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:29:14.044996  5085 leader_election.cc:290] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:13.996587  4800 tablet_service.cc:1511] Processing CreateTablet for tablet 0165ba3321bd42768963b30a87b715e5 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:29:14.049500  4800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0165ba3321bd42768963b30a87b715e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.049906  4872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0799f16a2c4d4001826b14d3b2f5f42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.061501  5086 ts_tablet_manager.cc:1434] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.144s	user 0.011s	sys 0.013s
I20260709 12:29:14.053594  4989 heartbeater.cc:499] Master 127.24.71.190:43079 was elected leader, sending a full tablet report...
I20260709 12:29:14.063126  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 1c15fed052da4a3392865a529df5eda0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:29:14.067090  4954 tablet_service.cc:1511] Processing CreateTablet for tablet 1c15fed052da4a3392865a529df5eda0 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:29:14.063162  4796 tablet_service.cc:1511] Processing CreateTablet for tablet 747d7984e025401681cfe15c121b8a64 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:29:14.068917  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc367942ca4feb9a1fd830092d860f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.069636  5091 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:14.069833  4796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747d7984e025401681cfe15c121b8a64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.072523  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c15fed052da4a3392865a529df5eda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.076619  4803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07913583dcd7461b82e4125d7bf9d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.078266  5082 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:14.078853  5082 ts_tablet_manager.cc:1403] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.395s	user 0.015s	sys 0.015s
I20260709 12:29:14.080587  4799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95c3d8ad3cb14b11aef810d11bc59682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.092964  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0799f16a2c4d4001826b14d3b2f5f42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.093376  5086 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:14.110548  4801 tablet_service.cc:1511] Processing CreateTablet for tablet 26d588e01b9943119c0c0427f2bc7b6f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:29:14.132015  4947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d588e01b9943119c0c0427f2bc7b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.149529  4801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d588e01b9943119c0c0427f2bc7b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.147615  4954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c15fed052da4a3392865a529df5eda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.169250  4872 tablet_service.cc:1511] Processing CreateTablet for tablet 3340b333db824849a2af865d2658fd89 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:29:14.170671  4872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3340b333db824849a2af865d2658fd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.152112  5082 raft_consensus.cc:359] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.171311  5082 raft_consensus.cc:385] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.171663  5082 raft_consensus.cc:740] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.175658  5091 tablet_bootstrap.cc:654] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.180721  5082 consensus_queue.cc:260] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.189616  4876 tablet_service.cc:1511] Processing CreateTablet for tablet 03b26e75deab440d913747aece108666 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:29:14.190994  5082 ts_tablet_manager.cc:1434] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.112s	user 0.009s	sys 0.000s
I20260709 12:29:14.170266  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 3340b333db824849a2af865d2658fd89 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:29:14.192663  5082 tablet_bootstrap.cc:492] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:14.183791  5086 tablet_bootstrap.cc:654] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.193065  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3340b333db824849a2af865d2658fd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.193895  4951 tablet_service.cc:1511] Processing CreateTablet for tablet 3340b333db824849a2af865d2658fd89 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:29:14.191082  4876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03b26e75deab440d913747aece108666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.217344  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:14.219376  4855 leader_election.cc:343] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.227289  5026 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc367942ca4feb9a1fd830092d860f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:29:14.230595  5026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc367942ca4feb9a1fd830092d860f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.231312  5083 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:14.231891  5083 ts_tablet_manager.cc:1403] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.334s	user 0.012s	sys 0.038s
I20260709 12:29:14.235517  5083 raft_consensus.cc:359] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.236518  5083 raft_consensus.cc:385] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.236910  5083 raft_consensus.cc:740] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.237720  5083 consensus_queue.cc:260] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.240718  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:14.241686  4951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3340b333db824849a2af865d2658fd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.222011  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 95c3d8ad3cb14b11aef810d11bc59682 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
W20260709 12:29:14.248364  4855 leader_election.cc:343] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.249137  4855 leader_election.cc:304] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:14.249504  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95c3d8ad3cb14b11aef810d11bc59682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.250404  5085 raft_consensus.cc:2749] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.250990  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 5149b4ff21da4751975490772301b992 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:29:14.252417  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5149b4ff21da4751975490772301b992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.261843  5082 tablet_bootstrap.cc:654] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.262837  5083 ts_tablet_manager.cc:1434] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
I20260709 12:29:14.266423  5023 tablet_service.cc:1511] Processing CreateTablet for tablet 7909fe384cf84b089c40d28a15fd8869 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:29:14.267098  5028 tablet_service.cc:1511] Processing CreateTablet for tablet 747d7984e025401681cfe15c121b8a64 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:29:14.267856  5023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7909fe384cf84b089c40d28a15fd8869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.268937  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:14.269088  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:14.270416  5025 tablet_service.cc:1511] Processing CreateTablet for tablet 03b26e75deab440d913747aece108666 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:29:14.267463  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 0799f16a2c4d4001826b14d3b2f5f42d (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:29:14.272652  5028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747d7984e025401681cfe15c121b8a64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.273361  5025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03b26e75deab440d913747aece108666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.276607  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0799f16a2c4d4001826b14d3b2f5f42d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:14.277405  5003 leader_election.cc:343] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:14.278172  5003 leader_election.cc:343] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.278563  5003 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:14.269253  5021 tablet_service.cc:1511] Processing CreateTablet for tablet 26d588e01b9943119c0c0427f2bc7b6f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:29:14.290099  5021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d588e01b9943119c0c0427f2bc7b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.292644  5090 raft_consensus.cc:2749] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.304347  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 03b26e75deab440d913747aece108666 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:29:14.305859  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03b26e75deab440d913747aece108666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.311759  5082 tablet_bootstrap.cc:492] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:14.312503  5082 ts_tablet_manager.cc:1403] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.120s	user 0.020s	sys 0.014s
I20260709 12:29:14.314621  5083 tablet_bootstrap.cc:492] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:14.326400  5090 raft_consensus.cc:493] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.327081  5090 raft_consensus.cc:515] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.329088  5024 tablet_service.cc:1511] Processing CreateTablet for tablet 5149b4ff21da4751975490772301b992 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:29:14.329919  5090 leader_election.cc:290] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:14.330556  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5149b4ff21da4751975490772301b992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.332057  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:14.329612  5086 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
W20260709 12:29:14.338774  5003 leader_election.cc:343] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.338832  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 5149b4ff21da4751975490772301b992 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:29:14.341463  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5149b4ff21da4751975490772301b992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.338961  5083 tablet_bootstrap.cc:654] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.332938  5082 raft_consensus.cc:359] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.370678  5082 raft_consensus.cc:385] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.371059  5082 raft_consensus.cc:740] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.360500  4945 tablet_service.cc:1511] Processing CreateTablet for tablet 23b4122ef56040c887e2e7481f657288 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:29:14.372583  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b4122ef56040c887e2e7481f657288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.373509  5021 tablet_service.cc:1511] Processing CreateTablet for tablet 23b4122ef56040c887e2e7481f657288 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:29:14.376411  5086 ts_tablet_manager.cc:1403] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.283s	user 0.015s	sys 0.005s
I20260709 12:29:14.382189  5021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b4122ef56040c887e2e7481f657288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.384416  5082 consensus_queue.cc:260] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.387696  5082 ts_tablet_manager.cc:1434] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.000s
I20260709 12:29:14.396878  5082 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:14.401163  5091 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:14.401737  5091 ts_tablet_manager.cc:1403] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.332s	user 0.027s	sys 0.035s
I20260709 12:29:14.404978  5091 raft_consensus.cc:359] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.405725  5091 raft_consensus.cc:385] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.384521  5086 raft_consensus.cc:359] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.407801  5086 raft_consensus.cc:385] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.408160  5086 raft_consensus.cc:740] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.409081  5086 consensus_queue.cc:260] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.411712  5086 ts_tablet_manager.cc:1434] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:29:14.411455  4802 tablet_service.cc:1511] Processing CreateTablet for tablet b65cf7fce855449990ccd780e435d919 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:29:14.412878  5086 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:14.412917  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65cf7fce855449990ccd780e435d919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.417299  5082 tablet_bootstrap.cc:654] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.422185  4876 tablet_service.cc:1511] Processing CreateTablet for tablet 23b4122ef56040c887e2e7481f657288 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:29:14.423570  4876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b4122ef56040c887e2e7481f657288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.406323  5091 raft_consensus.cc:740] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.429056  4951 tablet_service.cc:1511] Processing CreateTablet for tablet b65cf7fce855449990ccd780e435d919 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:29:14.430480  4951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65cf7fce855449990ccd780e435d919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.433993  5091 consensus_queue.cc:260] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.435580  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 92e5720333ba432baadd844189c9c80b (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:29:14.437090  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e5720333ba432baadd844189c9c80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.437728  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 92e5720333ba432baadd844189c9c80b (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:29:14.439445  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e5720333ba432baadd844189c9c80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.444058  4872 tablet_service.cc:1511] Processing CreateTablet for tablet b65cf7fce855449990ccd780e435d919 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:29:14.444885  5094 raft_consensus.cc:493] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.446071  4872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65cf7fce855449990ccd780e435d919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.445809  5094 raft_consensus.cc:515] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.432713  5086 tablet_bootstrap.cc:654] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.464259  5094 leader_election.cc:290] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:14.479954  5091 ts_tablet_manager.cc:1434] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.011s
I20260709 12:29:14.480454  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 4831fc84d91e4eafbc37bf86cd29ad5e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:29:14.482043  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4831fc84d91e4eafbc37bf86cd29ad5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.500576  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 4831fc84d91e4eafbc37bf86cd29ad5e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:29:14.502002  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4831fc84d91e4eafbc37bf86cd29ad5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.502723  4954 tablet_service.cc:1511] Processing CreateTablet for tablet 79100fd1ff6942f5b408d20820466ef2 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:29:14.504108  4954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79100fd1ff6942f5b408d20820466ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.509729  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 92e5720333ba432baadd844189c9c80b (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:29:14.511251  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e5720333ba432baadd844189c9c80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.519388  5086 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:14.521243  5082 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:14.521962  5082 ts_tablet_manager.cc:1403] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.021s
I20260709 12:29:14.525424  5082 raft_consensus.cc:359] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.525926  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:14.527387  4879 tablet_service.cc:1511] Processing CreateTablet for tablet d35c974e4cb84cf49a6143cc50956e93 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:29:14.525728  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 79100fd1ff6942f5b408d20820466ef2 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:29:14.527086  4945 tablet_service.cc:1511] Processing CreateTablet for tablet d35c974e4cb84cf49a6143cc50956e93 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:29:14.528761  4879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35c974e4cb84cf49a6143cc50956e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.529388  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79100fd1ff6942f5b408d20820466ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.529507  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35c974e4cb84cf49a6143cc50956e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.533277  5024 tablet_service.cc:1511] Processing CreateTablet for tablet 4831fc84d91e4eafbc37bf86cd29ad5e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:29:14.534096  5082 raft_consensus.cc:385] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.535161  5082 raft_consensus.cc:740] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.534775  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4831fc84d91e4eafbc37bf86cd29ad5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.540838  5028 tablet_service.cc:1511] Processing CreateTablet for tablet 4f790808b2394d2fbe8a000bb3f62ea8 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:29:14.542173  5028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f790808b2394d2fbe8a000bb3f62ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.534730  5086 ts_tablet_manager.cc:1403] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.122s	user 0.007s	sys 0.011s
I20260709 12:29:14.536541  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 79100fd1ff6942f5b408d20820466ef2 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:29:14.548300  5091 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:14.549302  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79100fd1ff6942f5b408d20820466ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.558746  5082 consensus_queue.cc:260] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.559568  4797 tablet_service.cc:1511] Processing CreateTablet for tablet 4f790808b2394d2fbe8a000bb3f62ea8 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:29:14.561021  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f790808b2394d2fbe8a000bb3f62ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.563032  5083 tablet_bootstrap.cc:492] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:14.563591  5083 ts_tablet_manager.cc:1403] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.262s	user 0.016s	sys 0.015s
I20260709 12:29:14.565366  5082 ts_tablet_manager.cc:1434] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 12:29:14.565932  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:14.566761  5085 raft_consensus.cc:493] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.585719  5085 raft_consensus.cc:515] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.536541  4802 tablet_service.cc:1511] Processing CreateTablet for tablet d35c974e4cb84cf49a6143cc50956e93 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:29:14.590843  5085 leader_election.cc:290] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:14.571214  5001 leader_election.cc:343] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.591712  5001 leader_election.cc:304] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:14.592195  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0d5e87b68e14a5580328547bf386316" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:14.577550  5086 raft_consensus.cc:359] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.593122  5086 raft_consensus.cc:385] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.579746  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:14.583679  4954 tablet_service.cc:1511] Processing CreateTablet for tablet 0783f64f319148009af4133e89e944ee (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 12:29:14.593760  4855 leader_election.cc:343] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.593776  4890 raft_consensus.cc:2468] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:14.594719  4954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0783f64f319148009af4133e89e944ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.567387  5083 raft_consensus.cc:359] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.592670  5090 raft_consensus.cc:2749] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.578684  4874 tablet_service.cc:1511] Processing CreateTablet for tablet 4f790808b2394d2fbe8a000bb3f62ea8 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:29:14.596110  4927 leader_election.cc:304] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:14.590852  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35c974e4cb84cf49a6143cc50956e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.596961  5083 raft_consensus.cc:385] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.597229  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f790808b2394d2fbe8a000bb3f62ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.597352  5083 raft_consensus.cc:740] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
W20260709 12:29:14.597584  4929 leader_election.cc:343] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.599169  5094 raft_consensus.cc:2804] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:14.599627  5094 raft_consensus.cc:493] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.599978  5094 raft_consensus.cc:3060] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:14.602034  5082 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:14.609261  5082 tablet_bootstrap.cc:654] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.609875  5091 tablet_bootstrap.cc:654] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.610844  5083 consensus_queue.cc:260] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.616353  5090 raft_consensus.cc:493] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.616847  5090 raft_consensus.cc:515] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.621690  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 0783f64f319148009af4133e89e944ee (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:29:14.622061  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:14.623100  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0783f64f319148009af4133e89e944ee. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:14.623628  5003 leader_election.cc:343] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.623194  5024 tablet_service.cc:1511] Processing CreateTablet for tablet 0783f64f319148009af4133e89e944ee (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:29:14.624260  5090 leader_election.cc:290] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:14.626457  5083 ts_tablet_manager.cc:1434] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:29:14.626374  5086 raft_consensus.cc:740] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.627501  5086 consensus_queue.cc:260] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:14.627990  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:14.630594  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 52764aff47af4493a2f25ca5f8b3387c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:29:14.631989  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52764aff47af4493a2f25ca5f8b3387c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.632622  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:14.636863  5001 leader_election.cc:343] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.637331  5001 leader_election.cc:304] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:14.638619  5090 raft_consensus.cc:2749] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.640084  5083 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:14.642966  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0d5e87b68e14a5580328547bf386316" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:14.643857  4951 tablet_service.cc:1511] Processing CreateTablet for tablet 52764aff47af4493a2f25ca5f8b3387c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 12:29:14.644531  4855 leader_election.cc:343] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.645061  4855 leader_election.cc:304] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:14.645720  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0783f64f319148009af4133e89e944ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.646467  5085 raft_consensus.cc:2749] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.645275  4951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52764aff47af4493a2f25ca5f8b3387c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.654032  5094 raft_consensus.cc:515] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.654699  5091 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:14.658748  5083 tablet_bootstrap.cc:654] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.665819  5091 ts_tablet_manager.cc:1403] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.167s	user 0.010s	sys 0.003s
I20260709 12:29:14.665064  5086 ts_tablet_manager.cc:1434] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.115s	user 0.008s	sys 0.000s
I20260709 12:29:14.669488  5086 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:14.669391  5091 raft_consensus.cc:359] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.673784  5091 raft_consensus.cc:385] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.674167  5091 raft_consensus.cc:740] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.675305  5091 consensus_queue.cc:260] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.675604  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 638da9762ab2493e85b9205f6c2679f9 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:29:14.676999  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 638da9762ab2493e85b9205f6c2679f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.678371  5091 ts_tablet_manager.cc:1434] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:29:14.681524  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:14.682211  4890 raft_consensus.cc:3060] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:14.675604  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 52764aff47af4493a2f25ca5f8b3387c (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:29:14.696566  5091 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:14.691334  4890 raft_consensus.cc:2468] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:14.697834  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 638da9762ab2493e85b9205f6c2679f9 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:29:14.699481  5116 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.714208  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:14.723168  5116 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.723658  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 638da9762ab2493e85b9205f6c2679f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.712133  5108 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.725801  5116 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:14.726037  5108 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.726835  5024 tablet_service.cc:1511] Processing CreateTablet for tablet 845a033bb8104895847b349dcb5eeccb (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 12:29:14.727830  4929 leader_election.cc:343] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.709864  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52764aff47af4493a2f25ca5f8b3387c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.718397  5083 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:14.711674  5086 tablet_bootstrap.cc:654] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.715771  5082 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:14.732409  5082 ts_tablet_manager.cc:1403] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.131s	user 0.021s	sys 0.019s
I20260709 12:29:14.721500  4945 tablet_service.cc:1511] Processing CreateTablet for tablet 638da9762ab2493e85b9205f6c2679f9 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:29:14.714685  5094 leader_election.cc:290] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:14.741343  4927 leader_election.cc:304] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:14.742451  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 638da9762ab2493e85b9205f6c2679f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.732786  5083 ts_tablet_manager.cc:1403] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.008s
I20260709 12:29:14.745465  5094 raft_consensus.cc:2804] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:14.746047  5094 raft_consensus.cc:697] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:14.746953  5094 consensus_queue.cc:237] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:14.749439  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:14.750200  4815 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:14.753041  4929 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:14.753615  4874 tablet_service.cc:1511] Processing CreateTablet for tablet 845a033bb8104895847b349dcb5eeccb (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:29:14.754046  5116 raft_consensus.cc:2804] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:14.751252  5083 raft_consensus.cc:359] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.754498  5116 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.755069  5116 raft_consensus.cc:3060] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:14.735910  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:14.749440  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 845a033bb8104895847b349dcb5eeccb (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:29:14.755779  5038 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:14.736125  5082 raft_consensus.cc:359] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.757349  5082 raft_consensus.cc:385] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.757692  5082 raft_consensus.cc:740] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.732729  5108 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:14.755097  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845a033bb8104895847b349dcb5eeccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.758482  5082 consensus_queue.cc:260] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.761176  5082 ts_tablet_manager.cc:1434] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.004s
I20260709 12:29:14.761894  5091 tablet_bootstrap.cc:654] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.762655  5082 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:14.740760  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845a033bb8104895847b349dcb5eeccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.751432  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 29feba7c56da402abe958d088341ea39 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:29:14.767932  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 29feba7c56da402abe958d088341ea39 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:29:14.767935  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29feba7c56da402abe958d088341ea39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.769565  4948 tablet_service.cc:1511] Processing CreateTablet for tablet 3a10ed8c982e41c2b060892645dd7992 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:29:14.770874  4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a10ed8c982e41c2b060892645dd7992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.770591  5028 tablet_service.cc:1511] Processing CreateTablet for tablet 29feba7c56da402abe958d088341ea39 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:29:14.771883  5028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29feba7c56da402abe958d088341ea39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.754787  5083 raft_consensus.cc:385] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.772483  5083 raft_consensus.cc:740] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.773257  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 3a10ed8c982e41c2b060892645dd7992 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:29:14.773522  5083 consensus_queue.cc:260] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.774565  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a10ed8c982e41c2b060892645dd7992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.756911  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845a033bb8104895847b349dcb5eeccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.780539  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29feba7c56da402abe958d088341ea39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.793008  5083 ts_tablet_manager.cc:1434] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.002s
I20260709 12:29:14.799734  5085 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.800338  5085 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.800788  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:14.801477  5038 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:14.802611  5085 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:14.803308  4780 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:14.804299  5108 raft_consensus.cc:2804] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:14.804746  5108 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:14.804665  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:14.806433  4855 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.807130  5108 raft_consensus.cc:3060] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:14.808027  5083 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:14.811759  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:14.814282  4855 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.814778  4855 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:14.816023  5085 raft_consensus.cc:2749] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.816272  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 3a10ed8c982e41c2b060892645dd7992 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:29:14.817293  5091 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:14.817714  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a10ed8c982e41c2b060892645dd7992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.817835  5091 ts_tablet_manager.cc:1403] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.122s	user 0.005s	sys 0.010s
I20260709 12:29:14.821334  5091 raft_consensus.cc:359] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.822069  5091 raft_consensus.cc:385] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.822338  5091 raft_consensus.cc:740] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.826359  5108 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.829375  5108 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:14.830114  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:14.830761  5038 raft_consensus.cc:3060] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:14.838909  5038 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:14.833669  5082 tablet_bootstrap.cc:654] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.840684  5116 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.847676  5083 tablet_bootstrap.cc:654] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.848501  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:14.838563  5091 consensus_queue.cc:260] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.851294  5091 ts_tablet_manager.cc:1434] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:29:14.852636  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:14.855876  5116 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:14.856839  4964 raft_consensus.cc:2393] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:14.861101  5091 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:14.861209  4780 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:14.863456  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:14.864711  5037 raft_consensus.cc:2393] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:14.866721  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:14.867955  4815 raft_consensus.cc:2393] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:14.869032  5108 raft_consensus.cc:2804] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:14.871963  4929 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:14.873375  5116 raft_consensus.cc:2749] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.877836  5086 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:14.879241  5091 tablet_bootstrap.cc:654] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.865135  4713 catalog_manager.cc:5697] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:14.885484  4797 tablet_service.cc:1511] Processing CreateTablet for tablet 322e052ba6ca4c2989a3fd9aea3efc60 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:29:14.887123  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322e052ba6ca4c2989a3fd9aea3efc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.888907  5086 ts_tablet_manager.cc:1403] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.220s	user 0.017s	sys 0.013s
I20260709 12:29:14.896370  5108 raft_consensus.cc:697] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:14.897482  5108 consensus_queue.cc:237] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.897897  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 322e052ba6ca4c2989a3fd9aea3efc60 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:29:14.899458  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322e052ba6ca4c2989a3fd9aea3efc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.899343  4948 tablet_service.cc:1511] Processing CreateTablet for tablet 322e052ba6ca4c2989a3fd9aea3efc60 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:29:14.900883  4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322e052ba6ca4c2989a3fd9aea3efc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.907356  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 15e4db1a6dac4e348f16817448c5005e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:29:14.907024  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 15e4db1a6dac4e348f16817448c5005e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:29:14.909373  5086 raft_consensus.cc:359] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.910475  5086 raft_consensus.cc:385] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.910722  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e4db1a6dac4e348f16817448c5005e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.918303  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e4db1a6dac4e348f16817448c5005e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.920022  5083 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:14.920114  5082 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:14.920802  5083 ts_tablet_manager.cc:1403] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.113s	user 0.011s	sys 0.005s
I20260709 12:29:14.920894  5082 ts_tablet_manager.cc:1403] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.159s	user 0.005s	sys 0.014s
I20260709 12:29:14.924202  5082 raft_consensus.cc:359] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.924952  5082 raft_consensus.cc:385] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.925274  5082 raft_consensus.cc:740] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.926192  5082 consensus_queue.cc:260] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.932111  5082 ts_tablet_manager.cc:1434] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:29:14.933277  5082 tablet_bootstrap.cc:492] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:14.934518  4874 tablet_service.cc:1511] Processing CreateTablet for tablet 15e4db1a6dac4e348f16817448c5005e (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:29:14.935956  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e4db1a6dac4e348f16817448c5005e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.910827  5086 raft_consensus.cc:740] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.939667  5086 consensus_queue.cc:260] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.941035  5083 raft_consensus.cc:359] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.941843  5083 raft_consensus.cc:385] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.942046  5086 ts_tablet_manager.cc:1434] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 12:29:14.943267  5086 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:14.949244  5108 raft_consensus.cc:493] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.949805  5108 raft_consensus.cc:515] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:14.942147  5083 raft_consensus.cc:740] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.955214  5091 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:14.956351  5091 ts_tablet_manager.cc:1403] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.020s
I20260709 12:29:14.956099  5083 consensus_queue.cc:260] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:14.958037  4945 tablet_service.cc:1511] Processing CreateTablet for tablet 34f3921f98794b60a18335cd330faa86 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:29:14.958710  5083 ts_tablet_manager.cc:1434] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.002s
I20260709 12:29:14.959383  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f3921f98794b60a18335cd330faa86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.959033  4712 catalog_manager.cc:5697] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:14.960327  5108 leader_election.cc:290] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:14.963331  5082 tablet_bootstrap.cc:654] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.963413  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a1cd832b5450f8efa34e455eeead0" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:14.965739  5091 raft_consensus.cc:359] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:14.968760  4780 leader_election.cc:343] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.968912  5091 raft_consensus.cc:385] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:14.972062  5091 raft_consensus.cc:740] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:14.972604  5083 tablet_bootstrap.cc:492] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:14.974102  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a1cd832b5450f8efa34e455eeead0" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:14.975584  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 34f3921f98794b60a18335cd330faa86 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 12:29:14.976394  4778 leader_election.cc:343] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.976990  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f3921f98794b60a18335cd330faa86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.976960  4778 leader_election.cc:304] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:14.966427  5086 tablet_bootstrap.cc:654] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.978674  5108 raft_consensus.cc:2749] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:14.981959  5083 tablet_bootstrap.cc:654] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:14.982379  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 34f3921f98794b60a18335cd330faa86 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:29:14.983791  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f3921f98794b60a18335cd330faa86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:14.988175  5085 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:14.988749  5085 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.991178  5085 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:14.973129  5091 consensus_queue.cc:260] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:14.993127  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:14.994634  5091 ts_tablet_manager.cc:1434] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
W20260709 12:29:14.994944  4855 leader_election.cc:343] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.995810  5091 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:14.996435  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:14.998274  4855 leader_election.cc:343] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:14.998750  4855 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:14.999922  5085 raft_consensus.cc:2749] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.003248  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 07d989f2ffa44c5480a996303eb45fbb (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:29:15.004823  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d989f2ffa44c5480a996303eb45fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.012199  5116 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.019582  5116 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.023255  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:15.024878  4929 leader_election.cc:343] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.024873  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.025925  5082 tablet_bootstrap.cc:492] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
W20260709 12:29:15.026907  4927 leader_election.cc:343] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.026937  5116 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:15.018606  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 07d989f2ffa44c5480a996303eb45fbb (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:29:15.018606  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 07d989f2ffa44c5480a996303eb45fbb (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:29:15.029104  4927 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.031186  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d989f2ffa44c5480a996303eb45fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.032397  5116 raft_consensus.cc:2749] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.031198  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d989f2ffa44c5480a996303eb45fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.029130  5082 ts_tablet_manager.cc:1403] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.096s	user 0.007s	sys 0.006s
I20260709 12:29:15.038591  5083 tablet_bootstrap.cc:492] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.039191  5083 ts_tablet_manager.cc:1403] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.069s	user 0.015s	sys 0.023s
I20260709 12:29:15.039846  5082 raft_consensus.cc:359] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.040717  5082 raft_consensus.cc:385] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.041021  5082 raft_consensus.cc:740] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.042615  5083 raft_consensus.cc:359] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.056353  5116 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.046867  4877 tablet_service.cc:1511] Processing CreateTablet for tablet a47bf01353704752b89641cdf6e88e53 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:29:15.053603  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 8729ed090b704e7197d4f2df2b07eb29 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:29:15.058090  5116 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.059079  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729ed090b704e7197d4f2df2b07eb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.059149  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47bf01353704752b89641cdf6e88e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.042922  4797 tablet_service.cc:1511] Processing CreateTablet for tablet a47bf01353704752b89641cdf6e88e53 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:29:15.061623  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47bf01353704752b89641cdf6e88e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.044816  5086 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:15.062909  5086 ts_tablet_manager.cc:1403] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.120s	user 0.020s	sys 0.004s
I20260709 12:29:15.064119  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:15.064550  5082 consensus_queue.cc:260] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.066368  5086 raft_consensus.cc:359] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.067101  5086 raft_consensus.cc:385] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.067431  5086 raft_consensus.cc:740] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
W20260709 12:29:15.067831  4929 leader_election.cc:343] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.068151  5083 raft_consensus.cc:385] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.061403  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:15.062044  5116 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:15.068608  5083 raft_consensus.cc:740] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
W20260709 12:29:15.069964  4929 leader_election.cc:343] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.069769  5083 consensus_queue.cc:260] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.070442  4929 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:15.072047  5116 raft_consensus.cc:2749] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.072314  5083 ts_tablet_manager.cc:1434] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 12:29:15.046806  4948 tablet_service.cc:1511] Processing CreateTablet for tablet a47bf01353704752b89641cdf6e88e53 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:29:15.011790  5091 tablet_bootstrap.cc:654] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.077491  5086 consensus_queue.cc:260] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.080716  5082 ts_tablet_manager.cc:1434] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 12:29:15.082981  5086 ts_tablet_manager.cc:1434] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:29:15.091665  5086 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:15.084184  4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47bf01353704752b89641cdf6e88e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.079140  5083 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.095350  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 8729ed090b704e7197d4f2df2b07eb29 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:29:15.105093  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729ed090b704e7197d4f2df2b07eb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.107409  5083 tablet_bootstrap.cc:654] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.110654  5082 tablet_bootstrap.cc:492] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:15.118680  4874 tablet_service.cc:1511] Processing CreateTablet for tablet 8729ed090b704e7197d4f2df2b07eb29 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:29:15.121336  5086 tablet_bootstrap.cc:654] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.128820  4945 tablet_service.cc:1511] Processing CreateTablet for tablet 827a3f43a1924ae880f566f2f314143f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:29:15.120185  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729ed090b704e7197d4f2df2b07eb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.133636  5091 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:15.134739  5091 ts_tablet_manager.cc:1403] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.139s	user 0.015s	sys 0.008s
I20260709 12:29:15.135906  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 827a3f43a1924ae880f566f2f314143f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:29:15.137301  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 827a3f43a1924ae880f566f2f314143f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.141948  5082 tablet_bootstrap.cc:654] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.145387  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 827a3f43a1924ae880f566f2f314143f (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:29:15.146821  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 827a3f43a1924ae880f566f2f314143f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.139593  5091 raft_consensus.cc:359] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.148939  5091 raft_consensus.cc:385] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.149482  5091 raft_consensus.cc:740] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.157403  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 827a3f43a1924ae880f566f2f314143f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.163196  4877 tablet_service.cc:1511] Processing CreateTablet for tablet 80aa6fe92f51449ab69485480db26075 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:29:15.164722  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80aa6fe92f51449ab69485480db26075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.165580  5090 raft_consensus.cc:493] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.166257  5090 raft_consensus.cc:515] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.170643  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0d5e87b68e14a5580328547bf386316" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.171407  4890 raft_consensus.cc:2468] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:15.172539  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0d5e87b68e14a5580328547bf386316" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:15.173564  4815 raft_consensus.cc:2468] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:15.175153  4951 tablet_service.cc:1511] Processing CreateTablet for tablet 80aa6fe92f51449ab69485480db26075 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:29:15.175529  5001 leader_election.cc:304] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:15.176688  4951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80aa6fe92f51449ab69485480db26075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.177255  5090 leader_election.cc:290] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:15.177942  5090 raft_consensus.cc:2804] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.178442  5090 raft_consensus.cc:493] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.178210  5091 consensus_queue.cc:260] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.178898  5090 raft_consensus.cc:3060] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.185635  5086 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:15.186254  5086 ts_tablet_manager.cc:1403] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.095s	user 0.020s	sys 0.007s
I20260709 12:29:15.189576  5086 raft_consensus.cc:359] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.190402  5086 raft_consensus.cc:385] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.190737  5086 raft_consensus.cc:740] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.191538  5086 consensus_queue.cc:260] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.193552  5083 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.197029  5083 ts_tablet_manager.cc:1403] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.118s	user 0.025s	sys 0.040s
I20260709 12:29:15.193925  5086 ts_tablet_manager.cc:1434] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 12:29:15.201261  5086 tablet_bootstrap.cc:492] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:15.202549  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 80aa6fe92f51449ab69485480db26075 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:29:15.203926  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80aa6fe92f51449ab69485480db26075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.206310  5082 tablet_bootstrap.cc:492] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:15.206844  5082 ts_tablet_manager.cc:1403] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.001s
I20260709 12:29:15.206601  4805 tablet_service.cc:1511] Processing CreateTablet for tablet e359e160bb414d9099cff45b96f6b40d (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:29:15.208132  5090 raft_consensus.cc:515] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.190008  5091 ts_tablet_manager.cc:1434] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:29:15.211988  4949 tablet_service.cc:1511] Processing CreateTablet for tablet e359e160bb414d9099cff45b96f6b40d (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:29:15.212693  4877 tablet_service.cc:1511] Processing CreateTablet for tablet e359e160bb414d9099cff45b96f6b40d (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:29:15.213111  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e359e160bb414d9099cff45b96f6b40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.213356  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e359e160bb414d9099cff45b96f6b40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.213840  4877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e359e160bb414d9099cff45b96f6b40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.215193  5085 raft_consensus.cc:493] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.215164  5082 raft_consensus.cc:359] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.215694  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0d5e87b68e14a5580328547bf386316" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:15.215826  5085 raft_consensus.cc:515] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.216413  4815 raft_consensus.cc:3060] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.218065  5085 leader_election.cc:290] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:15.210532  5083 raft_consensus.cc:359] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.219995  5022 tablet_service.cc:1511] Processing CreateTablet for tablet 64da30821dfb420183c42da517b4a6e1 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:29:15.216060  5082 raft_consensus.cc:385] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.224009  5082 raft_consensus.cc:740] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.224774  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:15.224906  5082 consensus_queue.cc:260] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:15.226302  4855 leader_election.cc:343] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.226971  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0d5e87b68e14a5580328547bf386316" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:15.227728  4890 raft_consensus.cc:3060] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.220623  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:15.226538  5022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64da30821dfb420183c42da517b4a6e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:15.234840  4855 leader_election.cc:343] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.235105  4890 raft_consensus.cc:2468] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:15.235363  4855 leader_election.cc:304] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:15.223655  5083 raft_consensus.cc:385] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.221017  5090 leader_election.cc:290] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:15.236394  5085 raft_consensus.cc:2749] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.236421  5083 raft_consensus.cc:740] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.237191  5001 leader_election.cc:304] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:15.238579  5086 tablet_bootstrap.cc:654] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.238888  4797 tablet_service.cc:1511] Processing CreateTablet for tablet 64da30821dfb420183c42da517b4a6e1 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:29:15.240262  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64da30821dfb420183c42da517b4a6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.241801  5090 raft_consensus.cc:2804] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.246853  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 64da30821dfb420183c42da517b4a6e1 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:29:15.247766  4954 tablet_service.cc:1511] Processing CreateTablet for tablet bea31c506847448087d1d664687c1d57 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:29:15.248641  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64da30821dfb420183c42da517b4a6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.249112  4954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea31c506847448087d1d664687c1d57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.251416  5091 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:15.254163  5090 raft_consensus.cc:697] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:15.259675  5090 consensus_queue.cc:237] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.242730  5083 consensus_queue.cc:260] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.265241  5083 ts_tablet_manager.cc:1434] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.001s
I20260709 12:29:15.266381  4815 raft_consensus.cc:2468] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:15.267563  5024 tablet_service.cc:1511] Processing CreateTablet for tablet bea31c506847448087d1d664687c1d57 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:29:15.270279  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea31c506847448087d1d664687c1d57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.266407  5083 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.274497  5037 raft_consensus.cc:3060] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.277748  5116 consensus_queue.cc:1048] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:29:15.287557  5108 raft_consensus.cc:493] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.288125  5108 raft_consensus.cc:515] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.289223  5082 ts_tablet_manager.cc:1434] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 12:29:15.290307  5082 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:15.290481  5108 leader_election.cc:290] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:15.279307  5091 tablet_bootstrap.cc:654] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.294016  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.296622  4889 raft_consensus.cc:2468] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:15.297838  5082 tablet_bootstrap.cc:654] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.295735  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.298621  4964 raft_consensus.cc:2468] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:15.299618  4778 leader_election.cc:304] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:15.301501  5108 raft_consensus.cc:2804] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.301956  5108 raft_consensus.cc:493] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.302366  5108 raft_consensus.cc:3060] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.323119  5083 tablet_bootstrap.cc:654] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.326355  4804 tablet_service.cc:1511] Processing CreateTablet for tablet bea31c506847448087d1d664687c1d57 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:29:15.328071  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea31c506847448087d1d664687c1d57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.329150  5091 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:15.329701  5091 ts_tablet_manager.cc:1403] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.002s
I20260709 12:29:15.332672  5085 raft_consensus.cc:493] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.333209  5085 raft_consensus.cc:515] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.335374  5085 leader_election.cc:290] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:15.338284  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.340624  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
W20260709 12:29:15.341877  4855 leader_election.cc:343] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:15.343588  4855 leader_election.cc:343] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.344059  4855 leader_election.cc:304] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:15.345255  5085 raft_consensus.cc:2749] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.345111  5125 consensus_queue.cc:1048] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:15.349545  5140 raft_consensus.cc:493] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.350023  5125 consensus_queue.cc:1048] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:15.350082  5140 raft_consensus.cc:515] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.353726  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.354401  4963 raft_consensus.cc:2468] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:15.355538  5125 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.356122  5003 leader_election.cc:304] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:15.356091  5125 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.357523  5140 leader_election.cc:290] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:15.358330  5140 raft_consensus.cc:2804] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.358932  5140 raft_consensus.cc:493] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.359335  5140 raft_consensus.cc:3060] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.359375  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:15.361138  4778 leader_election.cc:343] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.366254  5125 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:15.366412  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.367197  4963 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:15.368729  4780 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.371011  5125 raft_consensus.cc:2804] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.371513  5125 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.339367  5091 raft_consensus.cc:359] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.372303  5091 raft_consensus.cc:385] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.373751  5091 raft_consensus.cc:740] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.376353  5091 consensus_queue.cc:260] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.417349  5090 raft_consensus.cc:493] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.418064  5090 raft_consensus.cc:515] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.371876  5125 raft_consensus.cc:3060] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.388603  4871 tablet_service.cc:1511] Processing CreateTablet for tablet 83e9766db6054724a5055336fa6baa91 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:29:15.421929  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e9766db6054724a5055336fa6baa91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.389587  5085 raft_consensus.cc:493] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.390566  4949 tablet_service.cc:1511] Processing CreateTablet for tablet 83e9766db6054724a5055336fa6baa91 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:29:15.394024  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:15.379055  5086 tablet_bootstrap.cc:492] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:15.382369  5140 raft_consensus.cc:515] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.411697  5108 raft_consensus.cc:515] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.486564  5090 leader_election.cc:290] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:15.487941  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a1cd832b5450f8efa34e455eeead0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:15.488881  4814 raft_consensus.cc:2468] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:15.490051  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a1cd832b5450f8efa34e455eeead0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.490669  5083 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.491256  5083 ts_tablet_manager.cc:1403] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.225s	user 0.071s	sys 0.049s
I20260709 12:29:15.491708  5003 leader_election.cc:304] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:15.436640  5085 raft_consensus.cc:515] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.493850  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 45ceef50c6394676adfa8f58903d2b90 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:29:15.494406  5027 tablet_service.cc:1511] Processing CreateTablet for tablet 83e9766db6054724a5055336fa6baa91 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:29:15.495608  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e9766db6054724a5055336fa6baa91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.494294  5083 raft_consensus.cc:359] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.493176  5149 raft_consensus.cc:493] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.496102  5083 raft_consensus.cc:385] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.503793  5083 raft_consensus.cc:740] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.482488  5086 ts_tablet_manager.cc:1403] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.281s	user 0.014s	sys 0.008s
I20260709 12:29:15.510417  5083 consensus_queue.cc:260] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.495296  5085 leader_election.cc:290] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:15.495311  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45ceef50c6394676adfa8f58903d2b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.504856  5090 raft_consensus.cc:2804] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.528522  5090 raft_consensus.cc:493] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:15.523501  5003 leader_election.cc:343] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.528879  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.528961  5090 raft_consensus.cc:3060] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.530350  4711 catalog_manager.cc:5697] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:15.526500  5001 leader_election.cc:343] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.527292  5151 consensus_queue.cc:1048] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:15.534855  5120 log.cc:927] Time spent T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:29:15.536962  4949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e9766db6054724a5055336fa6baa91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.543839  5083 ts_tablet_manager.cc:1434] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 12:29:15.509095  4874 tablet_service.cc:1511] Processing CreateTablet for tablet 45ceef50c6394676adfa8f58903d2b90 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:29:15.546119  5108 leader_election.cc:290] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:15.548703  5083 tablet_bootstrap.cc:492] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.515759  5149 raft_consensus.cc:515] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:15.555157  4855 leader_election.cc:343] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.555913  5140 leader_election.cc:290] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:15.557020  4948 tablet_service.cc:1511] Processing CreateTablet for tablet 45ceef50c6394676adfa8f58903d2b90 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:29:15.558535  4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45ceef50c6394676adfa8f58903d2b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.559841  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:15.560658  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:15.560771  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:15.561468  4964 raft_consensus.cc:3060] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:15.561659  4855 leader_election.cc:343] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.562214  4855 leader_election.cc:304] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
W20260709 12:29:15.562374  5003 leader_election.cc:343] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.564976  5091 ts_tablet_manager.cc:1434] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.235s	user 0.028s	sys 0.038s
I20260709 12:29:15.565407  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:15.566097  4963 raft_consensus.cc:3060] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.566329  5091 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:15.570891  5083 tablet_bootstrap.cc:654] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.570410  4871 tablet_service.cc:1511] Processing CreateTablet for tablet fa2320b4fabb4defa55ae5284ad3f4e4 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:29:15.572046  5125 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.574527  5086 raft_consensus.cc:359] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.575506  5086 raft_consensus.cc:385] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.578245  5091 tablet_bootstrap.cc:654] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.579325  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:15.580060  4962 raft_consensus.cc:3060] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.584333  4945 tablet_service.cc:1511] Processing CreateTablet for tablet bb55bf7ad4704d18bb564eeb5320fee8 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:29:15.585772  4945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb55bf7ad4704d18bb564eeb5320fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.586275  4874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45ceef50c6394676adfa8f58903d2b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.587051  4797 tablet_service.cc:1511] Processing CreateTablet for tablet bb55bf7ad4704d18bb564eeb5320fee8 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:29:15.588423  4797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb55bf7ad4704d18bb564eeb5320fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.588831  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:15.590572  5149 leader_election.cc:290] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
W20260709 12:29:15.590719  4778 leader_election.cc:343] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.591917  4804 tablet_service.cc:1511] Processing CreateTablet for tablet fa2320b4fabb4defa55ae5284ad3f4e4 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:29:15.593106  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa2320b4fabb4defa55ae5284ad3f4e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.600505  4871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa2320b4fabb4defa55ae5284ad3f4e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.602375  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.605410  5024 tablet_service.cc:1511] Processing CreateTablet for tablet bb55bf7ad4704d18bb564eeb5320fee8 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:29:15.606775  5024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb55bf7ad4704d18bb564eeb5320fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:15.608804  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:15.609685  4889 raft_consensus.cc:3060] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.575930  5086 raft_consensus.cc:740] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.622786  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.624351  5086 consensus_queue.cc:260] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.625521  5001 leader_election.cc:304] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.633327  5151 raft_consensus.cc:493] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.634219  5151 raft_consensus.cc:515] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.574332  5082 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:15.637506  5082 ts_tablet_manager.cc:1403] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.347s	user 0.056s	sys 0.133s
W20260709 12:29:15.639863  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:15.641842  5027 tablet_service.cc:1511] Processing CreateTablet for tablet fa2320b4fabb4defa55ae5284ad3f4e4 (DEFAULT_TABLE table=test-workload [id=85954f13f54e4d7ea913890070b4bfa7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:29:15.642917  5027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa2320b4fabb4defa55ae5284ad3f4e4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:15.657095  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 95475.3us
I20260709 12:29:15.663303  5157 raft_consensus.cc:2749] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.663828  5082 raft_consensus.cc:359] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.664760  5082 raft_consensus.cc:385] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.665102  5082 raft_consensus.cc:740] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.665618  5090 raft_consensus.cc:515] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.665987  5082 consensus_queue.cc:260] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.669612  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a1cd832b5450f8efa34e455eeead0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:15.668216  5090 leader_election.cc:290] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:15.674535  5036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:15.668890  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a1cd832b5450f8efa34e455eeead0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:15.652405  5125 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:15.672981  5151 leader_election.cc:290] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:15.694940  4815 raft_consensus.cc:3060] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.675606  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.696503  5036 raft_consensus.cc:2468] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
W20260709 12:29:15.696849  4927 leader_election.cc:343] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.697479  4890 raft_consensus.cc:3060] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.698097  4929 leader_election.cc:304] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.699393  5151 raft_consensus.cc:2804] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.699889  5151 raft_consensus.cc:493] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.677495  4964 raft_consensus.cc:2468] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:15.691509  5082 ts_tablet_manager.cc:1434] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.001s
I20260709 12:29:15.700290  5151 raft_consensus.cc:3060] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.679925  5086 ts_tablet_manager.cc:1434] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.170s	user 0.007s	sys 0.003s
I20260709 12:29:15.712169  5091 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:15.712857  5091 ts_tablet_manager.cc:1403] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.147s	user 0.017s	sys 0.030s
I20260709 12:29:15.677945  4963 raft_consensus.cc:2468] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:15.686861  4889 raft_consensus.cc:2468] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:15.718681  4780 leader_election.cc:304] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:15.722609  5082 tablet_bootstrap.cc:492] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:15.725507  5003 leader_election.cc:304] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:15.727768  5140 raft_consensus.cc:2804] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.728585  5140 raft_consensus.cc:697] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:15.729513  4815 raft_consensus.cc:2468] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:15.729854  5140 consensus_queue.cc:237] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.730832  5003 leader_election.cc:304] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:15.728025  5086 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:15.732005  5147 raft_consensus.cc:2804] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.732626  5147 raft_consensus.cc:697] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:15.733116  5108 raft_consensus.cc:2804] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.733712  5108 raft_consensus.cc:697] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:15.733512  5147 consensus_queue.cc:237] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.734714  5108 consensus_queue.cc:237] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.743124  5149 raft_consensus.cc:2749] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.748881  5091 raft_consensus.cc:359] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.749804  5091 raft_consensus.cc:385] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.750145  5091 raft_consensus.cc:740] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.751119  5091 consensus_queue.cc:260] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.751976  5086 tablet_bootstrap.cc:654] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.763625  5082 tablet_bootstrap.cc:654] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.769069  5083 tablet_bootstrap.cc:492] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.769636  5083 ts_tablet_manager.cc:1403] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.221s	user 0.032s	sys 0.048s
I20260709 12:29:15.770982  5151 raft_consensus.cc:515] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.772619  5091 ts_tablet_manager.cc:1434] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.001s
I20260709 12:29:15.773444  5151 leader_election.cc:290] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:15.773906  5091 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:15.774576  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:15.775192  5038 raft_consensus.cc:3060] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.777686  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
W20260709 12:29:15.784204  4927 leader_election.cc:343] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.786953  4712 catalog_manager.cc:5697] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:15.788734  4890 raft_consensus.cc:2468] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:15.756083  4962 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:15.790042  5091 tablet_bootstrap.cc:654] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.790249  4780 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.772665  5083 raft_consensus.cc:359] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.793861  5038 raft_consensus.cc:2468] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:15.794754  5083 raft_consensus.cc:385] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.797463  5083 raft_consensus.cc:740] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.799512  5083 consensus_queue.cc:260] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.799569  4929 leader_election.cc:304] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.798605  5108 raft_consensus.cc:2804] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.803573  5151 raft_consensus.cc:2804] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.806443  5083 ts_tablet_manager.cc:1434] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.037s	user 0.001s	sys 0.007s
I20260709 12:29:15.806488  5151 raft_consensus.cc:697] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:15.807725  5083 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.808686  5151 consensus_queue.cc:237] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.806917  5108 raft_consensus.cc:697] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:15.823998  5083 tablet_bootstrap.cc:654] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.827159  5091 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:15.827206  5147 consensus_queue.cc:1048] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:15.828105  5091 ts_tablet_manager.cc:1403] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.015s
I20260709 12:29:15.832588  5086 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:15.832489  5091 raft_consensus.cc:359] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.833190  5086 ts_tablet_manager.cc:1403] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.105s	user 0.017s	sys 0.003s
I20260709 12:29:15.833316  5091 raft_consensus.cc:385] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.833930  5091 raft_consensus.cc:740] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.834681  5091 consensus_queue.cc:260] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.836320  5091 ts_tablet_manager.cc:1434] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:15.846561  5083 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.837116  5086 raft_consensus.cc:359] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.854480  5083 ts_tablet_manager.cc:1403] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.000s
I20260709 12:29:15.854658  5086 raft_consensus.cc:385] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.855033  5086 raft_consensus.cc:740] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.855393  5090 consensus_queue.cc:1048] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:15.827598  5108 consensus_queue.cc:237] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.857263  5125 raft_consensus.cc:493] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.857885  5125 raft_consensus.cc:515] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.860942  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:15.862740  4778 leader_election.cc:343] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.857896  5083 raft_consensus.cc:359] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.864477  5083 raft_consensus.cc:385] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.848527  4713 catalog_manager.cc:5697] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:15.864831  5083 raft_consensus.cc:740] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.865399  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:15.865597  5083 consensus_queue.cc:260] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.866271  5038 raft_consensus.cc:2468] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:15.867748  5083 ts_tablet_manager.cc:1434] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:29:15.868456  4780 leader_election.cc:304] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.869072  5083 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.869957  5155 raft_consensus.cc:2804] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.827164  5082 tablet_bootstrap.cc:492] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:15.870469  5155 raft_consensus.cc:493] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.871160  5082 ts_tablet_manager.cc:1403] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.149s	user 0.016s	sys 0.001s
I20260709 12:29:15.871079  5155 raft_consensus.cc:3060] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.876824  5090 raft_consensus.cc:493] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.877429  5090 raft_consensus.cc:515] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:15.853566  4712 catalog_manager.cc:5697] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:15.879678  5090 leader_election.cc:290] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:15.864431  5125 leader_election.cc:290] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:15.883136  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:15.885887  5001 leader_election.cc:343] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.879899  5155 raft_consensus.cc:515] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.837358  5091 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:15.882733  4712 catalog_manager.cc:5697] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:15.888793  5083 tablet_bootstrap.cc:654] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.883957  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:15.890661  5003 leader_election.cc:343] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.891160  5003 leader_election.cc:304] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.891577  5155 leader_election.cc:290] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:15.892400  5090 raft_consensus.cc:2749] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:15.892766  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:15.894826  5091 tablet_bootstrap.cc:654] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.892844  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:15.895694  5038 raft_consensus.cc:3060] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:15.900975  4778 leader_election.cc:343] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:15.902385  5038 raft_consensus.cc:2468] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:15.903824  4780 leader_election.cc:304] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:15.892552  5086 consensus_queue.cc:260] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.892282  5082 raft_consensus.cc:359] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.905337  5082 raft_consensus.cc:385] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.905086  5108 raft_consensus.cc:2804] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:15.905714  5082 raft_consensus.cc:740] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.906057  5108 raft_consensus.cc:697] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:15.907013  5108 consensus_queue.cc:237] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.908564  5082 consensus_queue.cc:260] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:15.909590  5086 ts_tablet_manager.cc:1434] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:29:15.910544  5086 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:15.911028  5082 ts_tablet_manager.cc:1434] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.000s
I20260709 12:29:15.908823  4711 catalog_manager.cc:5697] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:15.913312  5083 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.914073  5083 ts_tablet_manager.cc:1403] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.005s
I20260709 12:29:15.916972  5083 raft_consensus.cc:359] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.917654  5083 raft_consensus.cc:385] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.917914  5083 raft_consensus.cc:740] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.918725  5083 consensus_queue.cc:260] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.912647  5082 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:15.921140  5083 ts_tablet_manager.cc:1434] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:29:15.921694  5086 tablet_bootstrap.cc:654] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.922132  5083 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.925261  5091 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:15.925854  5091 ts_tablet_manager.cc:1403] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.007s
I20260709 12:29:15.929375  5091 raft_consensus.cc:359] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.930123  5091 raft_consensus.cc:385] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.930413  5091 raft_consensus.cc:740] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.930708  5083 tablet_bootstrap.cc:654] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.931182  5091 consensus_queue.cc:260] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.933581  5086 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:15.934208  5086 ts_tablet_manager.cc:1403] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.005s
I20260709 12:29:15.936695  5082 tablet_bootstrap.cc:654] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.938048  5086 raft_consensus.cc:359] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.938972  5086 raft_consensus.cc:385] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.939314  5086 raft_consensus.cc:740] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.941061  5155 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:15.941622  5091 ts_tablet_manager.cc:1434] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:29:15.941725  5155 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.942773  5091 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:15.946331  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:15.947285  4890 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:15.949087  4778 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:15.950274  5125 raft_consensus.cc:2804] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:15.950748  5125 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:15.951205  5125 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.952167  5091 tablet_bootstrap.cc:654] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.960563  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:15.961635  4962 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:15.959908  5155 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:15.963726  5082 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:15.964373  5082 ts_tablet_manager.cc:1403] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.004s
I20260709 12:29:15.966148  5083 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:15.966635  5083 ts_tablet_manager.cc:1403] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.018s
I20260709 12:29:15.967368  5082 raft_consensus.cc:359] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.969398  5083 raft_consensus.cc:359] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.970119  5083 raft_consensus.cc:385] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.970183  5125 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.970458  5083 raft_consensus.cc:740] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.968125  5082 raft_consensus.cc:385] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.971052  5082 raft_consensus.cc:740] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.971443  5083 consensus_queue.cc:260] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.971863  5082 consensus_queue.cc:260] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.973076  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:15.973652  4890 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.973966  5083 ts_tablet_manager.cc:1434] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:29:15.974344  5082 ts_tablet_manager.cc:1434] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:29:15.974949  5083 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:15.975334  5082 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:15.940618  5086 consensus_queue.cc:260] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:15.981870  5083 tablet_bootstrap.cc:654] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:15.985653  5091 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:15.986209  5091 ts_tablet_manager.cc:1403] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.002s
I20260709 12:29:15.986276  5086 ts_tablet_manager.cc:1434] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:29:15.987622  5086 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:15.990830  5125 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:15.992257  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:15.993072  4962 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:15.996415  5091 raft_consensus.cc:359] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:15.997167  5091 raft_consensus.cc:385] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:15.997455  5091 raft_consensus.cc:740] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:15.997841  4890 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:15.999208  4778 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.000131  5125 raft_consensus.cc:2804] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.000725  5091 consensus_queue.cc:260] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.000730  5125 raft_consensus.cc:697] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:16.002559  5125 consensus_queue.cc:237] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.002959  4962 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:15.999895  4712 catalog_manager.cc:5697] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:16.006251  5086 tablet_bootstrap.cc:654] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.006774  5091 ts_tablet_manager.cc:1434] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:29:16.000094  5082 tablet_bootstrap.cc:654] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.012790  5091 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:16.018824  5083 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:16.022102  5083 ts_tablet_manager.cc:1403] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.006s
I20260709 12:29:16.024844  5086 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:16.025421  5086 ts_tablet_manager.cc:1403] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.009s
I20260709 12:29:16.026050  5082 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:16.026700  5082 ts_tablet_manager.cc:1403] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.003s
I20260709 12:29:16.026065  5083 raft_consensus.cc:359] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.027123  5083 raft_consensus.cc:385] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.027484  5083 raft_consensus.cc:740] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.028396  5083 consensus_queue.cc:260] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.028764  5086 raft_consensus.cc:359] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.029603  5086 raft_consensus.cc:385] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.029909  5086 raft_consensus.cc:740] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.029937  5082 raft_consensus.cc:359] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.030856  5082 raft_consensus.cc:385] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.030784  5086 consensus_queue.cc:260] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.031376  5082 raft_consensus.cc:740] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.032275  5082 consensus_queue.cc:260] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.034824  5083 ts_tablet_manager.cc:1434] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:29:16.035934  5083 tablet_bootstrap.cc:492] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:16.033210  4713 catalog_manager.cc:5697] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:16.042208  5083 tablet_bootstrap.cc:654] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.042404  5091 tablet_bootstrap.cc:654] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.045919  5086 ts_tablet_manager.cc:1434] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:29:16.048199  5082 ts_tablet_manager.cc:1434] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:29:16.049315  5082 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:16.057391  5086 tablet_bootstrap.cc:492] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:16.059175  5083 tablet_bootstrap.cc:492] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:16.059942  5083 ts_tablet_manager.cc:1403] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.008s
I20260709 12:29:16.061498  5090 raft_consensus.cc:493] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.062057  5090 raft_consensus.cc:515] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.064831  5082 tablet_bootstrap.cc:654] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.065730  5086 tablet_bootstrap.cc:654] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.063057  5083 raft_consensus.cc:359] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.067974  5083 raft_consensus.cc:385] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.068320  5083 raft_consensus.cc:740] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.069041  5083 consensus_queue.cc:260] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.064317  5090 leader_election.cc:290] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:16.069720  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.070473  4890 raft_consensus.cc:2468] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:16.070683  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.071206  5083 ts_tablet_manager.cc:1434] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.003s
I20260709 12:29:16.071939  5001 leader_election.cc:304] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.072489  5083 tablet_bootstrap.cc:492] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:16.072964  5140 raft_consensus.cc:2804] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.073415  5140 raft_consensus.cc:493] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.073797  5140 raft_consensus.cc:3060] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:16.076817  5003 leader_election.cc:343] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.076416  5091 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:16.078193  5091 ts_tablet_manager.cc:1403] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.010s
I20260709 12:29:16.078295  5082 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:16.078979  5082 ts_tablet_manager.cc:1403] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.001s
I20260709 12:29:16.080641  5083 tablet_bootstrap.cc:654] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.081382  5091 raft_consensus.cc:359] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.082129  5091 raft_consensus.cc:385] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.082424  5091 raft_consensus.cc:740] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.082618  5082 raft_consensus.cc:359] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.083523  5082 raft_consensus.cc:385] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.083925  5082 raft_consensus.cc:740] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.083253  5091 consensus_queue.cc:260] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.084869  5082 consensus_queue.cc:260] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.086297  5091 ts_tablet_manager.cc:1434] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:16.087350  5082 ts_tablet_manager.cc:1434] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:29:16.087580  5091 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:16.088641  5082 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:16.096398  5082 tablet_bootstrap.cc:654] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.107048  5091 tablet_bootstrap.cc:654] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.106256  5140 raft_consensus.cc:515] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.110029  5155 raft_consensus.cc:493] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.110710  5155 raft_consensus.cc:515] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.112627  5083 tablet_bootstrap.cc:492] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:16.112926  5155 leader_election.cc:290] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.113282  5083 ts_tablet_manager.cc:1403] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.006s
I20260709 12:29:16.113164  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:16.114349  4815 raft_consensus.cc:3060] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.110051  5125 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.117152  5125 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.117707  5140 leader_election.cc:290] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:16.118763  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.119402  5125 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.119700  4962 raft_consensus.cc:2393] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:16.120069  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.120745  4890 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:16.121088  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.117178  5083 raft_consensus.cc:359] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.122169  5083 raft_consensus.cc:385] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.122464  5083 raft_consensus.cc:740] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.122750  4778 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.123222  5083 consensus_queue.cc:260] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.123888  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:16.124210  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.124490  4890 raft_consensus.cc:3060] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.123363  4780 leader_election.cc:304] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:16.125602  5083 ts_tablet_manager.cc:1434] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:29:16.110137  5086 tablet_bootstrap.cc:492] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
W20260709 12:29:16.126581  4780 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.126761  5086 ts_tablet_manager.cc:1403] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.070s	user 0.006s	sys 0.008s
I20260709 12:29:16.127679  5155 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.131271  5083 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:16.129391  5125 raft_consensus.cc:3060] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.129926  4815 raft_consensus.cc:2468] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:16.137174  5003 leader_election.cc:304] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:16.138113  5083 tablet_bootstrap.cc:654] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.138224  5140 raft_consensus.cc:2804] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.138933  5140 raft_consensus.cc:697] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:16.139897  5140 consensus_queue.cc:237] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.141358  5091 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:16.141896  5091 ts_tablet_manager.cc:1403] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.000s
I20260709 12:29:16.142164  5108 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.142663  5108 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.145161  5108 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.145115  5091 raft_consensus.cc:359] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.146327  5091 raft_consensus.cc:385] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.146580  5091 raft_consensus.cc:740] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.147243  5091 consensus_queue.cc:260] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.149220  5082 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:16.149678  5091 ts_tablet_manager.cc:1434] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:16.149816  5082 ts_tablet_manager.cc:1403] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.015s
I20260709 12:29:16.150524  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.154361  4962 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:16.152825  5091 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:16.153070  5082 raft_consensus.cc:359] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.131513  5155 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.155189  5108 consensus_queue.cc:1048] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.155534  5155 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.156608  4780 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:16.149571  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.160610  5037 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:16.160702  5108 raft_consensus.cc:2804] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.161321  5108 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.161808  5108 raft_consensus.cc:3060] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.161768  5082 raft_consensus.cc:385] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.162603  5082 raft_consensus.cc:740] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.162357  5091 tablet_bootstrap.cc:654] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.168030  5155 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.141669  4890 raft_consensus.cc:2468] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:16.168819  5125 raft_consensus.cc:2749] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.169241  5082 consensus_queue.cc:260] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.171590  5108 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.173301  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:16.173310  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:16.174423  5082 ts_tablet_manager.cc:1434] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:29:16.175634  5082 tablet_bootstrap.cc:492] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:16.130227  5086 raft_consensus.cc:359] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.176222  5108 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
W20260709 12:29:16.175446  4780 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.176378  5086 raft_consensus.cc:385] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.178768  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:16.179482  4962 raft_consensus.cc:3060] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.179703  5155 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.185434  5082 tablet_bootstrap.cc:654] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.185742  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:16.186676  5037 raft_consensus.cc:3060] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.187947  5157 raft_consensus.cc:493] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.188628  5157 raft_consensus.cc:515] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.189714  5140 consensus_queue.cc:1048] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:16.194907  5157 leader_election.cc:290] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.177199  5086 raft_consensus.cc:740] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.200430  4890 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.201673  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.202572  5038 raft_consensus.cc:2393] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:16.202731  4709 catalog_manager.cc:5697] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:16.207661  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999b4d2234f74d80ad85ddf37ec99b01" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.216602  4855 leader_election.cc:304] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:16.217976  5157 raft_consensus.cc:3060] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.221428  5125 consensus_queue.cc:1048] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.210839  5086 consensus_queue.cc:260] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.232214  5125 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.232775  5125 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.234923  5125 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.236487  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.244442  5086 ts_tablet_manager.cc:1434] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.117s	user 0.008s	sys 0.000s
W20260709 12:29:16.238157  4780 leader_election.cc:343] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.245957  5086 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:16.251395  5157 raft_consensus.cc:2749] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.245952  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.253512  5037 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:16.258019  4962 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:16.254316  4889 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:16.261025  4780 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:16.262742  5199 raft_consensus.cc:2804] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.263285  5199 raft_consensus.cc:697] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:16.263619  5083 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:16.264151  5083 ts_tablet_manager.cc:1403] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.133s	user 0.031s	sys 0.013s
I20260709 12:29:16.264356  5140 consensus_queue.cc:1048] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:16.267339  4778 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:16.267642  5083 raft_consensus.cc:359] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.268380  5083 raft_consensus.cc:385] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.268646  5083 raft_consensus.cc:740] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.266875  5091 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:16.264202  5199 consensus_queue.cc:237] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.269526  5091 ts_tablet_manager.cc:1403] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.117s	user 0.019s	sys 0.022s
I20260709 12:29:16.269856  5083 consensus_queue.cc:260] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.270507  5082 tablet_bootstrap.cc:492] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:16.270959  5082 ts_tablet_manager.cc:1403] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.096s	user 0.016s	sys 0.011s
I20260709 12:29:16.268859  5125 raft_consensus.cc:2804] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.273259  5125 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.273766  5125 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.280203  5082 raft_consensus.cc:359] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.282697  5082 raft_consensus.cc:385] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.283115  5082 raft_consensus.cc:740] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.284256  5082 consensus_queue.cc:260] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.281718  5091 raft_consensus.cc:359] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.285907  5091 raft_consensus.cc:385] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.286460  5091 raft_consensus.cc:740] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.292784  5090 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.293798  5086 tablet_bootstrap.cc:654] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.293457  5090 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.296057  5149 raft_consensus.cc:493] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.297936  5082 ts_tablet_manager.cc:1434] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.001s
I20260709 12:29:16.300612  5082 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:16.303354  5083 ts_tablet_manager.cc:1434] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:29:16.289338  5091 consensus_queue.cc:260] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.305070  5083 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:16.307614  5140 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.307251  5149 raft_consensus.cc:515] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.312433  5140 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.314271  5149 leader_election.cc:290] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.314631  5140 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.315189  5090 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.296139  5151 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.316160  5151 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.324666  5157 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.325296  5157 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.318482  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.327315  4889 raft_consensus.cc:2468] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:16.327646  5090 consensus_queue.cc:1048] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.335974  5091 ts_tablet_manager.cc:1434] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.002s
I20260709 12:29:16.337786  5091 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:16.319833  4890 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:16.335070  5157 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.336895  5116 raft_consensus.cc:493] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.338294  5001 leader_election.cc:304] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.342252  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.349735  4962 raft_consensus.cc:2393] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:16.349561  5090 raft_consensus.cc:2804] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.350407  5090 raft_consensus.cc:493] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.351958  5090 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.344784  5116 raft_consensus.cc:515] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.355139  4778 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:16.366204  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.366173  5091 tablet_bootstrap.cc:654] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.361086  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.370576  4962 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:16.371306  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.372018  4889 raft_consensus.cc:2468] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:16.372816  5199 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.373356  4927 leader_election.cc:304] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.378044  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.383167  5205 raft_consensus.cc:493] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.383841  5205 raft_consensus.cc:515] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.386150  5205 leader_election.cc:290] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.388200  4855 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:16.389582  5157 raft_consensus.cc:3060] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.392108  5151 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.393899  5151 raft_consensus.cc:2804] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.398301  5151 raft_consensus.cc:493] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.377390  5199 raft_consensus.cc:697] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:16.393128  5116 leader_election.cc:290] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.398996  5083 tablet_bootstrap.cc:654] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.399890  5125 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.405021  5116 raft_consensus.cc:493] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.404071  5151 raft_consensus.cc:3060] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.408921  5199 consensus_queue.cc:237] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.409659  5090 raft_consensus.cc:515] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.419616  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.419832  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.420437  5038 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:16.420964  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:16.422667  5003 leader_election.cc:343] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.423125  5082 tablet_bootstrap.cc:654] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.422859  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.423394  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.424264  4815 raft_consensus.cc:2468] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:16.425509  5086 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:16.425628  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.422885  5205 consensus_queue.cc:1048] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:29:16.426327  5086 ts_tablet_manager.cc:1403] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.181s	user 0.016s	sys 0.036s
I20260709 12:29:16.428941  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.414424  5116 raft_consensus.cc:515] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.430910  5157 raft_consensus.cc:2749] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.430931  4962 raft_consensus.cc:2468] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:16.432062  5116 leader_election.cc:290] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:16.432168  4929 leader_election.cc:343] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.437655  5157 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.439107  5125 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.443642  5090 leader_election.cc:290] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.446195  5116 consensus_queue.cc:1048] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:16.448378  5086 raft_consensus.cc:359] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.449354  5086 raft_consensus.cc:385] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.449700  5086 raft_consensus.cc:740] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.449993  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:16.450587  5086 consensus_queue.cc:260] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.451949  5091 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
W20260709 12:29:16.451859  4780 leader_election.cc:343] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.452700  5091 ts_tablet_manager.cc:1403] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.001s
I20260709 12:29:16.453943  5125 consensus_queue.cc:1048] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.454258  5086 ts_tablet_manager.cc:1434] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:29:16.455628  5086 tablet_bootstrap.cc:492] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:16.456199  5091 raft_consensus.cc:359] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.456987  5091 raft_consensus.cc:385] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.457314  5091 raft_consensus.cc:740] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.458223  5091 consensus_queue.cc:260] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.423404  5003 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:16.466446  5205 raft_consensus.cc:2804] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.467032  5205 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.467483  5205 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.474067  5212 raft_consensus.cc:493] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.475448  5082 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:16.474852  5212 raft_consensus.cc:515] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.476125  5082 ts_tablet_manager.cc:1403] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.177s	user 0.011s	sys 0.007s
I20260709 12:29:16.470754  4929 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:16.484153  5091 ts_tablet_manager.cc:1434] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:29:16.487345  5082 raft_consensus.cc:359] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.488108  5082 raft_consensus.cc:385] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.488413  5082 raft_consensus.cc:740] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.488897  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:16.489082  5082 consensus_queue.cc:260] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.489372  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:16.490043  4888 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.491328  5212 leader_election.cc:290] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.492062  5210 raft_consensus.cc:2804] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.492551  5210 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.493012  5210 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.495393  5125 consensus_queue.cc:1048] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.489552  4889 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.496805  5086 tablet_bootstrap.cc:654] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.441648  5157 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.505395  5091 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:16.469549  5003 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 45251.8us
I20260709 12:29:16.503309  5205 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.509817  5157 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.506966  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:16.508075  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 45085.8us
I20260709 12:29:16.513476  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:16.514695  4855 leader_election.cc:343] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.515792  5205 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:16.515923  4855 leader_election.cc:343] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.517409  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.518076  4815 raft_consensus.cc:2468] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:16.519518  4888 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:16.521085  4929 leader_election.cc:304] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:16.504134  4889 raft_consensus.cc:2468] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:16.524662  5151 raft_consensus.cc:515] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.525723  5082 ts_tablet_manager.cc:1434] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 12:29:16.527930  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:16.528910  4888 raft_consensus.cc:3060] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.526758  5082 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:16.535982  5083 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:16.536769  5083 ts_tablet_manager.cc:1403] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.232s	user 0.021s	sys 0.002s
I20260709 12:29:16.533728  4778 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:16.539033  5125 raft_consensus.cc:2804] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.539538  5125 raft_consensus.cc:697] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:16.540517  5125 consensus_queue.cc:237] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.547178  5108 consensus_queue.cc:1048] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:16.507577  5116 consensus_queue.cc:1048] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:29:16.549206  5001 leader_election.cc:304] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:16.550143  5003 leader_election.cc:343] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.550637  5003 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
W20260709 12:29:16.551815  5003 leader_election.cc:343] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.551505  4711 catalog_manager.cc:5697] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:16.552417  5090 raft_consensus.cc:2749] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.551419  5205 raft_consensus.cc:2804] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.558439  5205 raft_consensus.cc:697] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:16.559798  5205 consensus_queue.cc:237] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.561528  5091 tablet_bootstrap.cc:654] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.549450  5211 raft_consensus.cc:2804] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.565117  5211 raft_consensus.cc:493] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.565493  5211 raft_consensus.cc:3060] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.553118  5149 raft_consensus.cc:493] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.566440  5210 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.567239  5149 raft_consensus.cc:515] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.569278  5149 leader_election.cc:290] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.570708  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:16.571098  5082 tablet_bootstrap.cc:654] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.554929  4711 catalog_manager.cc:5697] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:16.555879  5003 leader_election.cc:304] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
W20260709 12:29:16.573418  4929 leader_election.cc:343] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.573894  5090 raft_consensus.cc:2804] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.574378  5090 raft_consensus.cc:493] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.574754  5090 raft_consensus.cc:3060] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.574555  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.576404  5116 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.557580  5083 raft_consensus.cc:359] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.564550  4888 raft_consensus.cc:2468] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:16.554486  5151 leader_election.cc:290] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.577248  5083 raft_consensus.cc:385] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.589986  5083 raft_consensus.cc:740] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.597075  5083 consensus_queue.cc:260] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.603006  5083 ts_tablet_manager.cc:1434] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.002s
I20260709 12:29:16.604290  5083 tablet_bootstrap.cc:492] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:16.610773  5220 raft_consensus.cc:493] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.611464  5220 raft_consensus.cc:515] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.614099  5220 leader_election.cc:290] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.614336  4887 raft_consensus.cc:3060] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.576915  5116 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.617558  5116 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.589193  5210 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.617982  5155 consensus_queue.cc:1048] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:29:16.624670  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:16.626674  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.659013  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c15fed052da4a3392865a529df5eda0" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:16.657250  4929 leader_election.cc:343] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:16.660701  4855 leader_election.cc:343] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.644558  4927 leader_election.cc:304] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.663036  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c15fed052da4a3392865a529df5eda0" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.678766  5155 raft_consensus.cc:493] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.697439  5155 raft_consensus.cc:515] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.699774  5155 leader_election.cc:290] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:16.699663  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.700827  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.701758  5037 raft_consensus.cc:2468] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:16.687088  5149 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.662528  5211 raft_consensus.cc:515] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.704411  5149 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.705297  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:16.706089  5211 leader_election.cc:290] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.706442  5149 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:16.706511  4963 raft_consensus.cc:2393] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:16.709439  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:16.710080  4815 raft_consensus.cc:3060] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.711791  5086 tablet_bootstrap.cc:492] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:16.712620  4813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:16.718137  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.719341  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:16.720052  5032 raft_consensus.cc:3060] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.720728  5036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54153631246c4cc68f62a6d2724ab306" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.721526  5029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:16.685030  5210 raft_consensus.cc:2804] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.724540  5210 raft_consensus.cc:697] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:16.724745  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.725483  5030 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:16.723802  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:16.725783  5210 consensus_queue.cc:237] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.726758  5031 raft_consensus.cc:2393] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:16.728392  5086 ts_tablet_manager.cc:1403] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.273s	user 0.016s	sys 0.019s
I20260709 12:29:16.741883  5091 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:16.742518  5091 ts_tablet_manager.cc:1403] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.238s	user 0.012s	sys 0.025s
I20260709 12:29:16.747008  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.756724  5037 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:16.758368  5083 tablet_bootstrap.cc:654] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:16.695086  5001 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 76977.2us
I20260709 12:29:16.760918  5086 raft_consensus.cc:359] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.761819  5086 raft_consensus.cc:385] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.762588  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:16.763032  5086 raft_consensus.cc:740] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.763381  4889 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:16.763737  5086 consensus_queue.cc:260] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.764206  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.766322  5082 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:16.766240  5086 ts_tablet_manager.cc:1434] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:29:16.766891  5082 ts_tablet_manager.cc:1403] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.240s	user 0.006s	sys 0.014s
I20260709 12:29:16.767445  5086 tablet_bootstrap.cc:492] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
W20260709 12:29:16.739548  5003 leader_election.cc:343] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:16.769102  4855 leader_election.cc:343] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.770839  5228 consensus_queue.cc:1048] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:16.771199  5082 raft_consensus.cc:359] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:16.772993  4780 leader_election.cc:343] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.773478  5091 raft_consensus.cc:359] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.779109  5091 raft_consensus.cc:385] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.779392  5091 raft_consensus.cc:740] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.777514  5082 raft_consensus.cc:385] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.780442  5082 raft_consensus.cc:740] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.780014  5091 consensus_queue.cc:260] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.782838  5082 consensus_queue.cc:260] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:16.787895  5091 ts_tablet_manager.cc:1434] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.003s
I20260709 12:29:16.749884  5090 raft_consensus.cc:515] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.789956  5091 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:16.789995  5082 ts_tablet_manager.cc:1434] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.007s
I20260709 12:29:16.803908  5091 tablet_bootstrap.cc:654] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.804742  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:16.807216  5082 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:16.815101  5090 leader_election.cc:290] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.811823  5125 consensus_queue.cc:1048] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.812788  5157 raft_consensus.cc:493] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:16.817165  5157 raft_consensus.cc:515] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.780913  4855 leader_election.cc:304] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
W20260709 12:29:16.819347  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 50290.1us
W20260709 12:29:16.785856  5003 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 47812.1us
I20260709 12:29:16.820515  5212 raft_consensus.cc:2749] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:16.821189  5003 leader_election.cc:343] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.823081  5032 raft_consensus.cc:2468] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:16.822801  4713 catalog_manager.cc:5697] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:16.819343  5233 consensus_queue.cc:1048] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:29:16.826407  4929 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:16.827665  4929 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:16.815850  5090 consensus_queue.cc:1048] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
W20260709 12:29:16.830427  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:16.830327  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:16.832084  4815 raft_consensus.cc:2468] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:16.832885  4855 leader_election.cc:304] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:16.833765  5212 raft_consensus.cc:2804] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.834147  5212 raft_consensus.cc:493] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.834216  4855 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:16.834573  5212 raft_consensus.cc:3060] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.811062  4709 catalog_manager.cc:5697] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:16.808652  5001 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:16.816792  5035 raft_consensus.cc:2393] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:16.838300  5220 raft_consensus.cc:2804] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.838699  5220 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.839068  5220 raft_consensus.cc:3060] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:16.838721  4780 leader_election.cc:343] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.839346  5157 leader_election.cc:290] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:16.851517  4780 leader_election.cc:304] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:16.849640  5090 raft_consensus.cc:2804] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:29:16.860450  5003 leader_election.cc:343] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.849844  5151 raft_consensus.cc:2804] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:16.862473  4929 leader_election.cc:304] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:16.853228  5211 raft_consensus.cc:2749] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.863708  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c5a10b9463451aaf68ce652335418b" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:16.855357  5155 raft_consensus.cc:2749] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.880673  5086 tablet_bootstrap.cc:654] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:16.872610  5090 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.872839  5151 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:16.883749  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:16.887490  4964 raft_consensus.cc:2393] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:16.853981  5233 consensus_queue.cc:1048] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:29:16.897333  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:16.890763  5090 raft_consensus.cc:3060] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.895181  5211 raft_consensus.cc:2804] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:16.899838  5211 raft_consensus.cc:697] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:16.891806  5151 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:16.899992  5083 tablet_bootstrap.cc:492] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:16.874938  5003 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:16.901032  5211 consensus_queue.cc:237] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:16.904395  5205 raft_consensus.cc:2749] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.906628  5205 consensus_queue.cc:1048] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.908391  4711 catalog_manager.cc:5697] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:16.909907  4780 consensus_peers.cc:597] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.910609  5003 leader_election.cc:304] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:16.913796  5140 raft_consensus.cc:2749] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:16.913609  5233 consensus_queue.cc:1048] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.919319  5083 ts_tablet_manager.cc:1403] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.315s	user 0.027s	sys 0.011s
I20260709 12:29:16.908468  5212 raft_consensus.cc:515] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:16.953694  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
W20260709 12:29:16.956224  4855 leader_election.cc:343] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:16.960898  5082 tablet_bootstrap.cc:654] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:16.965225  4780 consensus_peers.cc:597] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.935367  5212 leader_election.cc:290] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:16.952091  5003 leader_election.cc:343] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:16.983959  5239 log.cc:927] Time spent T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:29:16.991902  5090 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.993026  5083 raft_consensus.cc:359] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.993752  5083 raft_consensus.cc:385] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:16.993659  5125 consensus_queue.cc:1048] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:16.994163  5083 raft_consensus.cc:740] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:16.994863  5083 consensus_queue.cc:260] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:16.997264  5083 ts_tablet_manager.cc:1434] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 12:29:16.998423  5083 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.001277  5151 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.004941  5220 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.006580  5091 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.007211  5091 ts_tablet_manager.cc:1403] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.218s	user 0.027s	sys 0.033s
I20260709 12:29:17.007676  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:17.008488  5151 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.010330  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
W20260709 12:29:17.011976  4855 leader_election.cc:343] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.014223  5220 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
W20260709 12:29:17.001035  5003 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 57069.3us
I20260709 12:29:17.024423  5003 leader_election.cc:304] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.025332  5140 raft_consensus.cc:2749] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.028978  5220 raft_consensus.cc:493] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.029615  5220 raft_consensus.cc:515] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:17.030423  4929 leader_election.cc:343] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.032348  5090 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:17.033283  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:17.036576  4887 raft_consensus.cc:2393] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:17.036695  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:17.033617  5243 log.cc:927] Time spent T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:29:17.038247  4855 leader_election.cc:343] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.033407  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.039585  4815 raft_consensus.cc:2468] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:17.039672  4712 catalog_manager.cc:5697] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:17.041987  4855 leader_election.cc:304] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.035471  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.043040  5241 raft_consensus.cc:2804] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.043481  5241 raft_consensus.cc:493] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.043839  5241 raft_consensus.cc:3060] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.045065  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:17.045807  5034 raft_consensus.cc:3060] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.049918  5091 raft_consensus.cc:359] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.052356  5091 raft_consensus.cc:385] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.054270  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:17.055485  5038 raft_consensus.cc:2393] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:17.057132  5090 consensus_queue.cc:1048] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.058226  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:17.053609  5091 raft_consensus.cc:740] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.063122  4855 leader_election.cc:304] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:17.063700  5034 raft_consensus.cc:2468] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:17.064853  5212 raft_consensus.cc:2749] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.064832  4855 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:17.060691  5091 consensus_queue.cc:260] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.069798  5212 raft_consensus.cc:2749] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.070830  5091 ts_tablet_manager.cc:1434] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.063s	user 0.010s	sys 0.000s
I20260709 12:29:17.071995  5091 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.072369  5086 tablet_bootstrap.cc:492] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.072973  5086 ts_tablet_manager.cc:1403] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.306s	user 0.012s	sys 0.006s
I20260709 12:29:17.074512  5083 tablet_bootstrap.cc:654] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.076704  5086 raft_consensus.cc:359] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.077565  5086 raft_consensus.cc:385] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.077899  5086 raft_consensus.cc:740] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.066666  5220 leader_election.cc:290] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:17.087311  5108 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.087984  5108 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.090197  5108 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:17.093437  5086 consensus_queue.cc:260] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.082598  5090 raft_consensus.cc:493] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.097851  5090 raft_consensus.cc:515] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.100466  5090 leader_election.cc:290] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:17.102288  5151 raft_consensus.cc:493] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.103065  5151 raft_consensus.cc:515] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.106179  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:17.106581  5151 leader_election.cc:290] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:17.120764  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.121798  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:17.122233  5038 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:17.121412  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7909fe384cf84b089c40d28a15fd8869" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:17.124419  5001 leader_election.cc:343] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.124105  5241 raft_consensus.cc:515] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.127354  4780 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:17.128039  5241 leader_election.cc:290] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:17.128337  5108 raft_consensus.cc:2804] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.128870  5108 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.129220  5108 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:17.130437  4927 leader_election.cc:343] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.131578  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:17.120604  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.133148  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
W20260709 12:29:17.133296  4855 leader_election.cc:343] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.133591  4963 raft_consensus.cc:2393] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:17.133802  4811 raft_consensus.cc:3060] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.139599  5086 ts_tablet_manager.cc:1434] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 12:29:17.140856  5086 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.141894  4855 leader_election.cc:304] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:17.142647  4811 raft_consensus.cc:2468] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:17.143435  5241 raft_consensus.cc:2804] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:17.143930  5241 raft_consensus.cc:697] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:17.144786  5241 consensus_queue.cc:237] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.148586  5082 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:17.149062  5082 ts_tablet_manager.cc:1403] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.342s	user 0.014s	sys 0.013s
I20260709 12:29:17.150251  5086 tablet_bootstrap.cc:654] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.152206  4929 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:17.152642  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.153108  4855 leader_election.cc:304] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.155419  5211 raft_consensus.cc:2749] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.157557  5220 raft_consensus.cc:2804] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:29:17.158883  5003 leader_election.cc:343] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.159379  5003 leader_election.cc:304] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.160166  5140 raft_consensus.cc:2749] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.161105  5220 raft_consensus.cc:697] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:17.162146  5220 consensus_queue.cc:237] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.175010  5091 tablet_bootstrap.cc:654] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.167308  5083 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.184207  5083 ts_tablet_manager.cc:1403] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.186s	user 0.008s	sys 0.004s
I20260709 12:29:17.184646  5086 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.185410  5086 ts_tablet_manager.cc:1403] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.004s
I20260709 12:29:17.187115  5083 raft_consensus.cc:359] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.187965  5083 raft_consensus.cc:385] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.188306  5083 raft_consensus.cc:740] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.188330  5086 raft_consensus.cc:359] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:17.168048  5003 leader_election.cc:343] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.189182  5086 raft_consensus.cc:385] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.189549  5086 raft_consensus.cc:740] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.164956  5108 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.191804  5108 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:17.191884  5083 consensus_queue.cc:260] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.194553  5083 ts_tablet_manager.cc:1434] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:29:17.195565  5083 tablet_bootstrap.cc:492] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.196978  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:17.197934  5038 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.176456  5082 raft_consensus.cc:359] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.199705  5082 raft_consensus.cc:385] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.189353  5003 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.200101  5082 raft_consensus.cc:740] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.200834  5090 raft_consensus.cc:2749] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.201056  5082 consensus_queue.cc:260] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.202558  5083 tablet_bootstrap.cc:654] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.204686  5082 ts_tablet_manager.cc:1434] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 12:29:17.205734  4963 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.211478  5082 tablet_bootstrap.cc:492] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:17.220359  5151 consensus_queue.cc:1048] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:17.225716  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:17.226874  4962 raft_consensus.cc:2393] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:17.223824  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7909fe384cf84b089c40d28a15fd8869" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.227692  5212 raft_consensus.cc:493] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.227988  5034 raft_consensus.cc:2468] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:17.228205  5212 raft_consensus.cc:515] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.229384  4929 leader_election.cc:304] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.230284  5211 raft_consensus.cc:2804] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.230388  5212 leader_election.cc:290] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.230788  5211 raft_consensus.cc:493] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.231321  5211 raft_consensus.cc:3060] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.232548  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:17.234143  4855 leader_election.cc:343] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.232589  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
W20260709 12:29:17.236202  4855 leader_election.cc:343] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.236670  4855 leader_election.cc:304] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:17.237875  5212 raft_consensus.cc:2749] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.225080  5086 consensus_queue.cc:260] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.248811  5038 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:17.250571  4780 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.252900  5108 raft_consensus.cc:2804] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:17.253854  5108 raft_consensus.cc:697] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:17.254959  5108 consensus_queue.cc:237] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.256745  5090 consensus_queue.cc:1048] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.262296  5209 consensus_queue.cc:1048] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:17.271046  5086 ts_tablet_manager.cc:1434] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.001s
I20260709 12:29:17.272141  5086 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.273252  5091 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.273751  5091 ts_tablet_manager.cc:1403] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.202s	user 0.020s	sys 0.009s
I20260709 12:29:17.280053  4713 catalog_manager.cc:5697] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:17.287011  4713 catalog_manager.cc:5697] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:17.290097  5211 raft_consensus.cc:515] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.293867  5211 leader_election.cc:290] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:17.297523  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7909fe384cf84b089c40d28a15fd8869" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:17.302453  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7909fe384cf84b089c40d28a15fd8869" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:17.303248  5082 tablet_bootstrap.cc:654] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.307535  5038 raft_consensus.cc:3060] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:17.308168  4927 leader_election.cc:343] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.306425  5090 raft_consensus.cc:493] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.309762  5090 raft_consensus.cc:515] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.311059  5091 raft_consensus.cc:359] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.311972  5091 raft_consensus.cc:385] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.312336  5091 raft_consensus.cc:740] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.312819  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.313175  5091 consensus_queue.cc:260] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.317394  5091 ts_tablet_manager.cc:1434] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 12:29:17.319713  5086 tablet_bootstrap.cc:654] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.315660  5090 leader_election.cc:290] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.322933  5125 raft_consensus.cc:493] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:17.329522  5003 leader_election.cc:343] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.333406  5083 tablet_bootstrap.cc:492] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.333683  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.333978  5083 ts_tablet_manager.cc:1403] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.139s	user 0.024s	sys 0.024s
I20260709 12:29:17.334336  4811 raft_consensus.cc:2468] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:17.337055  5083 raft_consensus.cc:359] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.324743  5125 raft_consensus.cc:515] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.337836  5083 raft_consensus.cc:385] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.338176  5083 raft_consensus.cc:740] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.340098  5083 consensus_queue.cc:260] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.341065  5209 consensus_queue.cc:1048] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:17.341122  5091 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.344637  5083 ts_tablet_manager.cc:1434] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:29:17.345989  5083 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.352105  5083 tablet_bootstrap.cc:654] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.358647  4713 catalog_manager.cc:5697] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:17.369541  5125 leader_election.cc:290] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:17.372098  5086 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.372694  5086 ts_tablet_manager.cc:1403] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.101s	user 0.014s	sys 0.001s
I20260709 12:29:17.374537  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.375384  5032 raft_consensus.cc:2393] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:17.376502  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a7ec1dbc9448afaffb4f4291fbabed" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:17.357753  5003 leader_election.cc:304] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.380285  5090 raft_consensus.cc:2804] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.380726  5090 raft_consensus.cc:493] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.381057  5090 raft_consensus.cc:3060] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.382598  5038 raft_consensus.cc:2468] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:17.383237  4778 leader_election.cc:304] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.384517  5125 raft_consensus.cc:3060] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.393929  4929 leader_election.cc:304] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.395251  5211 raft_consensus.cc:2804] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:17.395784  5211 raft_consensus.cc:697] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:17.396699  5211 consensus_queue.cc:237] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.401691  5125 raft_consensus.cc:2749] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.399747  5086 raft_consensus.cc:359] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.407956  5086 raft_consensus.cc:385] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.408526  5086 raft_consensus.cc:740] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.415587  5086 consensus_queue.cc:260] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.418720  5082 tablet_bootstrap.cc:492] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:17.419275  5082 ts_tablet_manager.cc:1403] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.208s	user 0.020s	sys 0.017s
I20260709 12:29:17.425081  5086 ts_tablet_manager.cc:1434] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 12:29:17.426380  5086 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.416031  5090 raft_consensus.cc:515] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.428771  5211 consensus_queue.cc:1048] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.437039  5090 leader_election.cc:290] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.437534  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:17.437862  5083 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.438606  5083 ts_tablet_manager.cc:1403] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.093s	user 0.020s	sys 0.032s
W20260709 12:29:17.439639  5003 leader_election.cc:343] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.439816  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:17.440511  4815 raft_consensus.cc:3060] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.442093  5083 raft_consensus.cc:359] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.444931  5083 raft_consensus.cc:385] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.445273  5083 raft_consensus.cc:740] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.445976  5086 tablet_bootstrap.cc:654] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.447294  5083 consensus_queue.cc:260] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.451381  4815 raft_consensus.cc:2468] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:17.439309  5091 tablet_bootstrap.cc:654] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.443622  5082 raft_consensus.cc:359] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.458966  5082 raft_consensus.cc:385] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.459260  5082 raft_consensus.cc:740] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.460438  4709 catalog_manager.cc:5697] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:17.464761  4963 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.464439  5082 consensus_queue.cc:260] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.462453  5086 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.466939  5086 ts_tablet_manager.cc:1403] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.041s	user 0.003s	sys 0.015s
I20260709 12:29:17.469601  5083 ts_tablet_manager.cc:1434] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 12:29:17.470779  5083 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.474638  5082 ts_tablet_manager.cc:1434] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:29:17.475783  5082 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:17.480706  5003 leader_election.cc:304] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.482282  5090 raft_consensus.cc:2804] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:17.482980  5090 raft_consensus.cc:697] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:17.484514  5082 tablet_bootstrap.cc:654] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.484663  5147 raft_consensus.cc:493] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.484149  5090 consensus_queue.cc:237] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.485265  5147 raft_consensus.cc:515] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.492851  5147 leader_election.cc:290] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:17.493011  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747d7984e025401681cfe15c121b8a64" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.493959  5083 tablet_bootstrap.cc:654] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.496111  5091 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.496659  5091 ts_tablet_manager.cc:1403] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.156s	user 0.003s	sys 0.013s
I20260709 12:29:17.499433  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747d7984e025401681cfe15c121b8a64" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:17.503328  5003 leader_election.cc:343] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:17.504467  5001 leader_election.cc:343] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.509016  5086 raft_consensus.cc:359] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.505062  5001 leader_election.cc:304] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.509897  5086 raft_consensus.cc:385] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.510694  5086 raft_consensus.cc:740] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.511452  5086 consensus_queue.cc:260] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.511437  5147 raft_consensus.cc:2749] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.520434  4962 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.523936  5082 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:17.524533  5082 ts_tablet_manager.cc:1403] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.011s
I20260709 12:29:17.524616  5091 raft_consensus.cc:359] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.525549  5091 raft_consensus.cc:385] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.526013  5091 raft_consensus.cc:740] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.527102  5091 consensus_queue.cc:260] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.530113  5082 raft_consensus.cc:359] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.533178  5082 raft_consensus.cc:385] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.533520  5082 raft_consensus.cc:740] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.534801  5091 ts_tablet_manager.cc:1434] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.002s
I20260709 12:29:17.535405  5082 consensus_queue.cc:260] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.562873  5082 ts_tablet_manager.cc:1434] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.002s
I20260709 12:29:17.563635  5083 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.564467  5083 ts_tablet_manager.cc:1403] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.007s
I20260709 12:29:17.565388  5091 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.536607  5086 ts_tablet_manager.cc:1434] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 12:29:17.563915  5082 tablet_bootstrap.cc:492] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:17.568709  5086 tablet_bootstrap.cc:492] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.574954  5082 tablet_bootstrap.cc:654] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.575389  5091 tablet_bootstrap.cc:654] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.575529  5086 tablet_bootstrap.cc:654] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.586045  4713 catalog_manager.cc:5697] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:17.586779  5090 raft_consensus.cc:493] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.588913  5090 raft_consensus.cc:515] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.589524  5086 tablet_bootstrap.cc:492] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.590095  5086 ts_tablet_manager.cc:1403] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.001s
I20260709 12:29:17.591018  5090 leader_election.cc:290] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:17.588166  5091 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.591787  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0799f16a2c4d4001826b14d3b2f5f42d" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:17.592185  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0799f16a2c4d4001826b14d3b2f5f42d" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.588092  5220 raft_consensus.cc:493] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.593127  5220 raft_consensus.cc:515] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:17.593565  5003 leader_election.cc:343] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.594033  5083 raft_consensus.cc:359] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.594842  5083 raft_consensus.cc:385] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.595235  5220 leader_election.cc:290] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.595314  5083 raft_consensus.cc:740] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.594847  5082 tablet_bootstrap.cc:492] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:17.597313  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747d7984e025401681cfe15c121b8a64" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
W20260709 12:29:17.597985  5001 leader_election.cc:343] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.598076  5032 raft_consensus.cc:2468] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:17.598575  5001 leader_election.cc:304] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.599501  5090 raft_consensus.cc:2749] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.596388  5083 consensus_queue.cc:260] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.596781  5082 ts_tablet_manager.cc:1403] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.005s
I20260709 12:29:17.602053  5083 ts_tablet_manager.cc:1434] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:29:17.603188  5083 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.604460  5082 raft_consensus.cc:359] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.604889  4855 leader_election.cc:304] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:17.605288  5082 raft_consensus.cc:385] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.605662  5082 raft_consensus.cc:740] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.606288  5220 raft_consensus.cc:2804] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.606455  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747d7984e025401681cfe15c121b8a64" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.591791  5091 ts_tablet_manager.cc:1403] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.006s
W20260709 12:29:17.608332  4855 leader_election.cc:343] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.606747  5220 raft_consensus.cc:493] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.609452  5220 raft_consensus.cc:3060] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.610956  5083 tablet_bootstrap.cc:654] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.612311  5091 raft_consensus.cc:359] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.613018  5091 raft_consensus.cc:385] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.613320  5091 raft_consensus.cc:740] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.614140  5091 consensus_queue.cc:260] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.616336  5091 ts_tablet_manager.cc:1434] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:17.617473  5091 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.621829  5086 raft_consensus.cc:359] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.622959  5086 raft_consensus.cc:385] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.623658  5091 tablet_bootstrap.cc:654] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.629835  5220 raft_consensus.cc:515] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.623359  5086 raft_consensus.cc:740] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.643987  5086 consensus_queue.cc:260] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.632287  5082 consensus_queue.cc:260] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.644956  5091 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
W20260709 12:29:17.645128  4855 consensus_peers.cc:597] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.645612  5091 ts_tablet_manager.cc:1403] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.006s
I20260709 12:29:17.646443  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747d7984e025401681cfe15c121b8a64" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:17.647193  5032 raft_consensus.cc:3060] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.648947  5086 ts_tablet_manager.cc:1434] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.002s
I20260709 12:29:17.646538  5082 ts_tablet_manager.cc:1434] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 12:29:17.650045  5082 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:17.651111  5086 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.653175  5091 raft_consensus.cc:359] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.655978  5091 raft_consensus.cc:385] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.656912  5091 raft_consensus.cc:740] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.658337  5091 consensus_queue.cc:260] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.660974  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747d7984e025401681cfe15c121b8a64" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:17.661763  4815 raft_consensus.cc:3060] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.662668  5091 ts_tablet_manager.cc:1434] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:29:17.664878  5091 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.668963  5082 tablet_bootstrap.cc:654] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.669684  4815 raft_consensus.cc:2468] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:17.671916  4855 leader_election.cc:304] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:17.672549  5086 tablet_bootstrap.cc:654] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.672961  5241 raft_consensus.cc:2804] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:17.673520  5241 raft_consensus.cc:697] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:17.674422  5241 consensus_queue.cc:237] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.675729  5091 tablet_bootstrap.cc:654] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.676630  5211 raft_consensus.cc:493] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.677268  5211 raft_consensus.cc:515] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.679324  5220 leader_election.cc:290] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.670454  5032 raft_consensus.cc:2468] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:17.682715  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.683243  5083 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.683749  5083 ts_tablet_manager.cc:1403] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.081s	user 0.022s	sys 0.026s
I20260709 12:29:17.679944  5211 leader_election.cc:290] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.686347  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d588e01b9943119c0c0427f2bc7b6f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.687054  5083 raft_consensus.cc:359] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.690654  4815 raft_consensus.cc:2393] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:17.690656  5083 raft_consensus.cc:385] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.691329  5083 raft_consensus.cc:740] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.692102  5083 consensus_queue.cc:260] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.692548  4929 leader_election.cc:304] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:17.693965  5083 ts_tablet_manager.cc:1434] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:29:17.694437  5211 raft_consensus.cc:3060] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.695048  5083 tablet_bootstrap.cc:492] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.700500  5241 consensus_queue.cc:1048] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:17.711547  5212 consensus_queue.cc:1048] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.721056  5083 tablet_bootstrap.cc:654] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.727077  4710 catalog_manager.cc:5697] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:17.729835  5211 raft_consensus.cc:2749] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.739805  5091 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.764694  5091 ts_tablet_manager.cc:1403] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.015s
I20260709 12:29:17.767901  5091 raft_consensus.cc:359] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.768890  5091 raft_consensus.cc:385] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.769307  5091 raft_consensus.cc:740] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.770031  5086 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.770651  5086 ts_tablet_manager.cc:1403] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.120s	user 0.019s	sys 0.020s
I20260709 12:29:17.770473  5091 consensus_queue.cc:260] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.778208  5086 raft_consensus.cc:359] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.779017  5086 raft_consensus.cc:385] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.779373  5086 raft_consensus.cc:740] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.780222  5086 consensus_queue.cc:260] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.782823  5091 ts_tablet_manager.cc:1434] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:29:17.748723  5083 tablet_bootstrap.cc:492] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.784329  5091 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.784582  5083 ts_tablet_manager.cc:1403] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.009s
I20260709 12:29:17.785399  5290 consensus_queue.cc:1048] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.791558  5091 tablet_bootstrap.cc:654] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.793277  5086 ts_tablet_manager.cc:1434] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:29:17.795560  5086 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.803207  5086 tablet_bootstrap.cc:654] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.803524  5091 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.806542  5091 ts_tablet_manager.cc:1403] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.001s
I20260709 12:29:17.809587  5091 raft_consensus.cc:359] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.810316  5091 raft_consensus.cc:385] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.810592  5091 raft_consensus.cc:740] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.811194  5091 consensus_queue.cc:260] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.804358  5083 raft_consensus.cc:359] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.812104  5083 raft_consensus.cc:385] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.812434  5083 raft_consensus.cc:740] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.813177  5083 consensus_queue.cc:260] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.815469  5083 ts_tablet_manager.cc:1434] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 12:29:17.816485  5083 tablet_bootstrap.cc:492] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.822520  5083 tablet_bootstrap.cc:654] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.825004  5091 ts_tablet_manager.cc:1434] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:29:17.825793  5082 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:17.826133  5091 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.826285  5082 ts_tablet_manager.cc:1403] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.176s	user 0.024s	sys 0.023s
I20260709 12:29:17.830917  5082 raft_consensus.cc:359] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.831727  5082 raft_consensus.cc:385] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.831998  5082 raft_consensus.cc:740] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.832829  5082 consensus_queue.cc:260] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.834687  5108 consensus_queue.cc:1048] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.836083  5082 ts_tablet_manager.cc:1434] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:29:17.837218  5082 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:17.843115  5091 tablet_bootstrap.cc:654] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.845508  5082 tablet_bootstrap.cc:654] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.850315  5229 raft_consensus.cc:493] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.851111  5229 raft_consensus.cc:515] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.854269  5229 leader_election.cc:290] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:17.855537  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.856976  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:17.858736  5001 leader_election.cc:343] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:17.859431  5003 leader_election.cc:343] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.859889  5003 leader_election.cc:304] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.861296  5290 raft_consensus.cc:493] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.861832  5290 raft_consensus.cc:515] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.864147  5083 tablet_bootstrap.cc:492] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:17.864827  5083 ts_tablet_manager.cc:1403] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.002s
I20260709 12:29:17.867097  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7909fe384cf84b089c40d28a15fd8869" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.868341  5083 raft_consensus.cc:359] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.869310  5083 raft_consensus.cc:385] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.869621  5083 raft_consensus.cc:740] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.870260  5147 raft_consensus.cc:2749] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.879451  5082 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:17.872705  5091 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:17.883260  5299 consensus_queue.cc:1048] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:29:17.870489  5083 consensus_queue.cc:260] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:17.880259  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7909fe384cf84b089c40d28a15fd8869" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.871446  5290 leader_election.cc:290] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:17.887646  5147 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.888121  5034 raft_consensus.cc:2393] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:17.888561  5083 ts_tablet_manager.cc:1434] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:29:17.888250  5147 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.889572  5083 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:17.889827  4855 leader_election.cc:304] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:17.891180  5147 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:17.892004  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.892382  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.885838  5091 ts_tablet_manager.cc:1403] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.001s
I20260709 12:29:17.906075  5147 consensus_queue.cc:1048] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:17.906271  5003 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:17.907039  5003 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:17.907459  5003 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:17.910940  5147 raft_consensus.cc:2749] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.892591  5290 raft_consensus.cc:3060] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.892576  5299 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:17.883792  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:17.898351  5083 tablet_bootstrap.cc:654] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.907554  5091 raft_consensus.cc:359] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.921411  5091 raft_consensus.cc:385] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.885838  5082 ts_tablet_manager.cc:1403] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.049s	user 0.027s	sys 0.001s
I20260709 12:29:17.921782  5091 raft_consensus.cc:740] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.922540  5091 consensus_queue.cc:260] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.924933  5082 raft_consensus.cc:359] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.925715  5082 raft_consensus.cc:385] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.926011  5082 raft_consensus.cc:740] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.926717  5082 consensus_queue.cc:260] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.932605  5082 ts_tablet_manager.cc:1434] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:29:17.937064  5091 ts_tablet_manager.cc:1434] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.002s
I20260709 12:29:17.938597  5091 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:17.946861  5086 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:17.947794  5086 ts_tablet_manager.cc:1403] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.152s	user 0.013s	sys 0.035s
I20260709 12:29:17.948280  5108 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.948843  5108 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:17.949607  5082 tablet_bootstrap.cc:492] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:17.950896  5108 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:17.951606  5086 raft_consensus.cc:359] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.952672  5086 raft_consensus.cc:385] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:17.953039  5086 raft_consensus.cc:740] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:17.954041  5086 consensus_queue.cc:260] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.940264  5125 raft_consensus.cc:493] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.955741  5125 raft_consensus.cc:515] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.956033  5211 consensus_queue.cc:1048] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:17.957685  5086 ts_tablet_manager.cc:1434] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:29:17.958176  5211 raft_consensus.cc:493] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.959168  5086 tablet_bootstrap.cc:492] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:17.958860  5211 raft_consensus.cc:515] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.958247  5125 leader_election.cc:290] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:17.963644  5211 leader_election.cc:290] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:17.964919  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:17.965497  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:17.965804  4963 raft_consensus.cc:2393] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:17.961902  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:17.967128  4888 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:17.968457  4778 leader_election.cc:304] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:17.921617  5299 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:17.963661  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.972954  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:17.973658  5038 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:17.973824  5032 raft_consensus.cc:2393] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:17.969539  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.970172  5125 raft_consensus.cc:2804] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.975052  5125 raft_consensus.cc:493] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.980204  5147 raft_consensus.cc:493] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.980755  5147 raft_consensus.cc:515] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:17.982872  5147 leader_election.cc:290] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:17.982937  4929 leader_election.cc:304] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:17.983783  5211 raft_consensus.cc:2749] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:17.984907  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:17.986095  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:17.977425  5125 raft_consensus.cc:3060] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.986940  4888 raft_consensus.cc:2468] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:17.989846  5001 leader_election.cc:304] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:17.991675  5229 raft_consensus.cc:2804] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:17.992218  5229 raft_consensus.cc:493] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.992673  5229 raft_consensus.cc:3060] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:17.993849  5086 tablet_bootstrap.cc:654] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:17.941008  5302 raft_consensus.cc:493] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.001127  5090 consensus_queue.cc:1048] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.009222  5125 raft_consensus.cc:515] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.017051  5125 leader_election.cc:290] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.021428  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:18.022141  4889 raft_consensus.cc:3060] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.022675  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.024987  5211 raft_consensus.cc:493] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.024597  4888 raft_consensus.cc:2468] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:18.027102  5299 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:18.027673  4778 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.028184  4780 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:18.029547  5300 raft_consensus.cc:2804] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.029987  5300 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.029639  5299 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.030380  5300 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.032618  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.033331  5032 raft_consensus.cc:3060] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.025619  5211 raft_consensus.cc:515] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.038061  5212 raft_consensus.cc:493] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.038692  5212 raft_consensus.cc:515] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.041376  5212 leader_election.cc:290] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.042603  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0799f16a2c4d4001826b14d3b2f5f42d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.043365  5038 raft_consensus.cc:2468] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:18.014125  5290 raft_consensus.cc:2749] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.046090  4855 leader_election.cc:304] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.050127  5212 raft_consensus.cc:2804] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.050621  5212 raft_consensus.cc:493] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.997756  5302 raft_consensus.cc:515] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.054396  5211 leader_election.cc:290] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.051040  5212 raft_consensus.cc:3060] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.056926  5300 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.060773  4815 raft_consensus.cc:2468] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 0.
I20260709 12:29:18.059581  4889 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:18.069566  5083 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.070057  5083 ts_tablet_manager.cc:1403] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.181s	user 0.015s	sys 0.036s
I20260709 12:29:18.073779  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c15fed052da4a3392865a529df5eda0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.074281  5091 tablet_bootstrap.cc:654] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.074515  4814 raft_consensus.cc:2468] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:18.075759  5299 raft_consensus.cc:2749] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.077524  5302 leader_election.cc:290] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.078519  5083 raft_consensus.cc:359] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.079258  5083 raft_consensus.cc:385] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.079552  5083 raft_consensus.cc:740] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.083871  5083 consensus_queue.cc:260] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.094952  5229 raft_consensus.cc:515] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.097558  5083 ts_tablet_manager.cc:1434] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 12:29:18.099272  5083 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.096771  5306 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:18.109261  5303 log.cc:927] Time spent T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:29:18.097245  4929 leader_election.cc:304] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:18.125284  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:18.126616  4814 raft_consensus.cc:3060] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:18.114161  5305 log.cc:927] Time spent T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:29:18.125708  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0799f16a2c4d4001826b14d3b2f5f42d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.112648  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.113318  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c15fed052da4a3392865a529df5eda0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.134527  5211 consensus_queue.cc:1048] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:18.135972  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.137948  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:18.160159  4887 raft_consensus.cc:3060] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.118988  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3d8ad3cb14b11aef810d11bc59682" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.163269  5300 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
W20260709 12:29:18.145447  4780 leader_election.cc:343] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:18.146270  4855 leader_election.cc:343] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.111853  4778 leader_election.cc:304] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.109139  5306 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.169179  5091 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:18.169781  5091 ts_tablet_manager.cc:1403] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.231s	user 0.024s	sys 0.013s
I20260709 12:29:18.116377  5229 leader_election.cc:290] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.154772  5307 raft_consensus.cc:2804] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.159395  5038 raft_consensus.cc:2393] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:18.139334  5083 tablet_bootstrap.cc:654] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.170271  5302 raft_consensus.cc:2804] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.170823  5307 raft_consensus.cc:493] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.128328  5032 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
W20260709 12:29:18.166841  4778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 59302.6us
I20260709 12:29:18.175876  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.173482  5307 raft_consensus.cc:3060] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.176698  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
W20260709 12:29:18.157207  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 64667.3us
I20260709 12:29:18.177205  5302 raft_consensus.cc:697] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:18.188783  5091 raft_consensus.cc:359] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.188553  5302 consensus_queue.cc:237] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.189553  5091 raft_consensus.cc:385] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.189894  5091 raft_consensus.cc:740] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.190580  5091 consensus_queue.cc:260] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.171015  5306 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.186223  5082 tablet_bootstrap.cc:654] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.186985  4886 raft_consensus.cc:2393] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:18.195973  5091 ts_tablet_manager.cc:1434] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 12:29:18.172204  4889 raft_consensus.cc:2468] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:18.200409  5091 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:18.198282  4780 leader_election.cc:343] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.209959  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:18.211320  4890 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.215720  4963 raft_consensus.cc:3060] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.202322  5307 raft_consensus.cc:515] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.224464  5307 leader_election.cc:290] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.225189  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c15fed052da4a3392865a529df5eda0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:18.225931  4886 raft_consensus.cc:3060] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.228449  4778 leader_election.cc:304] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:18.252424  5083 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.253015  5083 ts_tablet_manager.cc:1403] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.155s	user 0.014s	sys 0.011s
I20260709 12:29:18.221936  4890 raft_consensus.cc:2468] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:18.257155  5091 tablet_bootstrap.cc:654] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.261020  5086 tablet_bootstrap.cc:492] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:18.261745  5086 ts_tablet_manager.cc:1403] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.303s	user 0.010s	sys 0.025s
I20260709 12:29:18.267985  5290 consensus_queue.cc:1048] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:29:18.272186  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.295840  5229 raft_consensus.cc:493] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.296455  5229 raft_consensus.cc:515] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.298128  5083 raft_consensus.cc:359] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.298858  5083 raft_consensus.cc:385] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.299113  5083 raft_consensus.cc:740] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.299268  5229 leader_election.cc:290] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:18.293344  5299 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.299882  5083 consensus_queue.cc:260] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.300808  5220 consensus_queue.cc:1048] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:18.301690  5083 ts_tablet_manager.cc:1434] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:29:18.296028  5034 raft_consensus.cc:2393] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:18.302762  5083 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.300554  5299 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.286378  4709 catalog_manager.cc:5697] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.294938  5086 raft_consensus.cc:359] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.305801  5086 raft_consensus.cc:385] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.306131  5086 raft_consensus.cc:740] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.311811  4885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.313457  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:18.313827  5310 log.cc:927] Time spent T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:29:18.314042  5299 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.315696  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
W20260709 12:29:18.315927  4780 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.316587  5034 raft_consensus.cc:2468] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
W20260709 12:29:18.318311  5001 leader_election.cc:343] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.309700  5086 consensus_queue.cc:260] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.322639  4780 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.327556  5086 ts_tablet_manager.cc:1434] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.004s
I20260709 12:29:18.323544  5299 raft_consensus.cc:2804] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.329694  5086 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:18.332150  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c15fed052da4a3392865a529df5eda0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:18.332865  4812 raft_consensus.cc:3060] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.333151  4814 raft_consensus.cc:2468] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:18.333794  5300 raft_consensus.cc:2749] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.334084  4929 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:18.335223  4778 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.335310  5003 leader_election.cc:304] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:18.336218  5298 raft_consensus.cc:2804] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.336827  5298 raft_consensus.cc:697] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:18.337760  5298 consensus_queue.cc:237] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.338907  5229 raft_consensus.cc:2804] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.339216  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.339461  5229 raft_consensus.cc:697] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:18.340446  5229 consensus_queue.cc:237] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:18.344978  5003 leader_election.cc:343] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.345584  5003 leader_election.cc:304] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:18.345778  4886 raft_consensus.cc:2468] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:18.347147  4927 leader_election.cc:304] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.347112  5090 raft_consensus.cc:2749] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.338903  5306 raft_consensus.cc:3060] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.348628  5211 raft_consensus.cc:2804] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.349208  5211 raft_consensus.cc:697] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:18.330996  5299 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.356092  5299 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.360333  4887 raft_consensus.cc:2468] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:18.358397  5212 raft_consensus.cc:515] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.390802  5212 leader_election.cc:290] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.394897  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0799f16a2c4d4001826b14d3b2f5f42d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:18.395603  4963 raft_consensus.cc:3060] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.350080  5211 consensus_queue.cc:237] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.415927  4963 raft_consensus.cc:2468] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:18.450627  5086 tablet_bootstrap.cc:654] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.458550  5309 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.459131  5309 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.461416  5309 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:18.455416  4855 leader_election.cc:304] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.462631  5322 raft_consensus.cc:2804] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.463158  5322 raft_consensus.cc:697] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:18.465494  4812 raft_consensus.cc:2468] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:18.469288  5306 raft_consensus.cc:2749] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.467155  5322 consensus_queue.cc:237] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.472177  5083 tablet_bootstrap.cc:654] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.482226  5147 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.482784  5147 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.484915  5147 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:18.486706  5082 tablet_bootstrap.cc:492] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:18.487242  5082 ts_tablet_manager.cc:1403] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.538s	user 0.007s	sys 0.013s
I20260709 12:29:18.487428  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0799f16a2c4d4001826b14d3b2f5f42d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.488266  5038 raft_consensus.cc:3060] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.490341  5082 raft_consensus.cc:359] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.491089  5082 raft_consensus.cc:385] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.491364  5082 raft_consensus.cc:740] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.493404  5091 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:18.493783  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.494246  5091 ts_tablet_manager.cc:1403] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.294s	user 0.018s	sys 0.003s
I20260709 12:29:18.493780  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.495589  4812 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:18.497164  4929 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:18.498047  5306 raft_consensus.cc:2804] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:18.498513  5306 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.498849  5306 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.498665  5082 consensus_queue.cc:260] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.500749  5091 raft_consensus.cc:359] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.501466  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.504568  5082 ts_tablet_manager.cc:1434] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 12:29:18.505604  5082 tablet_bootstrap.cc:492] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:18.501493  5091 raft_consensus.cc:385] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.509423  5091 raft_consensus.cc:740] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.514459  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.514058  5091 consensus_queue.cc:260] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.515226  5299 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.515852  5034 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:18.516979  5091 ts_tablet_manager.cc:1434] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:29:18.518069  5299 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.519423  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.520104  5034 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:18.527356  5003 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.528777  5038 raft_consensus.cc:2468] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:18.532300  5091 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:18.537781  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
W20260709 12:29:18.539120  4780 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.529917  5034 raft_consensus.cc:2468] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:18.539741  5306 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.543687  5082 tablet_bootstrap.cc:654] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:18.543943  5003 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.544260  4780 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.545858  5299 raft_consensus.cc:2804] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.546876  5299 raft_consensus.cc:697] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:18.544554  5003 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.552490  5147 raft_consensus.cc:2749] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.553642  5299 consensus_queue.cc:237] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.557533  5083 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.558279  5083 ts_tablet_manager.cc:1403] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.256s	user 0.039s	sys 0.012s
I20260709 12:29:18.563397  5306 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:18.565884  5091 tablet_bootstrap.cc:654] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.569011  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.569907  5038 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.571455  5086 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:18.572144  5086 ts_tablet_manager.cc:1403] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.244s	user 0.012s	sys 0.003s
I20260709 12:29:18.568964  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:18.577713  4812 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.592412  5298 consensus_queue.cc:1048] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.596210  5086 raft_consensus.cc:359] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.598358  5086 raft_consensus.cc:385] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.598726  5086 raft_consensus.cc:740] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.602460  5086 consensus_queue.cc:260] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.602823  4710 catalog_manager.cc:5697] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.615314  5083 raft_consensus.cc:359] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.616353  5083 raft_consensus.cc:385] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.616709  5083 raft_consensus.cc:740] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.617405  5083 consensus_queue.cc:260] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.607837  4812 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:18.621215  4929 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:18.622210  5327 raft_consensus.cc:2804] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:18.622690  5327 raft_consensus.cc:697] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:18.622845  4713 catalog_manager.cc:5697] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:18.623607  5327 consensus_queue.cc:237] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.619825  5083 ts_tablet_manager.cc:1434] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:29:18.629068  5083 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.635757  5038 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:18.617672  5082 tablet_bootstrap.cc:492] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:18.642750  5082 ts_tablet_manager.cc:1403] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.005s
I20260709 12:29:18.638850  5091 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:18.636941  4709 catalog_manager.cc:5697] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 1 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.646392  5091 ts_tablet_manager.cc:1403] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.114s	user 0.028s	sys 0.026s
I20260709 12:29:18.648603  5082 raft_consensus.cc:359] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.650867  5082 raft_consensus.cc:385] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.651247  5082 raft_consensus.cc:740] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.652032  5083 tablet_bootstrap.cc:654] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.652130  5082 consensus_queue.cc:260] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.654548  5082 ts_tablet_manager.cc:1434] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:29:18.655568  5082 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:18.657347  5108 consensus_queue.cc:1048] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.659274  4711 catalog_manager.cc:5697] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.661091  4711 catalog_manager.cc:5697] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:18.664065  5082 tablet_bootstrap.cc:654] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.644970  5086 ts_tablet_manager.cc:1434] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:29:18.667637  5086 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:18.681401  5083 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.681967  5083 ts_tablet_manager.cc:1403] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.003s
I20260709 12:29:18.649688  5091 raft_consensus.cc:359] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.684469  5091 raft_consensus.cc:385] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.684816  5091 raft_consensus.cc:740] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.686892  5091 consensus_queue.cc:260] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.688635  5322 raft_consensus.cc:493] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.689211  5322 raft_consensus.cc:515] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.690058  5299 raft_consensus.cc:493] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.690546  5299 raft_consensus.cc:515] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.691358  5322 leader_election.cc:290] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.693285  5086 tablet_bootstrap.cc:654] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.693910  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.694190  5322 raft_consensus.cc:493] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.694690  5322 raft_consensus.cc:515] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:18.695947  4855 leader_election.cc:343] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.696812  5322 leader_election.cc:290] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.696667  4709 catalog_manager.cc:5697] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 2, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.699309  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.700001  5034 raft_consensus.cc:2468] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:18.703552  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:18.705504  4855 leader_election.cc:343] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.706231  4855 leader_election.cc:304] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.706662  5083 raft_consensus.cc:359] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.707362  5083 raft_consensus.cc:385] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.707645  5083 raft_consensus.cc:740] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.708420  5322 raft_consensus.cc:2804] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.708825  5322 raft_consensus.cc:493] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.707679  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.709172  5322 raft_consensus.cc:3060] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.709384  5038 raft_consensus.cc:2468] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:18.710662  4855 leader_election.cc:304] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.710947  5083 consensus_queue.cc:260] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.711521  5290 raft_consensus.cc:2804] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.712136  5324 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.712035  5290 raft_consensus.cc:493] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.712632  5324 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.713045  5290 raft_consensus.cc:3060] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.713287  5083 ts_tablet_manager.cc:1434] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:29:18.714313  5083 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.716503  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.717552  5324 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.718108  5324 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:18.719525  5001 leader_election.cc:343] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.720330  5083 tablet_bootstrap.cc:654] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.719861  5290 raft_consensus.cc:515] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.721954  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.723037  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.718578  5324 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.723673  5038 raft_consensus.cc:2468] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:18.725037  4780 leader_election.cc:304] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:18.725929  5324 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:18.726276  5302 raft_consensus.cc:2804] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.726737  5302 raft_consensus.cc:493] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.727113  5302 raft_consensus.cc:3060] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.727888  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
W20260709 12:29:18.732282  5003 leader_election.cc:343] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.733140  5083 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.733698  5083 ts_tablet_manager.cc:1403] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:29:18.733773  5212 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.733621  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.734488  5212 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.734905  4961 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:18.736268  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.736933  4886 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:18.736861  5083 raft_consensus.cc:359] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.737658  5083 raft_consensus.cc:385] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.737950  5083 raft_consensus.cc:740] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.738116  5003 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:18.738139  5001 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.738731  5212 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.738679  5083 consensus_queue.cc:260] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.739311  5147 raft_consensus.cc:2749] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:18.739750  4855 leader_election.cc:343] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.739811  5229 raft_consensus.cc:2804] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:18.740473  5229 raft_consensus.cc:493] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.740851  5229 raft_consensus.cc:3060] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.742285  5082 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:18.742811  5082 ts_tablet_manager.cc:1403] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.087s	user 0.011s	sys 0.002s
I20260709 12:29:18.745604  5083 ts_tablet_manager.cc:1434] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:29:18.746657  5083 tablet_bootstrap.cc:492] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.748526  5322 raft_consensus.cc:515] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.750763  5322 leader_election.cc:290] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.753775  5082 raft_consensus.cc:359] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.746987  5302 raft_consensus.cc:515] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.754491  5082 raft_consensus.cc:385] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.754873  5082 raft_consensus.cc:740] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.755429  5082 consensus_queue.cc:260] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.755719  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.756527  4961 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:18.756775  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.756708  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.757633  5082 ts_tablet_manager.cc:1434] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:29:18.758647  5083 tablet_bootstrap.cc:654] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.758618  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:18.761416  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.757592  5034 raft_consensus.cc:3060] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.762120  5032 raft_consensus.cc:3060] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.771034  5032 raft_consensus.cc:2468] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:18.772935  5034 raft_consensus.cc:2468] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:18.773944  5229 raft_consensus.cc:515] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.774627  4780 leader_election.cc:304] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:18.780803  5108 raft_consensus.cc:2804] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.777854  5091 ts_tablet_manager.cc:1434] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.131s	user 0.007s	sys 0.000s
I20260709 12:29:18.775835  5290 leader_election.cc:290] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:18.782867  5091 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:18.783229  5086 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:18.784936  5086 ts_tablet_manager.cc:1403] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.118s	user 0.009s	sys 0.007s
I20260709 12:29:18.787523  5082 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:18.791891  5298 consensus_queue.cc:1048] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.799116  5038 raft_consensus.cc:2393] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:18.783716  5108 raft_consensus.cc:697] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:18.807212  5229 leader_election.cc:290] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:18.810169  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:18.806631  4855 leader_election.cc:304] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:18.830084  4889 raft_consensus.cc:3060] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:18.831771  4780 consensus_peers.cc:597] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:18.813649  5086 raft_consensus.cc:359] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.834412  5086 raft_consensus.cc:385] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.834746  5086 raft_consensus.cc:740] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.835418  5086 consensus_queue.cc:260] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.837133  5086 ts_tablet_manager.cc:1434] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 12:29:18.806006  5299 leader_election.cc:290] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.807940  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b05cbdc6f14118b098b8dd06d859c3" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:18.806226  5108 consensus_queue.cc:237] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.802760  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:18.837770  4961 raft_consensus.cc:3060] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.837981  5034 raft_consensus.cc:3060] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.819490  5290 raft_consensus.cc:493] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.844599  5290 raft_consensus.cc:515] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.846817  5290 leader_election.cc:290] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:18.820286  5302 leader_election.cc:290] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.821790  5083 tablet_bootstrap.cc:492] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.821650  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:18.856978  5083 ts_tablet_manager.cc:1403] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.111s	user 0.020s	sys 0.031s
W20260709 12:29:18.858021  4778 leader_election.cc:343] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.858040  4961 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:18.859596  5003 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:18.860047  5083 raft_consensus.cc:359] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.860724  5083 raft_consensus.cc:385] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.860904  5083 raft_consensus.cc:740] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.861477  5083 consensus_queue.cc:260] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.823968  5327 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.809188  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:18.863355  5327 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:18.865281  4778 leader_election.cc:343] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.865622  5327 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:18.828954  5091 tablet_bootstrap.cc:654] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.830718  5212 raft_consensus.cc:2804] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.867270  5212 raft_consensus.cc:697] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:18.867825  5086 tablet_bootstrap.cc:492] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:18.868297  5212 consensus_queue.cc:237] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:18.830533  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 45974.5us
I20260709 12:29:18.871703  4855 leader_election.cc:304] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:18.873745  4855 leader_election.cc:343] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.875262  5086 tablet_bootstrap.cc:654] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.814157  5082 tablet_bootstrap.cc:654] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.875486  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.860630  5229 raft_consensus.cc:2804] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:18.876717  5229 raft_consensus.cc:697] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:18.877085  5091 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:18.878638  5091 ts_tablet_manager.cc:1403] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.001s
I20260709 12:29:18.843953  4889 raft_consensus.cc:2468] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:18.879406  5290 raft_consensus.cc:2764] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:29:18.880053  5034 raft_consensus.cc:2468] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:18.877802  5229 consensus_queue.cc:237] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.881772  5091 raft_consensus.cc:359] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.856177  5147 consensus_queue.cc:1048] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:18.871047  5083 ts_tablet_manager.cc:1434] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:29:18.883558  5083 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.887352  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:18.888629  5324 consensus_queue.cc:1048] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.879163  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:18.891371  5083 tablet_bootstrap.cc:654] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.892055  5038 raft_consensus.cc:2393] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:18.893859  4929 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:18.882503  5091 raft_consensus.cc:385] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.894583  5091 raft_consensus.cc:740] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.895401  5091 consensus_queue.cc:260] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:18.896166  5327 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.898273  5091 ts_tablet_manager.cc:1434] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:29:18.900148  5091 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:18.896631  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.904557  4812 raft_consensus.cc:2468] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:18.895678  4855 leader_election.cc:304] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:18.915745  5290 raft_consensus.cc:2804] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.916628  5290 raft_consensus.cc:697] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:18.918255  4712 catalog_manager.cc:5697] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.918588  5290 consensus_queue.cc:237] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.920972  4855 leader_election.cc:304] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:18.922039  4855 leader_election.cc:343] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:18.923264  5212 raft_consensus.cc:2804] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:18.923581  5306 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.942708  5306 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:18.943060  4710 catalog_manager.cc:5697] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.946041  5306 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:18.948930  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:18.927908  5083 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:18.929347  5091 tablet_bootstrap.cc:654] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.931180  5082 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:18.933947  5211 consensus_queue.cc:1048] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.954851  5083 ts_tablet_manager.cc:1403] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.006s
I20260709 12:29:18.954888  5082 ts_tablet_manager.cc:1403] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.168s	user 0.011s	sys 0.009s
I20260709 12:29:18.958912  5082 raft_consensus.cc:359] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.959649  5082 raft_consensus.cc:385] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.959913  5082 raft_consensus.cc:740] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.962834  5083 raft_consensus.cc:359] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.958310  5327 raft_consensus.cc:2749] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:18.966867  5083 raft_consensus.cc:385] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.967167  5082 consensus_queue.cc:260] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.969852  5086 tablet_bootstrap.cc:492] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:18.970382  5086 ts_tablet_manager.cc:1403] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.103s	user 0.028s	sys 0.004s
I20260709 12:29:18.970678  5082 ts_tablet_manager.cc:1434] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 12:29:18.973896  5082 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:18.973974  5086 raft_consensus.cc:359] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.974712  5086 raft_consensus.cc:385] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.975016  5086 raft_consensus.cc:740] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.967316  5083 raft_consensus.cc:740] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
W20260709 12:29:18.975989  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:18.976645  5083 consensus_queue.cc:260] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:18.965220  5212 raft_consensus.cc:493] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.977811  5086 consensus_queue.cc:260] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:18.978288  5212 raft_consensus.cc:3060] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.982582  5082 tablet_bootstrap.cc:654] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.986707  5083 ts_tablet_manager.cc:1434] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.031s	user 0.000s	sys 0.007s
I20260709 12:29:18.988101  5083 tablet_bootstrap.cc:492] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:18.992305  5322 consensus_queue.cc:1048] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:18.993528  5086 ts_tablet_manager.cc:1434] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 12:29:18.994696  5086 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:18.995662  5307 raft_consensus.cc:493] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.996271  5307 raft_consensus.cc:515] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.002240  5229 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.003171  5229 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.004537  5299 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.005085  5299 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.005695  5229 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:19.002435  5220 raft_consensus.cc:493] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.006815  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.007087  5220 raft_consensus.cc:515] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:19.008708  5003 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.008692  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.009406  5220 leader_election.cc:290] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:19.012344  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.013747  5229 raft_consensus.cc:493] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.014353  5229 raft_consensus.cc:515] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:19.017486  4780 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.017894  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.006834  4709 catalog_manager.cc:5697] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 0 to 2, leader changed from <none> to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:19.019464  5003 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.019994  5229 leader_election.cc:290] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:19.020642  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:19.021454  5038 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:19.023641  5091 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:19.024269  5091 ts_tablet_manager.cc:1403] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.005s
W20260709 12:29:19.024528  5003 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.024576  4780 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:19.026237  5108 raft_consensus.cc:2804] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.025102  5003 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:19.026723  5108 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.027156  5108 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.027515  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.028669  5229 raft_consensus.cc:2749] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.029830  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
W20260709 12:29:19.030112  5003 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.030462  5086 tablet_bootstrap.cc:654] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.030678  5032 raft_consensus.cc:2393] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:19.030618  5003 leader_election.cc:304] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:19.007103  5299 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.033412  5229 raft_consensus.cc:2749] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.037411  5212 raft_consensus.cc:515] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.038782  5083 tablet_bootstrap.cc:654] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.039806  5212 leader_election.cc:290] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:19.041431  5038 raft_consensus.cc:1275] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:19.043092  5327 consensus_queue.cc:1048] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:19.025095  5307 leader_election.cc:290] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.043794  5108 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.046846  5091 raft_consensus.cc:359] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.047565  5091 raft_consensus.cc:385] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.047854  5091 raft_consensus.cc:740] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.069718  5091 consensus_queue.cc:260] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:19.065470  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 76118.7us
I20260709 12:29:19.083196  5091 ts_tablet_manager.cc:1434] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.002s
I20260709 12:29:19.084443  5091 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:19.067189  5324 raft_consensus.cc:493] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.089845  5290 consensus_queue.cc:1048] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.096248  5324 raft_consensus.cc:515] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.098500  5324 leader_election.cc:290] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:19.100494  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.100754  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:19.090766  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:19.102027  5003 leader_election.cc:343] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.091544  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:19.108706  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.101677  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:19.109814  5032 raft_consensus.cc:2393] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:19.104578  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:19.110679  5035 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:19.104966  5001 leader_election.cc:343] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.112433  5001 leader_election.cc:304] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.113044  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:19.106490  5306 consensus_queue.cc:1048] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:19.114071  4812 raft_consensus.cc:3060] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.107318  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:19.115744  5229 raft_consensus.cc:2749] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.123055  5035 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:19.127051  5082 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:19.127632  5082 ts_tablet_manager.cc:1403] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.154s	user 0.024s	sys 0.000s
I20260709 12:29:19.130250  5086 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:19.130856  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:19.146392  5108 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.131356  5090 raft_consensus.cc:493] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.147143  5090 raft_consensus.cc:515] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.149452  5090 leader_election.cc:290] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:19.149806  5086 ts_tablet_manager.cc:1403] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.010s
I20260709 12:29:19.150044  4713 catalog_manager.cc:5697] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.154124  5082 raft_consensus.cc:359] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.155155  5082 raft_consensus.cc:385] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.155506  5082 raft_consensus.cc:740] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.157732  5086 raft_consensus.cc:359] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.158797  5086 raft_consensus.cc:385] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.159133  5086 raft_consensus.cc:740] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.160006  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.161612  5082 consensus_queue.cc:260] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.160130  5086 consensus_queue.cc:260] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.165480  4811 raft_consensus.cc:1275] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:19.163347  5082 ts_tablet_manager.cc:1434] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 12:29:19.134302  4927 leader_election.cc:304] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
W20260709 12:29:19.143110  4855 leader_election.cc:343] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.167037  5082 tablet_bootstrap.cc:492] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:19.167873  4855 leader_election.cc:304] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:19.143615  5091 tablet_bootstrap.cc:654] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.168311  4927 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.165925  5086 ts_tablet_manager.cc:1434] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:29:19.170639  5086 tablet_bootstrap.cc:492] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:19.172676  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:19.177641  5001 leader_election.cc:343] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.177845  5322 raft_consensus.cc:3060] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:19.178838  5003 leader_election.cc:343] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.179414  5003 leader_election.cc:304] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.180848  5324 raft_consensus.cc:2749] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:19.188400  4780 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 51351.9us
I20260709 12:29:19.189038  5307 raft_consensus.cc:2749] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.188970  5327 raft_consensus.cc:3060] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.189180  5108 consensus_queue.cc:1048] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:29:19.207134  4780 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:19.208295  5125 raft_consensus.cc:493] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.208437  5108 raft_consensus.cc:2804] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.209419  5108 raft_consensus.cc:697] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:19.209025  5125 raft_consensus.cc:515] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.210176  5108 consensus_queue.cc:237] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.211565  5125 leader_election.cc:290] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.228880  5083 tablet_bootstrap.cc:492] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:19.228979  5322 raft_consensus.cc:2749] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.229628  5083 ts_tablet_manager.cc:1403] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.242s	user 0.009s	sys 0.026s
I20260709 12:29:19.232743  4812 raft_consensus.cc:2468] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:19.232738  5083 raft_consensus.cc:359] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.234192  5083 raft_consensus.cc:385] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.234500  5083 raft_consensus.cc:740] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.217314  5307 consensus_queue.cc:1048] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
W20260709 12:29:19.220425  4780 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.235205  5083 consensus_queue.cc:260] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.246977  5082 tablet_bootstrap.cc:654] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.252767  5086 tablet_bootstrap.cc:654] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.252620  4855 leader_election.cc:304] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:19.248935  5327 raft_consensus.cc:2749] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.251555  4713 catalog_manager.cc:5697] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.257053  5083 ts_tablet_manager.cc:1434] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 12:29:19.258517  5083 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:19.266593  5091 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:19.267385  5091 ts_tablet_manager.cc:1403] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.183s	user 0.018s	sys 0.009s
I20260709 12:29:19.272023  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:19.271888  5108 consensus_queue.cc:1048] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.272918  5035 raft_consensus.cc:2468] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:19.274272  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.274773  4780 leader_election.cc:304] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:19.275695  5125 raft_consensus.cc:2804] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.276115  5125 raft_consensus.cc:493] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.276582  5125 raft_consensus.cc:3060] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.260712  5322 raft_consensus.cc:2804] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.278848  5322 raft_consensus.cc:697] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
W20260709 12:29:19.282480  4780 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.279845  5322 consensus_queue.cc:237] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.295301  5125 raft_consensus.cc:515] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.299052  5307 raft_consensus.cc:493] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.299649  5307 raft_consensus.cc:515] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.302166  5307 leader_election.cc:290] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:19.296838  5091 raft_consensus.cc:359] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.302812  5091 raft_consensus.cc:385] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.303126  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.299108  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:19.304916  4811 raft_consensus.cc:2393] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
W20260709 12:29:19.307507  4780 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.310068  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:19.310361  4855 consensus_peers.cc:597] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:19.311785  5083 tablet_bootstrap.cc:654] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.303884  5091 raft_consensus.cc:740] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.313201  5091 consensus_queue.cc:260] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.315263  5125 leader_election.cc:290] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.316532  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:19.317189  5034 raft_consensus.cc:3060] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.320950  5091 ts_tablet_manager.cc:1434] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.053s	user 0.003s	sys 0.006s
I20260709 12:29:19.322131  5091 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:19.311785  4927 leader_election.cc:304] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.340622  5091 tablet_bootstrap.cc:654] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.342115  5086 tablet_bootstrap.cc:492] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:19.342674  5086 ts_tablet_manager.cc:1403] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.172s	user 0.015s	sys 0.003s
I20260709 12:29:19.335664  5307 raft_consensus.cc:3060] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.342085  5034 raft_consensus.cc:2468] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:19.354784  4887 raft_consensus.cc:1275] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:19.355700  5086 raft_consensus.cc:359] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.356652  5086 raft_consensus.cc:385] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.357065  5086 raft_consensus.cc:740] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.356808  5090 consensus_queue.cc:1048] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.359154  5220 raft_consensus.cc:493] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.359692  5220 raft_consensus.cc:515] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.359841  4712 catalog_manager.cc:5697] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.361110  5082 tablet_bootstrap.cc:492] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:19.361918  5220 leader_election.cc:290] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:19.364718  5082 ts_tablet_manager.cc:1403] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.198s	user 0.016s	sys 0.017s
I20260709 12:29:19.365971  5083 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:19.366506  5083 ts_tablet_manager.cc:1403] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.108s	user 0.006s	sys 0.009s
I20260709 12:29:19.358431  5086 consensus_queue.cc:260] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.369104  5086 ts_tablet_manager.cc:1434] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:29:19.370826  5307 raft_consensus.cc:2749] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.373400  5086 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:19.375419  5220 consensus_queue.cc:1048] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.375569  4780 leader_election.cc:304] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:19.375384  5083 raft_consensus.cc:359] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.383798  5083 raft_consensus.cc:385] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.384780  4961 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.392135  5083 raft_consensus.cc:740] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.394294  5302 raft_consensus.cc:2804] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.394435  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.398104  5302 raft_consensus.cc:697] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:19.393787  5083 consensus_queue.cc:260] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.401046  5082 raft_consensus.cc:359] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.403081  5083 ts_tablet_manager.cc:1434] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:29:19.403339  5082 raft_consensus.cc:385] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.403713  5082 raft_consensus.cc:740] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.404452  5082 consensus_queue.cc:260] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.406580  5082 ts_tablet_manager.cc:1434] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.002s
I20260709 12:29:19.407480  5082 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:19.412809  5091 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:19.413803  5082 tablet_bootstrap.cc:654] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.414513  5091 ts_tablet_manager.cc:1403] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.017s
I20260709 12:29:19.400375  5302 consensus_queue.cc:237] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.415563  5086 tablet_bootstrap.cc:654] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.416801  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.419411  5091 raft_consensus.cc:359] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.420133  5091 raft_consensus.cc:385] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.420527  5091 raft_consensus.cc:740] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.422268  5091 consensus_queue.cc:260] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.424546  5091 ts_tablet_manager.cc:1434] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
W20260709 12:29:19.425019  4855 leader_election.cc:343] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.430454  5083 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:19.433168  5125 consensus_queue.cc:1048] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:19.440471  4855 leader_election.cc:343] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.441011  4855 leader_election.cc:304] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:19.441943  5220 raft_consensus.cc:2749] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.443454  5220 consensus_queue.cc:1048] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:19.445917  5220 consensus_queue.cc:1048] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:19.430809  5091 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:19.449960  4964 raft_consensus.cc:1275] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:19.451997  5324 consensus_queue.cc:1048] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:19.456997  5125 consensus_queue.cc:1048] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.459642  5091 tablet_bootstrap.cc:654] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.486013  4710 catalog_manager.cc:5697] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.519972  5091 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:19.520716  5091 ts_tablet_manager.cc:1403] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.002s
I20260709 12:29:19.525051  5086 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:19.525767  5086 ts_tablet_manager.cc:1403] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.153s	user 0.013s	sys 0.012s
I20260709 12:29:19.528712  5086 raft_consensus.cc:359] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.529438  5086 raft_consensus.cc:385] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.529704  5086 raft_consensus.cc:740] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.530339  5083 tablet_bootstrap.cc:654] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.530350  5086 consensus_queue.cc:260] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.607738  5302 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.629377  5086 ts_tablet_manager.cc:1434] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.103s	user 0.002s	sys 0.009s
I20260709 12:29:19.646908  5091 raft_consensus.cc:359] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.648193  5091 raft_consensus.cc:385] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.648605  5091 raft_consensus.cc:740] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.649708  5091 consensus_queue.cc:260] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:19.661818  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 75121.3us
I20260709 12:29:19.663095  5229 raft_consensus.cc:493] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.663100  5091 ts_tablet_manager.cc:1434] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.142s	user 0.003s	sys 0.010s
I20260709 12:29:19.609444  5302 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:19.674968  4778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 91687.9us
W20260709 12:29:19.676376  4780 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 80529.8us
I20260709 12:29:19.697511  5090 raft_consensus.cc:493] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.698678  5090 raft_consensus.cc:515] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.709929  5302 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:19.664816  5229 raft_consensus.cc:515] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.719206  5229 leader_election.cc:290] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:19.737263  5090 leader_election.cc:290] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:19.738675  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:19.739308  5082 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
W20260709 12:29:19.752363  5360 log.cc:927] Time spent T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Append to log took a long time: real 0.075s	user 0.000s	sys 0.009s
I20260709 12:29:19.753124  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:19.760039  5212 consensus_queue.cc:1048] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.761085  5091 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:19.765623  5082 ts_tablet_manager.cc:1403] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.358s	user 0.021s	sys 0.013s
I20260709 12:29:19.769163  5086 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:19.806391  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:19.769718  5398 raft_consensus.cc:493] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.808588  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:19.808885  5398 raft_consensus.cc:515] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:19.811874  5398 leader_election.cc:290] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:19.809662  5035 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:19.767210  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.808530  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:19.818465  4778 leader_election.cc:343] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:19.800810  5003 leader_election.cc:343] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.825551  5300 consensus_queue.cc:1048] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.849030  5211 raft_consensus.cc:493] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.849903  5211 raft_consensus.cc:515] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.852916  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:19.857507  5211 leader_election.cc:290] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.862941  5091 tablet_bootstrap.cc:654] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.877060  4958 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.864130  4780 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
W20260709 12:29:19.899065  4780 leader_election.cc:343] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.910159  5086 tablet_bootstrap.cc:654] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.915467  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:19.919802  5001 leader_election.cc:343] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.930862  4964 raft_consensus.cc:3060] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.933343  5082 raft_consensus.cc:359] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.934386  5082 raft_consensus.cc:385] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.934823  5082 raft_consensus.cc:740] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.936622  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:19.939100  5001 leader_election.cc:343] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.939567  5001 leader_election.cc:304] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.940491  5229 raft_consensus.cc:2749] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.940402  5403 raft_consensus.cc:2804] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.941776  5403 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.942194  5403 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.944967  5091 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:19.945684  5091 ts_tablet_manager.cc:1403] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.185s	user 0.015s	sys 0.003s
I20260709 12:29:19.945865  5322 raft_consensus.cc:493] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.946532  5322 raft_consensus.cc:515] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:19.948930  5322 leader_election.cc:290] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:19.950011  5091 raft_consensus.cc:359] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.951092  5091 raft_consensus.cc:385] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.951421  5091 raft_consensus.cc:740] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.952528  5091 consensus_queue.cc:260] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.955672  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:19.956760  5034 raft_consensus.cc:2468] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
W20260709 12:29:19.958030  5003 leader_election.cc:343] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.958590  5003 leader_election.cc:304] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.958935  5394 consensus_queue.cc:1048] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.961509  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79100fd1ff6942f5b408d20820466ef2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:19.961728  4855 leader_election.cc:304] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:19.962693  5090 raft_consensus.cc:2749] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.962821  5212 raft_consensus.cc:2804] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.963358  5212 raft_consensus.cc:493] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:19.959901  4778 leader_election.cc:343] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.963162  5082 consensus_queue.cc:260] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:19.965256  4778 leader_election.cc:304] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:19.965355  5212 raft_consensus.cc:3060] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.980602  5083 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:19.981942  5091 ts_tablet_manager.cc:1434] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 12:29:19.984588  5083 ts_tablet_manager.cc:1403] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.555s	user 0.016s	sys 0.015s
I20260709 12:29:19.975112  5300 raft_consensus.cc:2749] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.989941  5211 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.991477  5211 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:19.993654  5091 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:19.995869  5211 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:19.996580  5082 ts_tablet_manager.cc:1434] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.192s	user 0.003s	sys 0.007s
I20260709 12:29:20.001366  4929 leader_election.cc:304] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:20.017997  5083 raft_consensus.cc:359] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.018285  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:20.021770  5086 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:20.022434  5086 ts_tablet_manager.cc:1403] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.254s	user 0.016s	sys 0.008s
I20260709 12:29:20.019549  5083 raft_consensus.cc:385] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.023679  5083 raft_consensus.cc:740] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
W20260709 12:29:20.024263  4855 leader_election.cc:343] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.030516  5083 consensus_queue.cc:260] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.052140  5299 raft_consensus.cc:493] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.046873  5037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.052763  5299 raft_consensus.cc:515] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.055505  5299 leader_election.cc:290] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:20.056524  5401 consensus_queue.cc:1048] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
W20260709 12:29:20.062733  4780 consensus_peers.cc:597] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:20.062693  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:20.065939  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:20.070622  5403 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.075088  5083 ts_tablet_manager.cc:1434] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.001s
I20260709 12:29:20.076884  5083 tablet_bootstrap.cc:492] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:20.080747  5082 tablet_bootstrap.cc:492] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:20.092935  5403 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.104300  5082 tablet_bootstrap.cc:654] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.107596  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:20.088984  5091 tablet_bootstrap.cc:654] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.093662  5125 consensus_queue.cc:1048] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:20.083070  5086 raft_consensus.cc:359] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.116972  5086 raft_consensus.cc:385] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.117413  5086 raft_consensus.cc:740] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.118337  5086 consensus_queue.cc:260] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.095268  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:20.097575  4929 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:20.100871  5083 tablet_bootstrap.cc:654] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.108704  4887 raft_consensus.cc:2468] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.116000  5212 raft_consensus.cc:515] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.119439  4961 raft_consensus.cc:2468] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.123250  5032 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.122205  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:20.126753  4780 leader_election.cc:304] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:20.131868  5125 raft_consensus.cc:2804] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:20.132356  5125 raft_consensus.cc:493] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.132714  5125 raft_consensus.cc:3060] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.133581  4886 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.156620  5086 ts_tablet_manager.cc:1434] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.134s	user 0.007s	sys 0.003s
I20260709 12:29:20.158200  5086 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
W20260709 12:29:20.150712  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 51305.9us
I20260709 12:29:20.194126  5307 raft_consensus.cc:2749] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:20.197266  5324 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.197839  5324 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.200001  5324 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:20.202018  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:20.203783  5003 leader_election.cc:343] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.208647  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:20.211071  5212 leader_election.cc:290] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:20.214744  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:20.215428  5038 raft_consensus.cc:3060] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:20.216177  5003 leader_election.cc:343] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.216768  5003 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:20.217384  5083 tablet_bootstrap.cc:492] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:20.218024  5083 ts_tablet_manager.cc:1403] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.141s	user 0.026s	sys 0.035s
I20260709 12:29:20.217962  5324 raft_consensus.cc:2749] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:20.223302  5083 raft_consensus.cc:359] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.224098  5083 raft_consensus.cc:385] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.224514  5083 raft_consensus.cc:740] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.225375  5083 consensus_queue.cc:260] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.228019  5083 ts_tablet_manager.cc:1434] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:20.232502  5083 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:20.238878  5082 tablet_bootstrap.cc:492] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:20.239650  5082 ts_tablet_manager.cc:1403] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.000s
I20260709 12:29:20.240749  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
W20260709 12:29:20.243044  4855 leader_election.cc:343] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.246404  5082 raft_consensus.cc:359] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.247152  5082 raft_consensus.cc:385] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.247431  5082 raft_consensus.cc:740] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.248208  5082 consensus_queue.cc:260] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.250393  5082 ts_tablet_manager.cc:1434] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.006s
I20260709 12:29:20.250931  5032 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.252343  4780 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:20.252732  5091 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:20.252543  5125 raft_consensus.cc:515] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.253265  5091 ts_tablet_manager.cc:1403] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.260s	user 0.012s	sys 0.002s
I20260709 12:29:20.253206  5403 raft_consensus.cc:2804] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:20.253921  5403 raft_consensus.cc:697] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:20.254935  5403 consensus_queue.cc:237] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.266011  4886 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.266850  5086 tablet_bootstrap.cc:654] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.266132  5083 tablet_bootstrap.cc:654] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.267412  5091 raft_consensus.cc:359] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.269563  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:20.270203  4961 raft_consensus.cc:3060] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.268150  5091 raft_consensus.cc:385] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.271435  5091 raft_consensus.cc:740] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.272277  5091 consensus_queue.cc:260] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.274685  5091 ts_tablet_manager.cc:1434] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:29:20.275833  5091 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:20.276595  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:20.277175  5082 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:20.277297  4890 raft_consensus.cc:3060] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.281385  5211 raft_consensus.cc:2749] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:20.287572  5125 leader_election.cc:290] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:20.288141  5212 raft_consensus.cc:493] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.303606  5212 raft_consensus.cc:515] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.294468  4961 raft_consensus.cc:2468] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.298381  5038 raft_consensus.cc:2468] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:20.301959  4890 raft_consensus.cc:2468] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.319499  4780 leader_election.cc:304] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:20.310222  4963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:20.323032  5398 raft_consensus.cc:2804] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:20.323539  5398 raft_consensus.cc:697] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:20.324502  5398 consensus_queue.cc:237] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:20.324959  4855 leader_election.cc:343] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.332221  4855 leader_election.cc:304] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:20.333479  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.333518  5394 raft_consensus.cc:2804] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:20.334286  5032 raft_consensus.cc:2468] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:20.334333  5394 raft_consensus.cc:697] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:20.311257  5212 leader_election.cc:290] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:20.335134  5086 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:20.337201  5086 ts_tablet_manager.cc:1403] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.179s	user 0.022s	sys 0.007s
I20260709 12:29:20.337150  4855 leader_election.cc:304] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:20.335999  5394 consensus_queue.cc:237] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.337685  5082 tablet_bootstrap.cc:654] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.348789  5403 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.347656  5212 raft_consensus.cc:2804] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:20.357300  5212 raft_consensus.cc:493] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.357991  5212 raft_consensus.cc:3060] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.354081  5083 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:20.364373  5083 ts_tablet_manager.cc:1403] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.132s	user 0.005s	sys 0.018s
I20260709 12:29:20.374367  4713 catalog_manager.cc:5697] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:20.375972  5091 tablet_bootstrap.cc:654] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.382493  5086 raft_consensus.cc:359] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.383278  5086 raft_consensus.cc:385] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.384516  5086 raft_consensus.cc:740] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.354833  5403 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.388425  5403 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:20.391052  5083 raft_consensus.cc:359] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.391839  5083 raft_consensus.cc:385] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.391713  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.392679  5038 raft_consensus.cc:2468] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.392786  5083 raft_consensus.cc:740] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.393754  5083 consensus_queue.cc:260] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.391726  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:20.395169  4780 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
W20260709 12:29:20.396100  4780 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.397290  5083 ts_tablet_manager.cc:1434] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:29:20.397774  5403 raft_consensus.cc:2804] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:20.398228  5403 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.398581  5403 raft_consensus.cc:3060] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.395738  4711 catalog_manager.cc:5697] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:20.405215  5083 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:20.407652  5403 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.410775  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:20.410956  5403 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
W20260709 12:29:20.412101  4780 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.412858  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:20.413545  5038 raft_consensus.cc:3060] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.414165  5082 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:20.414816  5082 ts_tablet_manager.cc:1403] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.138s	user 0.005s	sys 0.017s
I20260709 12:29:20.416517  5086 consensus_queue.cc:260] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.419917  5086 ts_tablet_manager.cc:1434] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 12:29:20.421633  5038 raft_consensus.cc:2468] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.422020  5212 raft_consensus.cc:515] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.423120  4780 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:20.425458  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:20.426072  5038 raft_consensus.cc:3060] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.429219  5398 raft_consensus.cc:2804] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:20.429757  5398 raft_consensus.cc:697] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:20.430124  5091 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:20.430925  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:20.431602  5086 tablet_bootstrap.cc:492] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:20.430601  5091 ts_tablet_manager.cc:1403] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.155s	user 0.010s	sys 0.003s
I20260709 12:29:20.430567  5398 consensus_queue.cc:237] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.422264  5083 tablet_bootstrap.cc:654] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:20.436914  4855 leader_election.cc:343] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.440896  5038 raft_consensus.cc:2468] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:20.440749  5082 raft_consensus.cc:359] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.441589  5082 raft_consensus.cc:385] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.441926  5082 raft_consensus.cc:740] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.442436  4855 leader_election.cc:304] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:20.443666  5395 raft_consensus.cc:2804] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:20.436789  5212 leader_election.cc:290] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:20.444196  5395 raft_consensus.cc:697] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:20.445122  5395 consensus_queue.cc:237] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.443336  5082 consensus_queue.cc:260] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.458973  5082 ts_tablet_manager.cc:1434] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.001s
I20260709 12:29:20.455818  5091 raft_consensus.cc:359] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.460345  5091 raft_consensus.cc:385] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.460846  5091 raft_consensus.cc:740] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.461666  5082 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:20.461829  5091 consensus_queue.cc:260] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.470181  5091 ts_tablet_manager.cc:1434] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:29:20.471345  5091 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:20.471627  5086 tablet_bootstrap.cc:654] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.466650  5229 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.473683  5229 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.477169  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:20.478641  5003 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.479681  5229 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.483150  5091 tablet_bootstrap.cc:654] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.486759  5082 tablet_bootstrap.cc:654] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.485970  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:20.496742  5001 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.497251  5001 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:20.503919  5324 raft_consensus.cc:2749] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:20.523129  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:20.526038  5083 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:20.526679  5083 ts_tablet_manager.cc:1403] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.122s	user 0.017s	sys 0.010s
I20260709 12:29:20.531633  4711 catalog_manager.cc:5697] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:20.533565  5083 raft_consensus.cc:359] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.536399  5083 raft_consensus.cc:385] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.536774  5083 raft_consensus.cc:740] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.539909  5083 consensus_queue.cc:260] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.542279  5091 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:20.543066  5091 ts_tablet_manager.cc:1403] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.009s
I20260709 12:29:20.568625  5091 raft_consensus.cc:359] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.569510  5091 raft_consensus.cc:385] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.569870  5091 raft_consensus.cc:740] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.575525  5091 consensus_queue.cc:260] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.574199  5083 ts_tablet_manager.cc:1434] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.002s
I20260709 12:29:20.584400  5086 tablet_bootstrap.cc:492] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:20.584710  5083 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:20.585076  5086 ts_tablet_manager.cc:1403] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.011s
I20260709 12:29:20.588191  5086 raft_consensus.cc:359] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.589514  5086 raft_consensus.cc:385] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.589831  5086 raft_consensus.cc:740] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.590011  5091 ts_tablet_manager.cc:1434] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.002s
I20260709 12:29:20.590152  5082 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:20.590816  5082 ts_tablet_manager.cc:1403] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.129s	user 0.027s	sys 0.047s
I20260709 12:29:20.590689  5086 consensus_queue.cc:260] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.591245  5091 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:20.595840  4711 catalog_manager.cc:5697] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:20.598122  5086 ts_tablet_manager.cc:1434] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:29:20.597729  4711 catalog_manager.cc:5697] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:20.604117  5091 tablet_bootstrap.cc:654] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.594309  5082 raft_consensus.cc:359] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.604607  4964 raft_consensus.cc:3060] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.604842  5082 raft_consensus.cc:385] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.605203  5082 raft_consensus.cc:740] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.607371  5086 tablet_bootstrap.cc:492] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:20.609524  5083 tablet_bootstrap.cc:654] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.612447  5082 consensus_queue.cc:260] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.616665  5086 tablet_bootstrap.cc:654] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.622032  5082 ts_tablet_manager.cc:1434] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:29:20.623272  5082 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:20.628451  5091 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:20.629062  5091 ts_tablet_manager.cc:1403] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.002s
I20260709 12:29:20.629832  5082 tablet_bootstrap.cc:654] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.633719  5091 raft_consensus.cc:359] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.636408  5083 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:20.636719  5091 raft_consensus.cc:385] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.637044  5083 ts_tablet_manager.cc:1403] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.003s
I20260709 12:29:20.637250  5091 raft_consensus.cc:740] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.640026  5086 tablet_bootstrap.cc:492] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:20.640560  5086 ts_tablet_manager.cc:1403] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.000s
I20260709 12:29:20.640727  5083 raft_consensus.cc:359] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.641566  5083 raft_consensus.cc:385] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.641878  5083 raft_consensus.cc:740] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.642688  5083 consensus_queue.cc:260] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.643597  5086 raft_consensus.cc:359] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.644160  5086 raft_consensus.cc:385] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.644433  5086 raft_consensus.cc:740] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.644966  5091 consensus_queue.cc:260] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.645211  5086 consensus_queue.cc:260] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.647495  5091 ts_tablet_manager.cc:1434] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 12:29:20.647935  5086 ts_tablet_manager.cc:1434] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:20.649025  5091 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:20.649101  5086 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:20.652486  5229 raft_consensus.cc:493] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.653048  5229 raft_consensus.cc:515] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.655310  5083 ts_tablet_manager.cc:1434] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:29:20.655666  5229 leader_election.cc:290] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:20.658120  5083 tablet_bootstrap.cc:492] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:20.660006  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:20.661485  5001 leader_election.cc:343] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.661739  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:20.666425  5091 tablet_bootstrap.cc:654] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:20.671602  5003 leader_election.cc:343] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.672127  5003 leader_election.cc:304] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:20.672966  5433 raft_consensus.cc:2749] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:20.678401  5300 raft_consensus.cc:493] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.678995  5300 raft_consensus.cc:515] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.681221  5300 leader_election.cc:290] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.681922  5086 tablet_bootstrap.cc:654] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.683141  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:20.683853  4886 raft_consensus.cc:2468] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.684583  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.685470  5038 raft_consensus.cc:2468] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.685377  4778 leader_election.cc:304] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:20.688252  5300 raft_consensus.cc:2804] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:20.688874  5300 raft_consensus.cc:493] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.689399  5300 raft_consensus.cc:3060] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.694056  5086 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:20.694618  5086 ts_tablet_manager.cc:1403] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.004s
I20260709 12:29:20.697611  5086 raft_consensus.cc:359] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.698284  5086 raft_consensus.cc:385] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.698546  5086 raft_consensus.cc:740] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.699273  5086 consensus_queue.cc:260] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.699918  5398 consensus_queue.cc:1048] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:20.701617  5086 ts_tablet_manager.cc:1434] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:20.702716  5086 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:20.714109  5086 tablet_bootstrap.cc:654] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.733088  5091 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:20.733774  5091 ts_tablet_manager.cc:1403] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.012s
I20260709 12:29:20.742262  5091 raft_consensus.cc:359] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.743433  5091 raft_consensus.cc:385] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.743781  5091 raft_consensus.cc:740] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.752926  5091 consensus_queue.cc:260] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.767447  5229 raft_consensus.cc:493] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.771116  5229 raft_consensus.cc:515] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.761675  5398 raft_consensus.cc:493] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.773056  5398 raft_consensus.cc:515] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.773090  5229 leader_election.cc:290] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:20.774303  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:20.775787  5003 leader_election.cc:343] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.777655  5300 raft_consensus.cc:515] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.780299  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:20.780503  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:20.781059  5032 raft_consensus.cc:3060] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.781294  5398 leader_election.cc:290] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.783448  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.784185  5034 raft_consensus.cc:2468] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.785640  4780 leader_election.cc:304] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:20.785858  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:20.787479  5398 raft_consensus.cc:2804] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:29:20.787763  4778 leader_election.cc:343] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.788404  5298 consensus_queue.cc:1048] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:20.789119  5398 raft_consensus.cc:493] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.789674  5398 raft_consensus.cc:3060] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.790324  5300 leader_election.cc:290] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.793910  5032 raft_consensus.cc:2468] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.794265  5307 raft_consensus.cc:493] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.795441  4780 leader_election.cc:304] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:20.795779  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:20.795872  5307 raft_consensus.cc:515] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.796662  5300 raft_consensus.cc:2804] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:20.797609  5300 raft_consensus.cc:697] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
W20260709 12:29:20.798646  5003 leader_election.cc:343] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:20.798530  5300 consensus_queue.cc:237] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.799983  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:20.801774  5082 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:20.802318  5082 ts_tablet_manager.cc:1403] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.179s	user 0.047s	sys 0.053s
I20260709 12:29:20.803812  5307 leader_election.cc:290] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:20.796595  4886 raft_consensus.cc:3060] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.804652  5082 raft_consensus.cc:359] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.799146  5003 leader_election.cc:304] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:20.811100  5082 raft_consensus.cc:385] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.811553  5082 raft_consensus.cc:740] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.812345  5082 consensus_queue.cc:260] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.812885  5229 raft_consensus.cc:2749] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:20.815162  5082 ts_tablet_manager.cc:1434] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:29:20.815348  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.816423  5032 raft_consensus.cc:2393] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:20.816598  5082 tablet_bootstrap.cc:492] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:20.817673  4929 leader_election.cc:304] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:20.818683  5211 raft_consensus.cc:3060] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:20.825223  5083 tablet_bootstrap.cc:654] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.827230  5082 tablet_bootstrap.cc:654] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:20.833896  4886 raft_consensus.cc:2468] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:20.842880  5086 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:20.843617  5086 ts_tablet_manager.cc:1403] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.141s	user 0.034s	sys 0.043s
I20260709 12:29:20.849756  5125 consensus_queue.cc:1048] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:20.855960  5091 ts_tablet_manager.cc:1434] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.119s	user 0.006s	sys 0.001s
I20260709 12:29:20.856161  5086 raft_consensus.cc:359] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.857349  5086 raft_consensus.cc:385] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:20.857769  5086 raft_consensus.cc:740] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:20.858744  5086 consensus_queue.cc:260] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:20.867449  5300 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.878269  5086 ts_tablet_manager.cc:1434] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.005s
I20260709 12:29:20.908938  5091 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:20.877988  5211 raft_consensus.cc:2749] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:20.870536  5300 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.929744  5399 raft_consensus.cc:493] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.930384  5399 raft_consensus.cc:515] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.939630  5086 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:20.947683  5394 consensus_queue.cc:1048] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:29:20.952682  5082 tablet_bootstrap.cc:492] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:20.953296  5082 ts_tablet_manager.cc:1403] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.137s	user 0.032s	sys 0.041s
I20260709 12:29:20.948990  5448 raft_consensus.cc:493] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.954136  5448 raft_consensus.cc:515] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.947839  5307 raft_consensus.cc:493] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.968721  5307 raft_consensus.cc:515] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.975095  5307 leader_election.cc:290] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:20.962468  5398 raft_consensus.cc:515] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:20.977324  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:20.977504  5398 leader_election.cc:290] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.978139  5038 raft_consensus.cc:2393] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:20.979197  5448 leader_election.cc:290] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:20.983824  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:20.984661  4812 raft_consensus.cc:2468] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:20.984779  5399 leader_election.cc:290] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:20.984947  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:20.985705  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:20.986220  5300 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:20.986362  4887 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:20.988497  5394 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.988981  5394 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:20.991109  5394 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:20.993055  4711 catalog_manager.cc:5697] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:20.995905  5091 tablet_bootstrap.cc:654] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.000690  4929 leader_election.cc:304] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
W20260709 12:29:21.003321  4778 leader_election.cc:343] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.004163  4778 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:21.004712  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 58603.4us
I20260709 12:29:21.002946  5082 raft_consensus.cc:359] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:20.992223  5086 tablet_bootstrap.cc:654] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.005650  5082 raft_consensus.cc:385] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.006290  5082 raft_consensus.cc:740] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.014416  4963 raft_consensus.cc:3060] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.016362  5082 consensus_queue.cc:260] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.004819  5448 raft_consensus.cc:2804] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:21.020313  5082 ts_tablet_manager.cc:1434] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 12:29:21.008270  5300 raft_consensus.cc:2804] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:21.021421  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:21.023386  5038 raft_consensus.cc:3060] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.032961  5091 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:21.033659  5091 ts_tablet_manager.cc:1403] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.164s	user 0.010s	sys 0.003s
I20260709 12:29:21.022073  5448 raft_consensus.cc:493] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.023958  5300 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.077934  5082 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:21.043490  5448 raft_consensus.cc:3060] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.073740  5300 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.075731  5038 raft_consensus.cc:2468] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:21.076582  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:21.086421  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:21.088910  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.095690  4959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.095930  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.079237  5452 consensus_queue.cc:1048] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:29:21.101285  5083 tablet_bootstrap.cc:492] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:21.102002  5083 ts_tablet_manager.cc:1403] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.444s	user 0.015s	sys 0.002s
W20260709 12:29:21.102988  4780 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 84286.4us
I20260709 12:29:21.109167  5125 consensus_queue.cc:1048] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:21.110800  5091 raft_consensus.cc:359] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.111357  5091 raft_consensus.cc:385] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.109266  4780 leader_election.cc:304] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:21.114630  5125 raft_consensus.cc:2804] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:21.115288  5125 raft_consensus.cc:697] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:21.111528  5091 raft_consensus.cc:740] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.116366  5125 consensus_queue.cc:237] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.088965  4888 raft_consensus.cc:2468] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:21.123457  5091 consensus_queue.cc:260] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.108691  5433 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.131742  5433 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.124840  5298 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:21.146512  4780 leader_election.cc:343] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.165992  5091 ts_tablet_manager.cc:1434] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.132s	user 0.008s	sys 0.002s
I20260709 12:29:21.148398  4958 raft_consensus.cc:3060] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.192533  5462 consensus_queue.cc:1048] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:21.145162  5452 consensus_queue.cc:1048] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 12:29:21.200363  4927 leader_election.cc:304] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
W20260709 12:29:21.213663  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 46303.2us
W20260709 12:29:21.213429  4780 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 63981.9us
I20260709 12:29:21.215211  5082 tablet_bootstrap.cc:654] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.213308  5083 raft_consensus.cc:359] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.218127  5091 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:21.218140  5083 raft_consensus.cc:385] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.218856  5083 raft_consensus.cc:740] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.224660  5083 consensus_queue.cc:260] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:21.226361  4927 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 88125.7us
W20260709 12:29:21.227746  4855 leader_election.cc:343] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.162169  5298 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.240842  5298 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.236869  4815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:21.246640  5433 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:21.250352  5448 raft_consensus.cc:515] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.238547  5449 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.238770  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:21.272985  4778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 105623us
W20260709 12:29:21.254537  5003 leader_election.cc:343] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.285208  5083 ts_tablet_manager.cc:1434] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.183s	user 0.010s	sys 0.002s
I20260709 12:29:21.285458  5091 tablet_bootstrap.cc:654] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.302429  5449 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.304826  5449 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:21.319521  5083 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:21.291653  5398 raft_consensus.cc:493] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.319366  4964 raft_consensus.cc:2393] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:21.291487  4811 raft_consensus.cc:2468] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:21.379601  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:21.380434  5030 raft_consensus.cc:2468] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
W20260709 12:29:21.381151  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 61519.3us
I20260709 12:29:21.366863  5467 raft_consensus.cc:493] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.383358  5448 leader_election.cc:290] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:21.387879  5086 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:21.388440  5086 ts_tablet_manager.cc:1403] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.449s	user 0.018s	sys 0.027s
I20260709 12:29:21.326627  5300 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.368073  5398 raft_consensus.cc:515] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.372748  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.399029  5398 leader_election.cc:290] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:21.399546  5460 raft_consensus.cc:493] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.399907  5300 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:21.400158  5460 raft_consensus.cc:515] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.402835  5083 tablet_bootstrap.cc:654] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.404735  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:21.405364  4888 raft_consensus.cc:3060] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.393682  5467 raft_consensus.cc:515] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.407832  5467 leader_election.cc:290] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.408635  5460 leader_election.cc:290] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
W20260709 12:29:21.411147  5463 log.cc:927] Time spent T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:29:21.417461  5082 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:21.417999  5082 ts_tablet_manager.cc:1403] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.343s	user 0.032s	sys 0.014s
I20260709 12:29:21.411652  5466 raft_consensus.cc:2749] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:21.419323  5086 raft_consensus.cc:359] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.421803  5086 raft_consensus.cc:385] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.422463  5086 raft_consensus.cc:740] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.423357  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:21.423703  5086 consensus_queue.cc:260] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.427687  5086 ts_tablet_manager.cc:1434] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 12:29:21.430219  5086 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:21.439229  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80aa6fe92f51449ab69485480db26075" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:21.430124  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:21.440976  4812 raft_consensus.cc:2393] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
W20260709 12:29:21.442324  5001 leader_election.cc:343] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.443056  5001 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:21.443641  4885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:21.444327  4885 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.446017  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:21.446632  4780 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:21.447219  5476 raft_consensus.cc:2749] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:21.448650  5125 raft_consensus.cc:2804] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:21.449021  5125 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.449342  5125 raft_consensus.cc:3060] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.452317  4712 catalog_manager.cc:5697] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:21.458264  4778 leader_election.cc:343] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.458310  4885 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:21.461879  5398 consensus_queue.cc:1048] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:21.462760  5125 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:21.463063  5001 leader_election.cc:343] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:21.465840  4780 leader_election.cc:343] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.468993  5082 raft_consensus.cc:359] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.469774  5082 raft_consensus.cc:385] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.470068  5082 raft_consensus.cc:740] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.470798  5082 consensus_queue.cc:260] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.473402  5456 consensus_queue.cc:1048] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:21.473400  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.475289  5082 ts_tablet_manager.cc:1434] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:29:21.476459  5082 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:21.486279  4888 raft_consensus.cc:2468] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:21.487671  5125 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.491180  4778 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:21.492162  5125 raft_consensus.cc:2804] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:21.492647  5125 raft_consensus.cc:697] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:21.493389  5125 consensus_queue.cc:237] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.494567  5478 raft_consensus.cc:493] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.495108  5478 raft_consensus.cc:515] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.495827  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.494578  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:21.498483  4964 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.499832  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:21.500747  5032 raft_consensus.cc:2468] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:21.503557  5091 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:21.497284  5478 leader_election.cc:290] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.508172  5091 ts_tablet_manager.cc:1403] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.302s	user 0.037s	sys 0.042s
I20260709 12:29:21.512451  5091 raft_consensus.cc:359] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.520025  5091 raft_consensus.cc:385] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.521314  5091 raft_consensus.cc:740] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.522691  5091 consensus_queue.cc:260] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.514993  4964 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:21.530781  5091 ts_tablet_manager.cc:1434] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
W20260709 12:29:21.533867  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 317900us
I20260709 12:29:21.536141  5091 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:21.520524  5082 tablet_bootstrap.cc:654] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.540570  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:21.550209  5091 tablet_bootstrap.cc:654] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.555346  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:21.559717  5462 consensus_queue.cc:1048] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:21.560662  5038 raft_consensus.cc:3060] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.562359  5083 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:21.562902  5083 ts_tablet_manager.cc:1403] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.249s	user 0.011s	sys 0.001s
I20260709 12:29:21.564245  4780 leader_election.cc:304] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:21.569968  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:21.571758  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80aa6fe92f51449ab69485480db26075" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:21.581665  5483 log.cc:927] Time spent T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:29:21.582578  5086 tablet_bootstrap.cc:654] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.582466  5125 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.583762  5125 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.584596  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "638da9762ab2493e85b9205f6c2679f9" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:21.585260  4812 raft_consensus.cc:3060] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.586036  5125 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
W20260709 12:29:21.588171  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 186952us
I20260709 12:29:21.594421  4927 leader_election.cc:304] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:21.590071  4780 leader_election.cc:343] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.595834  4780 leader_election.cc:304] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:21.596606  4855 leader_election.cc:304] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:21.597730  5481 raft_consensus.cc:2764] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:29:21.601450  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:21.605531  5038 raft_consensus.cc:2468] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:21.607012  4855 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:21.608532  5479 raft_consensus.cc:2804] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:21.609072  5479 raft_consensus.cc:697] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:21.587109  5462 raft_consensus.cc:2804] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:21.609582  5462 raft_consensus.cc:493] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.609906  5479 consensus_queue.cc:237] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.613493  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:21.617173  5125 raft_consensus.cc:2749] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:21.617790  4780 leader_election.cc:343] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.617077  4812 raft_consensus.cc:2468] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:21.588377  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
W20260709 12:29:21.623113  4778 leader_election.cc:343] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:21.643625  5003 leader_election.cc:343] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.644107  5003 leader_election.cc:304] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
W20260709 12:29:21.653295  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:21.657409  4929 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:21.655135  5083 raft_consensus.cc:359] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.668833  5083 raft_consensus.cc:385] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.609952  5462 raft_consensus.cc:3060] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.669202  5083 raft_consensus.cc:740] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.606531  5032 raft_consensus.cc:2468] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:21.628887  5481 raft_consensus.cc:2749] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:21.671416  5083 consensus_queue.cc:260] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.677764  5082 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:21.678340  5082 ts_tablet_manager.cc:1403] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.202s	user 0.033s	sys 0.027s
I20260709 12:29:21.678730  5091 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:21.679206  5091 ts_tablet_manager.cc:1403] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.013s
I20260709 12:29:21.655261  5460 raft_consensus.cc:2749] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:21.685617  5086 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:21.688650  5086 ts_tablet_manager.cc:1403] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.259s	user 0.010s	sys 0.010s
I20260709 12:29:21.654711  5212 consensus_queue.cc:1048] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:29:21.693034  5086 raft_consensus.cc:359] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.693761  5086 raft_consensus.cc:385] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.694491  5086 raft_consensus.cc:740] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.695277  5086 consensus_queue.cc:260] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.696960  5083 ts_tablet_manager.cc:1434] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.134s	user 0.008s	sys 0.002s
I20260709 12:29:21.698074  5086 ts_tablet_manager.cc:1434] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:21.735106  4959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.742543  5083 tablet_bootstrap.cc:492] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:21.758944  4712 catalog_manager.cc:5697] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:21.779434  4780 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 130101us
I20260709 12:29:21.757061  5082 raft_consensus.cc:359] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.769548  5086 tablet_bootstrap.cc:492] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:21.785882  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:21.786707  5032 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:21.789880  5086 tablet_bootstrap.cc:654] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.792963  5082 raft_consensus.cc:385] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.793330  5082 raft_consensus.cc:740] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.794085  5082 consensus_queue.cc:260] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.801066  5091 raft_consensus.cc:359] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.801891  5091 raft_consensus.cc:385] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.802181  5091 raft_consensus.cc:740] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.802865  5091 consensus_queue.cc:260] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.808415  5229 raft_consensus.cc:493] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.809134  5229 raft_consensus.cc:515] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.810153  5083 tablet_bootstrap.cc:654] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.811611  5229 leader_election.cc:290] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:21.813412  5082 ts_tablet_manager.cc:1434] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.135s	user 0.004s	sys 0.003s
I20260709 12:29:21.813628  4959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.814282  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:21.815896  5086 tablet_bootstrap.cc:492] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:21.816479  5086 ts_tablet_manager.cc:1403] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.000s
I20260709 12:29:21.819621  5086 raft_consensus.cc:359] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:21.816480  5003 leader_election.cc:343] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.821213  4885 raft_consensus.cc:3060] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.823311  5091 ts_tablet_manager.cc:1434] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.144s	user 0.007s	sys 0.000s
W20260709 12:29:21.829895  5003 leader_election.cc:343] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.830391  5003 leader_election.cc:304] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:21.828985  5082 tablet_bootstrap.cc:492] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
W20260709 12:29:21.831799  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 181867us
I20260709 12:29:21.836431  5229 raft_consensus.cc:2749] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:21.837653  5086 raft_consensus.cc:385] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.837987  5086 raft_consensus.cc:740] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.839537  5473 raft_consensus.cc:2749] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:21.840378  5086 consensus_queue.cc:260] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
W20260709 12:29:21.841141  4855 leader_election.cc:343] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.842870  5086 ts_tablet_manager.cc:1434] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:29:21.830565  5091 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:21.843994  5086 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:21.846753  5082 tablet_bootstrap.cc:654] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.846633  5464 consensus_queue.cc:1048] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:21.853672  5091 tablet_bootstrap.cc:654] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.860802  4855 leader_election.cc:304] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:21.861232  5399 raft_consensus.cc:493] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.861740  5399 raft_consensus.cc:515] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:21.863037  5452 raft_consensus.cc:2804] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:21.863500  5452 raft_consensus.cc:493] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.863868  5452 raft_consensus.cc:3060] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.863937  5399 leader_election.cc:290] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:21.866547  5086 tablet_bootstrap.cc:654] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:21.890765  5086 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:21.891281  5086 ts_tablet_manager.cc:1403] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.007s
I20260709 12:29:21.901803  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:21.907263  5086 raft_consensus.cc:359] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.908078  5086 raft_consensus.cc:385] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:21.908376  5086 raft_consensus.cc:740] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:21.909029  5086 consensus_queue.cc:260] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.911360  5086 ts_tablet_manager.cc:1434] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:29:21.912428  5086 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
W20260709 12:29:21.918527  4855 leader_election.cc:343] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.831518  4780 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
W20260709 12:29:21.920698  4780 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 94332.3us
I20260709 12:29:21.920959  5496 raft_consensus.cc:2804] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:21.921679  5496 raft_consensus.cc:697] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:21.922470  4780 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:21.830786  5462 raft_consensus.cc:515] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.922729  5496 consensus_queue.cc:237] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.931444  5462 leader_election.cc:290] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.936467  5464 raft_consensus.cc:2804] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:21.937024  5464 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.937389  5464 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.939106  5086 tablet_bootstrap.cc:654] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:21.945433  4780 leader_election.cc:343] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:21.947860  5452 raft_consensus.cc:515] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.950764  5452 leader_election.cc:290] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.952144  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:21.952803  5032 raft_consensus.cc:3060] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:21.965610  5479 raft_consensus.cc:493] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.966183  5479 raft_consensus.cc:515] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.967913  5464 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:21.968730  5479 leader_election.cc:290] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:21.969640  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:21.970645  4964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:21.973058  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d989f2ffa44c5480a996303eb45fbb" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:22.003268  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:21.978495  5083 tablet_bootstrap.cc:492] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:22.004122  4886 raft_consensus.cc:2393] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:22.004220  5031 raft_consensus.cc:3060] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.004424  5083 ts_tablet_manager.cc:1403] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.262s	user 0.027s	sys 0.014s
I20260709 12:29:21.974186  4711 catalog_manager.cc:5697] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:21.979418  5473 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.005942  5473 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:21.979036  5082 tablet_bootstrap.cc:492] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:22.007691  5082 ts_tablet_manager.cc:1403] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.179s	user 0.023s	sys 0.042s
I20260709 12:29:22.007402  5083 raft_consensus.cc:359] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.979693  5032 raft_consensus.cc:2468] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:22.008173  5083 raft_consensus.cc:385] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.008486  4964 raft_consensus.cc:3060] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.008543  5083 raft_consensus.cc:740] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.009411  5083 consensus_queue.cc:260] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.010445  5082 raft_consensus.cc:359] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.011152  5082 raft_consensus.cc:385] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.011421  5082 raft_consensus.cc:740] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.011498  5083 ts_tablet_manager.cc:1434] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:22.003335  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:22.013679  4855 leader_election.cc:304] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:22.014118  5083 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:22.015664  5452 raft_consensus.cc:2804] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.016227  5452 raft_consensus.cc:697] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:22.016985  4964 raft_consensus.cc:2468] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:22.017107  5452 consensus_queue.cc:237] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:21.995225  5467 consensus_queue.cc:1048] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:22.022051  4929 leader_election.cc:304] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.022950  5479 raft_consensus.cc:2749] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.012172  5082 consensus_queue.cc:260] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.023969  5083 tablet_bootstrap.cc:654] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.024712  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.025636  5032 raft_consensus.cc:2393] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:22.025847  5082 ts_tablet_manager.cc:1434] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.001s
I20260709 12:29:22.027269  5082 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:22.030833  5031 raft_consensus.cc:2468] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:22.042384  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:22.043253  4812 raft_consensus.cc:2468] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:22.052735  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:22.013679  4962 raft_consensus.cc:3060] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.008095  5473 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:22.057598  4929 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:22.086953  4780 leader_election.cc:304] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:22.095347  5471 raft_consensus.cc:3060] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.095589  5456 raft_consensus.cc:2804] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.087821  4713 catalog_manager.cc:5697] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:22.109797  5403 consensus_queue.cc:1048] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:22.111476  4711 catalog_manager.cc:5697] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:22.099169  4855 leader_election.cc:304] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:22.130009  5086 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:22.130585  5086 ts_tablet_manager.cc:1403] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.218s	user 0.014s	sys 0.003s
I20260709 12:29:22.134181  5464 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:22.135124  5464 consensus_queue.cc:1048] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:22.124042  5481 raft_consensus.cc:2804] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:22.137454  5481 raft_consensus.cc:493] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.137791  5481 raft_consensus.cc:3060] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.138501  5082 tablet_bootstrap.cc:654] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.156395  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:22.157217  5083 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:22.157863  5083 ts_tablet_manager.cc:1403] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.144s	user 0.014s	sys 0.013s
I20260709 12:29:22.162804  5449 consensus_queue.cc:1048] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:22.177481  4778 leader_election.cc:343] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.186215  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:22.186985  5038 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.169345  5456 raft_consensus.cc:697] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:22.191718  5083 raft_consensus.cc:359] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.194384  5083 raft_consensus.cc:385] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.193136  5086 raft_consensus.cc:359] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.195688  5086 raft_consensus.cc:385] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.198007  5091 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:22.198587  5091 ts_tablet_manager.cc:1403] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.368s	user 0.054s	sys 0.069s
I20260709 12:29:22.195972  5086 raft_consensus.cc:740] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.194733  5083 raft_consensus.cc:740] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.183632  5211 consensus_queue.cc:1048] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:29:22.209124  5038 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:22.214793  4962 raft_consensus.cc:2468] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:22.204774  5471 raft_consensus.cc:2749] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.225051  5083 consensus_queue.cc:260] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.227542  5086 consensus_queue.cc:260] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.219732  5481 raft_consensus.cc:515] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.240556  5083 ts_tablet_manager.cc:1434] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.002s
I20260709 12:29:22.241015  5466 raft_consensus.cc:493] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.247834  5086 ts_tablet_manager.cc:1434] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.117s	user 0.004s	sys 0.004s
I20260709 12:29:22.241827  5466 raft_consensus.cc:515] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.250630  5466 leader_election.cc:290] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:22.250861  5082 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:22.251593  5082 ts_tablet_manager.cc:1403] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.225s	user 0.009s	sys 0.012s
I20260709 12:29:22.258906  4780 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.260640  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.261319  5038 raft_consensus.cc:2468] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 0.
I20260709 12:29:22.261308  5091 raft_consensus.cc:359] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.262171  5091 raft_consensus.cc:385] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.262467  5091 raft_consensus.cc:740] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.263092  5091 consensus_queue.cc:260] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.264640  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:22.267781  4929 leader_election.cc:343] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.268534  4929 leader_election.cc:304] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:22.275794  5091 ts_tablet_manager.cc:1434] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.001s
I20260709 12:29:22.274766  5449 raft_consensus.cc:2804] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:22.286041  5449 raft_consensus.cc:493] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.286471  5449 raft_consensus.cc:3060] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.284353  5082 raft_consensus.cc:359] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.275666  5398 raft_consensus.cc:2804] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.292582  5398 raft_consensus.cc:697] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:22.293221  5083 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Bootstrap starting.
I20260709 12:29:22.293555  5398 consensus_queue.cc:237] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.294062  5394 raft_consensus.cc:493] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.297662  5394 raft_consensus.cc:515] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.299961  5394 leader_election.cc:290] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:22.300755  5082 raft_consensus.cc:385] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.301093  5082 raft_consensus.cc:740] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.302858  5461 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.303419  5461 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.302867  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:22.305049  5086 tablet_bootstrap.cc:492] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:22.306182  5461 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:22.305094  4814 raft_consensus.cc:3060] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.307148  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:22.308982  4778 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.257969  5456 consensus_queue.cc:237] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.311159  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.311782  5433 raft_consensus.cc:493] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.311985  5032 raft_consensus.cc:2468] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:22.312361  5433 raft_consensus.cc:515] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.311076  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80aa6fe92f51449ab69485480db26075" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.313858  5030 raft_consensus.cc:2468] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:22.314819  5091 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:22.314942  5433 leader_election.cc:290] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:22.317749  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80aa6fe92f51449ab69485480db26075" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:22.322077  5449 raft_consensus.cc:515] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:22.324177  4855 leader_election.cc:343] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.328915  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e9766db6054724a5055336fa6baa91" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
W20260709 12:29:22.332285  5001 leader_election.cc:343] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.297991  5481 leader_election.cc:290] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:22.333555  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:22.301836  5082 consensus_queue.cc:260] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.338076  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e9766db6054724a5055336fa6baa91" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:22.339545  4855 leader_election.cc:304] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:22.324834  4780 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.340992  5394 raft_consensus.cc:2804] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:22.341589  5394 raft_consensus.cc:493] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.342005  5394 raft_consensus.cc:3060] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.344782  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:22.345289  5083 tablet_bootstrap.cc:654] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.345484  5031 raft_consensus.cc:3060] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.348328  5457 raft_consensus.cc:2804] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:22.349869  5457 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.350190  5457 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.350554  5449 leader_election.cc:290] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:22.349388  5003 leader_election.cc:343] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.352463  5003 leader_election.cc:304] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.353142  5086 tablet_bootstrap.cc:654] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.354364  5433 raft_consensus.cc:2749] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.354700  4812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
W20260709 12:29:22.356175  4929 leader_election.cc:343] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.352080  5091 tablet_bootstrap.cc:654] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.372942  5086 tablet_bootstrap.cc:492] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:22.373531  5086 ts_tablet_manager.cc:1403] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.069s	user 0.003s	sys 0.009s
I20260709 12:29:22.375882  5082 ts_tablet_manager.cc:1434] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.000s
I20260709 12:29:22.395200  5082 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:22.385258  5086 raft_consensus.cc:359] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.387042  4713 catalog_manager.cc:5697] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:22.349148  4855 leader_election.cc:343] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:22.401094  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 52007us
I20260709 12:29:22.395455  5083 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6: No bootstrap required, opened a new log
I20260709 12:29:22.402709  5083 ts_tablet_manager.cc:1403] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent bootstrapping tablet: real 0.110s	user 0.010s	sys 0.006s
I20260709 12:29:22.415562  5083 raft_consensus.cc:359] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.425213  5083 raft_consensus.cc:385] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.387560  4814 raft_consensus.cc:2468] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:22.425695  5083 raft_consensus.cc:740] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.426688  5083 consensus_queue.cc:260] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.418259  5091 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:22.427856  5091 ts_tablet_manager.cc:1403] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.148s	user 0.013s	sys 0.005s
I20260709 12:29:22.430725  5091 raft_consensus.cc:359] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.427055  4855 leader_election.cc:304] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:22.432088  5481 raft_consensus.cc:2804] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.427990  5457 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.432500  5091 raft_consensus.cc:385] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.442700  5091 raft_consensus.cc:740] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.439568  5457 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:22.440039  5456 raft_consensus.cc:493] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.432621  5481 raft_consensus.cc:697] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:22.444595  5091 consensus_queue.cc:260] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.444804  5481 consensus_queue.cc:237] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.424775  5082 tablet_bootstrap.cc:654] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.441365  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:22.447324  5035 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.442361  5083 ts_tablet_manager.cc:1434] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 12:29:22.439095  5031 raft_consensus.cc:2468] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:22.449712  4929 leader_election.cc:304] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:22.450474  5449 raft_consensus.cc:2804] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.450939  5449 raft_consensus.cc:697] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:22.451237  5091 ts_tablet_manager.cc:1434] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:29:22.452356  5091 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:22.451766  5449 consensus_queue.cc:237] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.419101  5086 raft_consensus.cc:385] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.456522  5086 raft_consensus.cc:740] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.457413  5086 consensus_queue.cc:260] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.460269  5091 tablet_bootstrap.cc:654] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.436704  5394 raft_consensus.cc:515] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.463104  5394 leader_election.cc:290] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:22.443431  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
W20260709 12:29:22.465466  4778 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.466421  5082 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:22.466920  5082 ts_tablet_manager.cc:1403] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.000s
I20260709 12:29:22.467435  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80aa6fe92f51449ab69485480db26075" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:22.466657  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80aa6fe92f51449ab69485480db26075" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:22.468124  4955 raft_consensus.cc:3060] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.468664  5031 raft_consensus.cc:3060] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.469609  5082 raft_consensus.cc:359] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.470698  5082 raft_consensus.cc:385] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.470990  5082 raft_consensus.cc:740] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.472134  5082 consensus_queue.cc:260] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.473117  5456 raft_consensus.cc:515] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.473861  4713 catalog_manager.cc:5697] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:22.474696  5082 ts_tablet_manager.cc:1434] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:22.475886  5086 ts_tablet_manager.cc:1434] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
I20260709 12:29:22.476511  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:22.477023  5086 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:22.477648  5082 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:22.477203  4890 raft_consensus.cc:2468] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:22.481887  4778 leader_election.cc:304] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:22.482635  5125 raft_consensus.cc:2804] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:22.482978  5125 raft_consensus.cc:493] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:22.483278  5125 raft_consensus.cc:3060] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.489472  5456 leader_election.cc:290] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:22.494238  5125 raft_consensus.cc:515] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.497097  5125 leader_election.cc:290] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:22.501139  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:22.501832  4890 raft_consensus.cc:3060] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.502725  5457 consensus_queue.cc:1048] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:22.501196  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:22.509290  5399 consensus_queue.cc:1048] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:22.512976  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:22.515738  5082 tablet_bootstrap.cc:654] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.529212  5035 raft_consensus.cc:2468] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
W20260709 12:29:22.532534  4780 leader_election.cc:343] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:22.542842  4780 leader_election.cc:343] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.531962  5460 raft_consensus.cc:493] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.549841  5460 raft_consensus.cc:515] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.552418  4890 raft_consensus.cc:2468] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:22.553864  4778 leader_election.cc:304] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:22.555161  5031 raft_consensus.cc:2468] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:22.557327  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:22.561606  5403 raft_consensus.cc:2804] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.562263  5403 raft_consensus.cc:697] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:22.563213  5403 consensus_queue.cc:237] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.565308  4855 leader_election.cc:304] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:22.565706  5003 leader_election.cc:343] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.566962  5394 raft_consensus.cc:2804] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.567201  5086 tablet_bootstrap.cc:654] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.567509  5394 raft_consensus.cc:697] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:22.571403  4780 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.573050  5091 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:22.573757  5091 ts_tablet_manager.cc:1403] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.122s	user 0.019s	sys 0.019s
I20260709 12:29:22.568482  5394 consensus_queue.cc:237] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:22.577226  5460 leader_election.cc:290] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:22.579782  4955 raft_consensus.cc:2468] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:22.580367  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:22.593000  5461 raft_consensus.cc:2804] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.597860  5461 raft_consensus.cc:697] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:22.598855  5461 consensus_queue.cc:237] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.584146  5433 raft_consensus.cc:493] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.594530  5494 consensus_queue.cc:1048] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
W20260709 12:29:22.609615  5003 leader_election.cc:343] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.610136  5003 leader_election.cc:304] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:22.613319  4710 catalog_manager.cc:5697] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 1 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:22.604768  5433 raft_consensus.cc:515] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.617230  5433 leader_election.cc:290] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:22.619171  5456 consensus_queue.cc:1048] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:22.613369  5472 raft_consensus.cc:2749] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.622054  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:22.623845  5003 leader_election.cc:343] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.646042  5082 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:22.646617  5082 ts_tablet_manager.cc:1403] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.171s	user 0.018s	sys 0.001s
I20260709 12:29:22.649510  5082 raft_consensus.cc:359] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.650182  5082 raft_consensus.cc:385] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.650564  5082 raft_consensus.cc:740] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.651253  5082 consensus_queue.cc:260] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.676981  5091 raft_consensus.cc:359] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.681617  5091 raft_consensus.cc:385] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.682901  5082 ts_tablet_manager.cc:1434] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.002s
I20260709 12:29:22.681924  5091 raft_consensus.cc:740] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.686120  5091 consensus_queue.cc:260] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:22.685482  4712 catalog_manager.cc:5697] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:22.705802  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:22.713807  5082 tablet_bootstrap.cc:492] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:22.718521  5091 ts_tablet_manager.cc:1434] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.144s	user 0.008s	sys 0.002s
I20260709 12:29:22.703708  5481 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.722913  5091 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:22.733755  5481 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.727092  4713 catalog_manager.cc:5697] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:22.709415  5403 raft_consensus.cc:493] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.739800  5403 raft_consensus.cc:515] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:22.740876  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:22.741930  5403 leader_election.cc:290] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
W20260709 12:29:22.713356  5001 leader_election.cc:343] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.730288  5082 tablet_bootstrap.cc:654] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.745013  5091 tablet_bootstrap.cc:654] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.746306  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.747198  5032 raft_consensus.cc:2393] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:22.747560  5001 leader_election.cc:304] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.755548  5532 raft_consensus.cc:2749] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.797200  5457 raft_consensus.cc:493] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.801147  5457 raft_consensus.cc:515] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:22.801972  5481 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:22.803558  5457 leader_election.cc:290] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:22.809576  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:22.810967  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:22.811782  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea31c506847448087d1d664687c1d57" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.812665  5031 raft_consensus.cc:2393] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 1: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:22.826893  4855 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:22.830699  5399 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.835415  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:22.856882  4811 raft_consensus.cc:3060] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.857867  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:22.859947  5452 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.862561  5452 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:22.871274  4780 leader_election.cc:304] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:22.871976  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:22.872988  5034 raft_consensus.cc:2393] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 1: Already voted for candidate 0ec932ee892248cda1f4e99e4ba7887e in this term.
I20260709 12:29:22.874343  5452 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:22.877877  4780 leader_election.cc:343] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.878943  5457 consensus_queue.cc:1048] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:22.891067  4778 leader_election.cc:343] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:22.893172  4713 catalog_manager.cc:5697] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:22.897260  5082 tablet_bootstrap.cc:492] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:22.898025  5082 ts_tablet_manager.cc:1403] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.184s	user 0.010s	sys 0.009s
I20260709 12:29:22.901991  4778 leader_election.cc:304] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:22.902799  5091 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:22.903523  5091 ts_tablet_manager.cc:1403] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.181s	user 0.013s	sys 0.007s
I20260709 12:29:22.906086  5125 consensus_queue.cc:1048] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:22.907636  5467 raft_consensus.cc:2749] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.905442  5399 raft_consensus.cc:2749] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:22.906126  5481 consensus_queue.cc:1048] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:29:22.900295  5461 consensus_queue.cc:1048] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:29:22.916126  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:22.938104  4778 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 56721.7us
I20260709 12:29:22.953308  4855 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:22.985087  5086 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:22.985574  5086 ts_tablet_manager.cc:1403] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.509s	user 0.038s	sys 0.035s
I20260709 12:29:22.960028  5541 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.984395  5461 consensus_queue.cc:1048] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:29:23.028527  5464 raft_consensus.cc:2749] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.029587  5471 consensus_queue.cc:1048] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:23.067619  5091 raft_consensus.cc:359] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.068518  5091 raft_consensus.cc:385] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.068804  5091 raft_consensus.cc:740] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.069473  5091 consensus_queue.cc:260] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.071631  5091 ts_tablet_manager.cc:1434] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.168s	user 0.005s	sys 0.001s
I20260709 12:29:23.055296  5307 consensus_queue.cc:1048] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:29:23.092775  5086 raft_consensus.cc:359] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.075578  5082 raft_consensus.cc:359] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.101742  5086 raft_consensus.cc:385] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:29:23.108918  4855 consensus_peers.cc:597] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:23.126271  5086 raft_consensus.cc:740] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.127164  5086 consensus_queue.cc:260] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.133651  5086 ts_tablet_manager.cc:1434] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.148s	user 0.010s	sys 0.001s
I20260709 12:29:23.134811  5086 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:23.097101  5091 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:23.136097  4780 consensus_peers.cc:597] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:23.140380  5082 raft_consensus.cc:385] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.140793  5082 raft_consensus.cc:740] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.141821  5082 consensus_queue.cc:260] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.148423  5082 ts_tablet_manager.cc:1434] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.250s	user 0.013s	sys 0.007s
I20260709 12:29:23.151322  5125 consensus_queue.cc:1048] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:23.152412  4713 catalog_manager.cc:5697] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.155092  5086 tablet_bootstrap.cc:654] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.157194  5082 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
I20260709 12:29:23.166975  5394 consensus_queue.cc:1048] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:23.169714  5082 tablet_bootstrap.cc:654] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.174305  5541 raft_consensus.cc:2749] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.194118  5471 raft_consensus.cc:493] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.212419  5471 raft_consensus.cc:515] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.215155  5125 consensus_queue.cc:1048] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:23.221804  5471 leader_election.cc:290] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:23.224138  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:23.232182  5091 tablet_bootstrap.cc:654] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.243000  5086 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:23.243642  5086 ts_tablet_manager.cc:1403] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.109s	user 0.011s	sys 0.020s
I20260709 12:29:23.234290  5403 consensus_queue.cc:1048] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:29:23.241825  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47bf01353704752b89641cdf6e88e53" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:23.252671  4927 leader_election.cc:304] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:23.253790  5449 raft_consensus.cc:2749] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.237206  5399 consensus_queue.cc:1048] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:29:23.246582  5086 raft_consensus.cc:359] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.262704  5086 raft_consensus.cc:385] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.263142  5086 raft_consensus.cc:740] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.264012  5086 consensus_queue.cc:260] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.271363  5086 ts_tablet_manager.cc:1434] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 12:29:23.272521  5086 tablet_bootstrap.cc:492] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:23.293378  5082 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:23.294045  5082 ts_tablet_manager.cc:1403] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.137s	user 0.010s	sys 0.010s
I20260709 12:29:23.300668  5082 raft_consensus.cc:359] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.301384  5082 raft_consensus.cc:385] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.301640  5082 raft_consensus.cc:740] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.302327  5082 consensus_queue.cc:260] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.319657  5494 raft_consensus.cc:493] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.320441  5494 raft_consensus.cc:515] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.327569  5082 ts_tablet_manager.cc:1434] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:29:23.326615  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e9766db6054724a5055336fa6baa91" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:23.336094  5086 tablet_bootstrap.cc:654] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.341138  5082 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Bootstrap starting.
W20260709 12:29:23.344131  4855 leader_election.cc:343] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.366010  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e9766db6054724a5055336fa6baa91" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:23.353371  5494 leader_election.cc:290] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:23.389125  5030 raft_consensus.cc:2468] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:23.411106  5086 tablet_bootstrap.cc:492] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:23.411696  5086 ts_tablet_manager.cc:1403] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.139s	user 0.010s	sys 0.006s
I20260709 12:29:23.413345  5091 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: No bootstrap required, opened a new log
I20260709 12:29:23.413893  5091 ts_tablet_manager.cc:1403] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.317s	user 0.016s	sys 0.013s
I20260709 12:29:23.416105  5082 tablet_bootstrap.cc:654] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.421286  5091 raft_consensus.cc:359] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.421900  5091 raft_consensus.cc:385] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.422214  5091 raft_consensus.cc:740] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.422477  4855 leader_election.cc:304] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:23.422993  5091 consensus_queue.cc:260] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.427001  5481 raft_consensus.cc:2804] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.427443  5481 raft_consensus.cc:493] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.427788  5481 raft_consensus.cc:3060] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.439239  5494 raft_consensus.cc:493] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.439944  5494 raft_consensus.cc:515] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.442339  5494 leader_election.cc:290] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:23.442957  5481 raft_consensus.cc:515] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.445204  5481 leader_election.cc:290] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:23.447993  5091 ts_tablet_manager.cc:1434] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.034s	user 0.001s	sys 0.005s
I20260709 12:29:23.451186  5086 raft_consensus.cc:359] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.452857  5086 raft_consensus.cc:385] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.453235  5086 raft_consensus.cc:740] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.454072  5086 consensus_queue.cc:260] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.454739  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:23.455494  5030 raft_consensus.cc:2468] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:23.456393  5086 ts_tablet_manager.cc:1434] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.003s
I20260709 12:29:23.457559  5086 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:23.458901  5082 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: No bootstrap required, opened a new log
I20260709 12:29:23.461612  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e9766db6054724a5055336fa6baa91" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:23.461980  5082 ts_tablet_manager.cc:1403] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent bootstrapping tablet: real 0.128s	user 0.020s	sys 0.011s
I20260709 12:29:23.462291  5030 raft_consensus.cc:3060] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.460166  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:23.464591  4811 raft_consensus.cc:2468] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:23.460355  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e9766db6054724a5055336fa6baa91" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:23.465569  4958 raft_consensus.cc:3060] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.465369  5082 raft_consensus.cc:359] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.467048  5082 raft_consensus.cc:385] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.467329  5082 raft_consensus.cc:740] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.467957  5082 consensus_queue.cc:260] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.470221  5082 ts_tablet_manager.cc:1434] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:23.461236  4855 leader_election.cc:304] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:23.480312  5086 tablet_bootstrap.cc:654] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.486574  4958 raft_consensus.cc:2468] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:23.491093  5494 raft_consensus.cc:2804] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.500669  5494 raft_consensus.cc:493] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.501087  5494 raft_consensus.cc:3060] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.500897  4855 leader_election.cc:304] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:23.502004  5399 raft_consensus.cc:2804] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.502507  5399 raft_consensus.cc:697] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:23.503391  5399 consensus_queue.cc:237] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:23.511565  5494 raft_consensus.cc:515] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.516635  5481 raft_consensus.cc:493] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.517220  5481 raft_consensus.cc:515] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.519556  5481 leader_election.cc:290] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:23.522330  5494 leader_election.cc:290] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:23.497489  5030 raft_consensus.cc:2468] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:23.522784  4813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:23.523530  4813 raft_consensus.cc:2468] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 0.
I20260709 12:29:23.526973  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:23.527617  4811 raft_consensus.cc:3060] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.528983  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:23.529646  4855 leader_election.cc:304] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:23.530602  4855 leader_election.cc:343] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.531910  5494 raft_consensus.cc:2804] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.532349  5494 raft_consensus.cc:493] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.532665  5494 raft_consensus.cc:3060] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.537113  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:23.537819  5034 raft_consensus.cc:3060] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.544623  5494 raft_consensus.cc:515] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.549337  4814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:23.549990  4814 raft_consensus.cc:3060] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.552628  4711 catalog_manager.cc:5697] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.562556  4814 raft_consensus.cc:2468] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:23.565508  5494 leader_election.cc:290] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:23.567009  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:23.568195  4855 leader_election.cc:304] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
W20260709 12:29:23.569170  4855 leader_election.cc:343] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.574591  5494 raft_consensus.cc:2804] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.579011  5494 raft_consensus.cc:697] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:23.580271  5494 consensus_queue.cc:237] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.603332  5034 raft_consensus.cc:2468] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:23.605291  4855 leader_election.cc:304] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:23.601902  4711 catalog_manager.cc:5697] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.607072  5541 raft_consensus.cc:2804] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.610608  5541 raft_consensus.cc:697] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:23.611546  5541 consensus_queue.cc:237] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:23.606415  4811 raft_consensus.cc:2468] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:23.608883  5086 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:23.617359  5086 ts_tablet_manager.cc:1403] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.160s	user 0.009s	sys 0.003s
I20260709 12:29:23.618889  4958 raft_consensus.cc:3060] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.620885  5086 raft_consensus.cc:359] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.621651  5086 raft_consensus.cc:385] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.621903  5086 raft_consensus.cc:740] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.622553  5086 consensus_queue.cc:260] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.626401  5086 ts_tablet_manager.cc:1434] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:23.638808  4960 raft_consensus.cc:3060] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.656343  5471 raft_consensus.cc:493] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:23.656968  5471 raft_consensus.cc:515] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.661584  5471 leader_election.cc:290] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:23.661684  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:23.662575  5086 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed: Bootstrap starting.
I20260709 12:29:23.665468  4713 catalog_manager.cc:5697] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:23.673776  5086 tablet_bootstrap.cc:654] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.684785  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:23.690060  5086 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed: No bootstrap required, opened a new log
I20260709 12:29:23.690647  5086 ts_tablet_manager.cc:1403] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 12:29:23.693534  5086 raft_consensus.cc:359] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.694283  5086 raft_consensus.cc:385] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.694551  5086 raft_consensus.cc:740] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.695204  5086 consensus_queue.cc:260] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.697855  5086 ts_tablet_manager.cc:1434] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.002s
I20260709 12:29:23.702343  5449 raft_consensus.cc:493] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.702965  5449 raft_consensus.cc:515] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:23.703411  4927 leader_election.cc:304] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:23.705338  5449 leader_election.cc:290] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:23.705569  5471 raft_consensus.cc:2749] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:23.707065  4811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:23.707119  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:23.708156  4811 raft_consensus.cc:2393] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 1: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:23.711452  4927 leader_election.cc:304] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:23.712373  5471 raft_consensus.cc:3060] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.719456  5471 raft_consensus.cc:2749] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:23.782945  4760 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:23.902247  5539 raft_consensus.cc:493] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.902835  5539 raft_consensus.cc:515] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.904903  5539 leader_election.cc:290] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:23.907378  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:23.908102  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:23.908493  4960 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:23.908967  5034 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 0.
I20260709 12:29:23.909834  4780 leader_election.cc:304] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:23.910532  5403 raft_consensus.cc:2804] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.910867  5403 raft_consensus.cc:493] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.911288  5403 raft_consensus.cc:3060] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.917762  5403 raft_consensus.cc:515] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.919606  5403 leader_election.cc:290] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:23.920509  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:23.920707  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:23.921113  4960 raft_consensus.cc:3060] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.921253  5034 raft_consensus.cc:3060] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.927964  5034 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:23.928633  4960 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 1.
I20260709 12:29:23.929225  4780 leader_election.cc:304] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:23.930018  5403 raft_consensus.cc:2804] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.930620  5403 raft_consensus.cc:697] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:23.931428  5403 consensus_queue.cc:237] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:23.941582  4713 catalog_manager.cc:5697] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 1 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:24.034726  5494 consensus_queue.cc:1048] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.050676  5541 consensus_queue.cc:1048] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.058511  5494 consensus_queue.cc:1048] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.068899  5494 consensus_queue.cc:1048] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.176683  5481 consensus_queue.cc:1048] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.194152  5481 consensus_queue.cc:1048] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.417433  5125 consensus_queue.cc:1048] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.434136  5403 consensus_queue.cc:1048] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:25.489305 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
W20260709 12:29:25.503804  4929 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:25.515651  4929 consensus_peers.cc:597] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.530558  4929 consensus_peers.cc:597] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.538403 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:25.539669 24862 tablet_replica.cc:333] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.540625 24862 raft_consensus.cc:2243] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.541215 24862 raft_consensus.cc:2272] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.543815 24862 tablet_replica.cc:333] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.544669 24862 raft_consensus.cc:2243] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.545753 24862 raft_consensus.cc:2272] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.547986 24862 tablet_replica.cc:333] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.548914 24862 raft_consensus.cc:2243] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.549458 24862 raft_consensus.cc:2272] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.551906 24862 tablet_replica.cc:333] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.552825 24862 raft_consensus.cc:2243] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.553658 24862 raft_consensus.cc:2272] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.556032 24862 tablet_replica.cc:333] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.556741 24862 raft_consensus.cc:2243] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.557607 24862 raft_consensus.cc:2272] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.559523 24862 tablet_replica.cc:333] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.560141 24862 raft_consensus.cc:2243] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.561002 24862 raft_consensus.cc:2272] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.563123 24862 tablet_replica.cc:333] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.563953 24862 raft_consensus.cc:2243] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.565102 24862 raft_consensus.cc:2272] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.567031 24862 tablet_replica.cc:333] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.567826 24862 raft_consensus.cc:2243] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.568409 24862 raft_consensus.cc:2272] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.570588 24862 tablet_replica.cc:333] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.571240 24862 raft_consensus.cc:2243] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.572016 24862 raft_consensus.cc:2272] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.573982 24862 tablet_replica.cc:333] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.574630 24862 raft_consensus.cc:2243] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.575062 24862 raft_consensus.cc:2272] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.577288 24862 tablet_replica.cc:333] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.577976 24862 raft_consensus.cc:2243] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.578725 24862 raft_consensus.cc:2272] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.580643 24862 tablet_replica.cc:333] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.581280 24862 raft_consensus.cc:2243] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.582093 24862 raft_consensus.cc:2272] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.583971 24862 tablet_replica.cc:333] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.584574 24862 raft_consensus.cc:2243] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.585011 24862 raft_consensus.cc:2272] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.586826 24862 tablet_replica.cc:333] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.587455 24862 raft_consensus.cc:2243] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.588511 24862 raft_consensus.cc:2272] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.591075 24862 tablet_replica.cc:333] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.591850 24862 raft_consensus.cc:2243] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.592847 24862 raft_consensus.cc:2272] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.594658 24862 tablet_replica.cc:333] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.595305 24862 raft_consensus.cc:2243] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.596160 24862 raft_consensus.cc:2272] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.598789 24862 tablet_replica.cc:333] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.599607 24862 raft_consensus.cc:2243] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.600551 24862 raft_consensus.cc:2272] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.603091 24862 tablet_replica.cc:333] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.603724 24862 raft_consensus.cc:2243] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.604770 24862 raft_consensus.cc:2272] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.606978 24862 tablet_replica.cc:333] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.607659 24862 raft_consensus.cc:2243] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.608506 24862 raft_consensus.cc:2272] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.610620 24862 tablet_replica.cc:333] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.611282 24862 raft_consensus.cc:2243] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.611788 24862 raft_consensus.cc:2272] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.614171 24862 tablet_replica.cc:333] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.615028 24862 raft_consensus.cc:2243] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.616173 24862 raft_consensus.cc:2272] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.619465 24862 tablet_replica.cc:333] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.620283 24862 raft_consensus.cc:2243] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.620769 24862 raft_consensus.cc:2272] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.623698 24862 tablet_replica.cc:333] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.624563 24862 raft_consensus.cc:2243] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.625522 24862 raft_consensus.cc:2272] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.628846 24862 tablet_replica.cc:333] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.629731 24862 raft_consensus.cc:2243] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:29:25.630801  4855 consensus_peers.cc:597] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.630868 24862 raft_consensus.cc:2272] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.631829  4855 consensus_peers.cc:597] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.634181 24862 tablet_replica.cc:333] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.634934 24862 raft_consensus.cc:2243] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.635404 24862 raft_consensus.cc:2272] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.635843  5003 consensus_peers.cc:597] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.637611 24862 tablet_replica.cc:333] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.638295 24862 raft_consensus.cc:2243] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.638770 24862 raft_consensus.cc:2272] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.641471 24862 tablet_replica.cc:333] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.642086 24862 raft_consensus.cc:2243] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.642961 24862 raft_consensus.cc:2272] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.643992  4855 consensus_peers.cc:597] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.645692 24862 tablet_replica.cc:333] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.646361 24862 raft_consensus.cc:2243] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.646972 24862 raft_consensus.cc:2272] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.649235 24862 tablet_replica.cc:333] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.649853 24862 raft_consensus.cc:2243] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.650731 24862 raft_consensus.cc:2272] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.652851 24862 tablet_replica.cc:333] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.653501 24862 raft_consensus.cc:2243] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.654003 24862 raft_consensus.cc:2272] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.657277 24862 tablet_replica.cc:333] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.658157 24862 raft_consensus.cc:2243] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.658658 24862 raft_consensus.cc:2272] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.660480  5003 consensus_peers.cc:597] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.661351 24862 tablet_replica.cc:333] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.662207 24862 raft_consensus.cc:2243] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.663208 24862 raft_consensus.cc:2272] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.666322 24862 tablet_replica.cc:333] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.667119 24862 raft_consensus.cc:2243] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:29:25.667737  4855 consensus_peers.cc:597] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.668177 24862 raft_consensus.cc:2272] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.669525  5003 consensus_peers.cc:597] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.670708 24862 tablet_replica.cc:333] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.671427 24862 raft_consensus.cc:2243] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.672297 24862 raft_consensus.cc:2272] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.674923 24862 tablet_replica.cc:333] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.675601 24862 raft_consensus.cc:2243] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.676088 24862 raft_consensus.cc:2272] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.678139 24862 tablet_replica.cc:333] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.678658 24862 raft_consensus.cc:2243] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.679029 24862 raft_consensus.cc:2272] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.681205 24862 tablet_replica.cc:333] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.681739 24862 raft_consensus.cc:2243] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.682448 24862 raft_consensus.cc:2272] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.684309 24862 tablet_replica.cc:333] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.684865 24862 raft_consensus.cc:2243] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.685261 24862 raft_consensus.cc:2272] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.687362 24862 tablet_replica.cc:333] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.687947 24862 raft_consensus.cc:2243] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.688772 24862 raft_consensus.cc:2272] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.691531 24862 tablet_replica.cc:333] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.692314 24862 raft_consensus.cc:2243] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.693318 24862 raft_consensus.cc:2272] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.694360  4855 consensus_peers.cc:597] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.696128 24862 tablet_replica.cc:333] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.696875 24862 raft_consensus.cc:2243] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.697811 24862 raft_consensus.cc:2272] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.700364 24862 tablet_replica.cc:333] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.701030 24862 raft_consensus.cc:2243] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.701517 24862 raft_consensus.cc:2272] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.703685 24862 tablet_replica.cc:333] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.704386 24862 raft_consensus.cc:2243] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.705132 24862 raft_consensus.cc:2272] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.707165 24862 tablet_replica.cc:333] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.707875 24862 raft_consensus.cc:2243] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.708425 24862 raft_consensus.cc:2272] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.710517 24862 tablet_replica.cc:333] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: stopping tablet replica
I20260709 12:29:25.711206 24862 raft_consensus.cc:2243] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.711903 24862 raft_consensus.cc:2272] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.731447  4855 consensus_peers.cc:597] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.790832 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
W20260709 12:29:25.808415  5003 consensus_peers.cc:597] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.868223 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:25.877414  5589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:25.879050  4929 consensus_peers.cc:597] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.880226  5590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:25.883592  5592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:25.884711 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:25.890587 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:25.890924 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:25.891204 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600165891175 us; error 0 us; skew 500 ppm
I20260709 12:29:25.891992 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:25.895253 24862 webserver.cc:533] Webserver started at http://127.24.71.129:34921/ using document root <none> and password file <none>
I20260709 12:29:25.895813 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:25.896049 24862 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:29:25.900409 24862 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:25.908784  5597 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:25.909834 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:25.910281 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "0ec932ee892248cda1f4e99e4ba7887e"
format_stamp: "Formatted at 2026-07-09 12:29:12 on dist-test-slave-3zgk"
I20260709 12:29:25.911307 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:29:25.917428  4929 consensus_peers.cc:597] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.918010  4929 consensus_peers.cc:597] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.923810 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:25.925720 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260709 12:29:25.928043  5003 consensus_peers.cc:597] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.930537 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260709 12:29:25.933889  5003 consensus_peers.cc:597] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.937091  5605 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:29:25.979627  4855 consensus_peers.cc:597] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:26.213207 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:29:26.213641 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.278s	user 0.005s	sys 0.004s
I20260709 12:29:26.214010 24862 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:29:26.220279  5605 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.236488  5605 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.237484  5605 tablet_bootstrap.cc:492] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.238132  5605 ts_tablet_manager.cc:1403] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 12:29:26.241421  5605 raft_consensus.cc:359] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.242089  5605 raft_consensus.cc:740] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.242961  5605 consensus_queue.cc:260] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.246485  5605 ts_tablet_manager.cc:1434] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:29:26.247323  5605 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.272816  5605 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.273751  5605 tablet_bootstrap.cc:492] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.274533  5605 ts_tablet_manager.cc:1403] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 12:29:26.277063  5605 raft_consensus.cc:359] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.277729  5605 raft_consensus.cc:740] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.278405  5605 consensus_queue.cc:260] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.280303  5605 ts_tablet_manager.cc:1434] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:29:26.281051  5605 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.296082  5605 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.297055  5605 tablet_bootstrap.cc:492] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.297685  5605 ts_tablet_manager.cc:1403] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:29:26.300177  5605 raft_consensus.cc:359] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.300906  5605 raft_consensus.cc:740] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.301709  5605 consensus_queue.cc:260] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.303522  5605 ts_tablet_manager.cc:1434] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:26.304328  5605 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.323972  5605 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.325176  5605 tablet_bootstrap.cc:492] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.325948  5605 ts_tablet_manager.cc:1403] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:29:26.329150  5605 raft_consensus.cc:359] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.329912  5605 raft_consensus.cc:740] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.330991  5605 consensus_queue.cc:260] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.332572  5605 ts_tablet_manager.cc:1434] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:26.333312  5605 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.360172  5605 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.361150  5605 tablet_bootstrap.cc:492] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.361742  5605 ts_tablet_manager.cc:1403] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.004s
I20260709 12:29:26.364099  5605 raft_consensus.cc:359] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.364760  5605 raft_consensus.cc:740] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.365397  5605 consensus_queue.cc:260] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.366886  5605 ts_tablet_manager.cc:1434] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:26.367561  5605 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.379433  5605 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.380402  5605 tablet_bootstrap.cc:492] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.380963  5605 ts_tablet_manager.cc:1403] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:29:26.382956  5605 raft_consensus.cc:359] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.383494  5605 raft_consensus.cc:740] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.384088  5605 consensus_queue.cc:260] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.385667  5605 ts_tablet_manager.cc:1434] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:26.386406  5605 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.399958  5605 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.401646  5605 tablet_bootstrap.cc:492] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.402366  5605 ts_tablet_manager.cc:1403] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:29:26.405103  5605 raft_consensus.cc:359] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.405973  5605 raft_consensus.cc:740] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.406670  5605 consensus_queue.cc:260] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.410327  5605 ts_tablet_manager.cc:1434] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:26.412215  5605 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.426128  5605 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.426962  5605 tablet_bootstrap.cc:492] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.427668  5605 ts_tablet_manager.cc:1403] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:29:26.430183  5605 raft_consensus.cc:359] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.430972  5605 raft_consensus.cc:740] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.431746  5605 consensus_queue.cc:260] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.433399  5605 ts_tablet_manager.cc:1434] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.004s
I20260709 12:29:26.434309  5605 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.452330  5605 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.453282  5605 tablet_bootstrap.cc:492] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.453860  5605 ts_tablet_manager.cc:1403] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:29:26.456055 24862 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:29:26.456485 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.242s	user 0.226s	sys 0.016s
I20260709 12:29:26.456069  5605 raft_consensus.cc:359] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.457122  5605 raft_consensus.cc:740] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.457890  5605 consensus_queue.cc:260] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.459851  5605 ts_tablet_manager.cc:1434] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 12:29:26.460842  5605 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.480146  5605 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.481254  5605 tablet_bootstrap.cc:492] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.482007  5605 ts_tablet_manager.cc:1403] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:29:26.484684  5605 raft_consensus.cc:359] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.485301  5605 raft_consensus.cc:740] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.486044  5605 consensus_queue.cc:260] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.488368  5605 ts_tablet_manager.cc:1434] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:26.489203  5605 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.510056  5605 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.511168  5605 tablet_bootstrap.cc:492] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.511978  5605 ts_tablet_manager.cc:1403] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 12:29:26.515208  5605 raft_consensus.cc:359] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.515961  5605 raft_consensus.cc:740] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.516804  5605 consensus_queue.cc:260] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.519166  5605 ts_tablet_manager.cc:1434] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:26.520083  5605 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.530320  5471 raft_consensus.cc:493] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.530994  5471 raft_consensus.cc:515] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.533459  5471 leader_election.cc:290] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.535459  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
W20260709 12:29:26.545249  4929 leader_election.cc:336] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111)
I20260709 12:29:26.546679  4929 leader_election.cc:304] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:26.548120  5471 raft_consensus.cc:2749] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.560824  5605 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.562409  5605 tablet_bootstrap.cc:492] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.563045  5605 ts_tablet_manager.cc:1403] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.000s
I20260709 12:29:26.565907  5605 raft_consensus.cc:359] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.566716  5605 raft_consensus.cc:740] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.567500  5605 consensus_queue.cc:260] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.569060  5603 raft_consensus.cc:493] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.569793  5605 ts_tablet_manager.cc:1434] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:26.569658  5603 raft_consensus.cc:515] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.570715  5605 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.573274  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:26.579479  5603 leader_election.cc:290] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
W20260709 12:29:26.580204  5003 leader_election.cc:336] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111)
I20260709 12:29:26.580761  5003 leader_election.cc:304] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:26.581568  5603 raft_consensus.cc:2749] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.589972  5605 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.590977  5605 tablet_bootstrap.cc:492] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.591828  5605 ts_tablet_manager.cc:1403] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.000s
I20260709 12:29:26.594749  5605 raft_consensus.cc:359] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.595494  5605 raft_consensus.cc:740] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.596306  5605 consensus_queue.cc:260] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.598266  5605 ts_tablet_manager.cc:1434] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:26.599088  5605 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.607263 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:44881
I20260709 12:29:26.607378  5681 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:44881 every 8 connection(s)
I20260709 12:29:26.612401  5605 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.613481  5605 tablet_bootstrap.cc:492] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.614390  5605 ts_tablet_manager.cc:1403] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 12:29:26.617116  5605 raft_consensus.cc:359] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.617934  5605 raft_consensus.cc:740] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.618693  5605 consensus_queue.cc:260] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.620661  5605 ts_tablet_manager.cc:1434] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:26.621677  5605 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.643308  5471 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.643767  5541 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.643963  5471 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.644418  5541 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.646390  5605 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.647469  5605 tablet_bootstrap.cc:492] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.647614  5683 heartbeater.cc:344] Connected to a master server at 127.24.71.190:43079
I20260709 12:29:26.648101  5683 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:26.648277  5605 ts_tablet_manager.cc:1403] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.008s
I20260709 12:29:26.649231  5683 heartbeater.cc:507] Master 127.24.71.190:43079 requested a full tablet report, sending...
I20260709 12:29:26.650104  5609 raft_consensus.cc:493] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.650604  5609 raft_consensus.cc:515] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.651353  5605 raft_consensus.cc:359] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.652136  5605 raft_consensus.cc:740] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.648101  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.652662  5609 leader_election.cc:290] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:26.653213  5605 consensus_queue.cc:260] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.656137  5471 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.664806  5449 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.665392  5449 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.667557  5449 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.673218  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.674700  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.675516  4713 ts_manager.cc:194] Re-registered known tserver with Master: 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.673717  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.652541  5541 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:26.670495  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.671430  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.656159  5605 ts_tablet_manager.cc:1434] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:26.681348  5616 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.682714  5449 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.686429  5605 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.686259  5616 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.687919  5449 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:26.690958  4855 leader_election.cc:343] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.691737  4855 leader_election.cc:304] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:26.692509  5609 raft_consensus.cc:2749] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:26.693435  4855 leader_election.cc:343] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.694330  4855 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:26.695406  5541 raft_consensus.cc:2749] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.696614  5616 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.697254  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.697536  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.697817  5449 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.698276  5655 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:26.698629  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.700021  4929 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:26.700090  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.700941  5449 raft_consensus.cc:2804] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:26.701341  4929 leader_election.cc:343] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.703778  4929 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:26.704512  5471 raft_consensus.cc:2749] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.701339  5449 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.703316  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.706471  5449 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:26.709110  4929 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.709600  4929 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:26.710202  5471 raft_consensus.cc:2749] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.713865  5605 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.715071  5605 tablet_bootstrap.cc:492] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.714679  5449 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.715801  5605 ts_tablet_manager.cc:1403] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.004s
I20260709 12:29:26.718966  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:26.719372  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:26.719800  5655 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.722064  5449 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.718963  5605 raft_consensus.cc:359] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.722827  5605 raft_consensus.cc:740] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.724910  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.726327  5616 raft_consensus.cc:493] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
W20260709 12:29:26.726779  5003 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.726888  5616 raft_consensus.cc:515] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.727265  5003 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:26.728427  5472 raft_consensus.cc:2749] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.729131  5616 leader_election.cc:290] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.729713  5655 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:26.730216  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:26.730849  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.731142  4929 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:26.731499  5654 raft_consensus.cc:2468] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:26.732285  5449 raft_consensus.cc:2804] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:26.726119  5605 consensus_queue.cc:260] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.732868  5003 leader_election.cc:304] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:26.733659  5616 raft_consensus.cc:2804] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.734014  5616 raft_consensus.cc:493] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.734018  5449 raft_consensus.cc:697] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:26.734381  5616 raft_consensus.cc:3060] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.734988  5449 consensus_queue.cc:237] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.735657  5472 raft_consensus.cc:493] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.736178  5472 raft_consensus.cc:515] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.738449  5472 leader_election.cc:290] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.740556  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.741585  4955 raft_consensus.cc:2468] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:26.742564  5616 raft_consensus.cc:515] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.745908  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:26.746222  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.747414  5605 ts_tablet_manager.cc:1434] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 12:29:26.748310  5605 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.749301  4713 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50858
I20260709 12:29:26.752380  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:26.753329  5654 raft_consensus.cc:3060] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.754951  5683 heartbeater.cc:499] Master 127.24.71.190:43079 was elected leader, sending a full tablet report...
I20260709 12:29:26.757805  5616 leader_election.cc:290] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
W20260709 12:29:26.760316  5003 leader_election.cc:343] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.764883  5003 leader_election.cc:304] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:26.766111  5616 raft_consensus.cc:2804] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.766662  5616 raft_consensus.cc:493] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.767218  5616 raft_consensus.cc:3060] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.772722  5605 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.773774  5605 tablet_bootstrap.cc:492] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.774430  5654 raft_consensus.cc:2468] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:26.774463  5605 ts_tablet_manager.cc:1403] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.004s
I20260709 12:29:26.777226  5609 raft_consensus.cc:493] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.777855  5609 raft_consensus.cc:515] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.780444  5605 raft_consensus.cc:359] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.781145  5471 raft_consensus.cc:493] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.781601  5605 raft_consensus.cc:740] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.781731  5471 raft_consensus.cc:515] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.782954  5605 consensus_queue.cc:260] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.785051  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:26.785120  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.785984  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.786298  5654 raft_consensus.cc:2393] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
W20260709 12:29:26.787567  4929 leader_election.cc:343] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.788008  4929 leader_election.cc:304] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:26.788865  5608 raft_consensus.cc:2749] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.779970  5609 leader_election.cc:290] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.784050  5471 leader_election.cc:290] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.777272  5616 raft_consensus.cc:515] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.776894  5003 leader_election.cc:304] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:26.792999  5472 raft_consensus.cc:2804] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:26.793470  5472 raft_consensus.cc:697] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:26.794281  5472 consensus_queue.cc:237] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.790828  5605 ts_tablet_manager.cc:1434] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:29:26.797040  5605 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.798978  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:26.799454  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "845a033bb8104895847b349dcb5eeccb" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.799625  4955 raft_consensus.cc:3060] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.800890  4855 leader_election.cc:304] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:26.801479  5616 leader_election.cc:290] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.801999  5609 raft_consensus.cc:3060] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.803049  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827a3f43a1924ae880f566f2f314143f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:26.809497  4955 raft_consensus.cc:2468] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:26.822696  5003 leader_election.cc:304] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
W20260709 12:29:26.823607  5003 leader_election.cc:343] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.823523  5616 raft_consensus.cc:2804] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:26.825084  5616 raft_consensus.cc:697] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:26.825934  5616 consensus_queue.cc:237] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.828357  5607 raft_consensus.cc:493] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.828893  5607 raft_consensus.cc:515] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.831735  5607 leader_election.cc:290] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:26.832844  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.833703  5655 raft_consensus.cc:2468] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:26.835451  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.837929  5610 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.838548  5610 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.839093  4855 leader_election.cc:304] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:26.840263  5541 raft_consensus.cc:2804] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.840651  5541 raft_consensus.cc:493] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.840770  5610 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.841058  5541 raft_consensus.cc:3060] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.841307  5610 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.841787  5610 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.844686  5610 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.844868  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:26.848968  5449 raft_consensus.cc:493] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.849303  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.849488  5449 raft_consensus.cc:515] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.823458  4711 catalog_manager.cc:5697] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.848991  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.851402  5449 leader_election.cc:290] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.853246  5610 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.853806  5610 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.855087  5607 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.855993  5610 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.856666  5449 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.853844  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.857816  5655 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:26.857281  5449 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.859345  4929 leader_election.cc:304] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:26.859953  5609 raft_consensus.cc:2749] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.860450  5471 raft_consensus.cc:2804] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.860769  5449 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.861025  5471 raft_consensus.cc:493] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.861545  5471 raft_consensus.cc:3060] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.857411  5541 raft_consensus.cc:515] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.864393  5541 leader_election.cc:290] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:26.865573  5541 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.865901  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:26.866117  5541 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.866420  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:26.867043  5655 raft_consensus.cc:3060] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.868595  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
W20260709 12:29:26.870157  4929 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.874711  5655 raft_consensus.cc:2468] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:26.875021  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.875438  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.875530  5610 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.876448  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.878436  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:26.879365  4960 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:26.879729  5603 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.880319  5603 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.882462  5603 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.882948  5605 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:26.877354  5541 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.875627  5030 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:26.884044  5616 raft_consensus.cc:493] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.884516  5471 raft_consensus.cc:515] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.884688  5616 raft_consensus.cc:515] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.884056  5605 tablet_bootstrap.cc:492] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:26.876551  5610 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.887063  5605 ts_tablet_manager.cc:1403] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.090s	user 0.025s	sys 0.014s
I20260709 12:29:26.889326  4855 leader_election.cc:304] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:26.890232  5696 raft_consensus.cc:2804] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:26.890400  5653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:26.890875  5696 raft_consensus.cc:697] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:26.891350  5653 raft_consensus.cc:3060] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.891965  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:26.891945  5696 consensus_queue.cc:237] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.892758  5034 raft_consensus.cc:3060] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.894866  5610 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.895881  5541 raft_consensus.cc:493] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.896428  5541 raft_consensus.cc:515] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.897087  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.897689  5030 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:26.899405  5541 leader_election.cc:290] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.899924  5608 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.900262  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.900446  5608 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:26.902016  4855 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.902841  5608 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.903724  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.904091  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:26.903982  5605 raft_consensus.cc:359] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.904738  5605 raft_consensus.cc:740] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.904994  4887 raft_consensus.cc:2468] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:26.907051  5605 consensus_queue.cc:260] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.909742  5653 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:26.913916  5616 leader_election.cc:290] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.910089  5605 ts_tablet_manager.cc:1434] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:29:26.917258  4927 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:26.912226  5655 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:26.918051  5449 raft_consensus.cc:2804] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.918517  5449 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.900840  5034 raft_consensus.cc:2468] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:26.915601  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.857311  5607 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.918865  5449 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.920480  5607 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:26.905567  5003 leader_election.cc:343] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.916929  5608 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.921161  5003 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:26.920044  4855 leader_election.cc:304] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:26.886682  5471 leader_election.cc:290] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.924048  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.924345  5609 raft_consensus.cc:2804] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.924777  5654 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:26.921444  5608 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:26.929131  5003 leader_election.cc:343] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.924822  5609 raft_consensus.cc:493] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.926960  5610 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.929587  5003 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:26.929852  5610 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:26.930711  5003 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.927695  4855 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:26.931980  5610 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.932502  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.932610  5610 raft_consensus.cc:2749] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.930410  5472 raft_consensus.cc:2749] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:26.933274  5654 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:26.933580  5609 raft_consensus.cc:3060] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.934128  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:26.934890  4887 raft_consensus.cc:2468] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:26.918041  5605 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:26.936255  5001 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:26.928332  5449 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:26.932768  5607 raft_consensus.cc:2804] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.944271  5607 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.944766  5472 raft_consensus.cc:2804] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.945287  5472 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.945736  5472 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.946539  5608 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.918008  4929 leader_election.cc:304] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:26.944654  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:26.952723  5541 raft_consensus.cc:493] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.953490  5541 raft_consensus.cc:515] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.954141  5696 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.952549  4712 catalog_manager.cc:5697] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.954723  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.955605  5654 raft_consensus.cc:2393] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:26.954949  4710 catalog_manager.cc:5697] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.955621  5541 leader_election.cc:290] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.956871  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.954720  5696 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:26.958377  4855 leader_election.cc:343] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.959616  5696 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:26.944648  5607 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.960515  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.961475  4887 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.961758  5610 raft_consensus.cc:493] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
W20260709 12:29:26.961956  4855 leader_election.cc:343] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:26.962318  5610 raft_consensus.cc:515] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.962757  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.963368  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.963650  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.964615  5654 raft_consensus.cc:2393] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:26.964643  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.966086  5032 raft_consensus.cc:2468] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:26.968588  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.970832  5603 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.972817  5603 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.972115  5610 leader_election.cc:290] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.944733  5449 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.976929  5003 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
W20260709 12:29:26.977972  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 60029.1us
I20260709 12:29:26.979430  4855 leader_election.cc:304] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:26.980516  5696 raft_consensus.cc:2804] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.980902  5696 raft_consensus.cc:493] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.981263  5696 raft_consensus.cc:3060] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.981716  5471 raft_consensus.cc:2804] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:26.982240  5471 raft_consensus.cc:697] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:26.982903  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:26.983659  5030 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:26.970824  5472 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:26.986719  5034 raft_consensus.cc:2393] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:26.987155  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.988276  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:26.986188  5471 consensus_queue.cc:237] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:26.989315  5610 raft_consensus.cc:2804] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:26.989733  5610 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:26.990093  5610 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.991999  5472 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:26.993984  5603 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.001431  4887 raft_consensus.cc:2468] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:27.002800  4927 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.009155  5449 raft_consensus.cc:2804] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.009614  5449 raft_consensus.cc:697] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:27.010437  5449 consensus_queue.cc:237] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.011451  5608 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.012830  5608 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.013971  5699 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.018726  5699 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.009629  5607 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.016857  5609 raft_consensus.cc:515] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.001226  5541 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.032968  5449 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.034628  5708 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.034860  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.035372  5708 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.035735  5609 leader_election.cc:290] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.036578  5699 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.037160  5699 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.037590  5699 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.038946  5607 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.033229  5003 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
W20260709 12:29:27.046761  5003 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.047873  5003 leader_election.cc:343] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.033769  5449 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.053400  5616 raft_consensus.cc:493] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.057511  5616 raft_consensus.cc:515] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.061929  5449 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.063486  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:27.064391  5030 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.064997  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.065949  5449 raft_consensus.cc:493] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.044502  5699 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.032410  5541 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.067698  5449 raft_consensus.cc:515] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.069895  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.070479  4889 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.062476  5607 raft_consensus.cc:493] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.062202  5472 raft_consensus.cc:493] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.074014  5616 leader_election.cc:290] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.074761  5608 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.074585  5607 raft_consensus.cc:515] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.075465  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.075714  5708 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.076464  5703 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.076622  5707 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.077028  5472 raft_consensus.cc:515] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.077320  5707 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.077823  5607 leader_election.cc:290] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.078572  5702 raft_consensus.cc:2804] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.077054  5703 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.079066  5702 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.079479  5472 leader_election.cc:290] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.079610  5707 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.067080  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.073673  4929 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.087294  5608 raft_consensus.cc:2804] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.087699  5608 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.088059  5608 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.088111  4929 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:27.089249  5713 raft_consensus.cc:2804] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.089666  5713 raft_consensus.cc:493] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.090101  5713 raft_consensus.cc:3060] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.090325  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.090379  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.092741  5703 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.094525  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.095288  4890 raft_consensus.cc:2468] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:27.096067  5449 leader_election.cc:290] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.096627  5001 leader_election.cc:304] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.097473  5703 raft_consensus.cc:2804] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.097860  5703 raft_consensus.cc:493] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.098222  5703 raft_consensus.cc:3060] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.099828  5541 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.099996  5707 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.093694  5655 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.100617  5707 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.102763  5707 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.107621  5653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.108079  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.108919  5652 raft_consensus.cc:2468] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:27.109462  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.110014  5608 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:27.110595  5003 leader_election.cc:343] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.110478  5654 raft_consensus.cc:2468] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
W20260709 12:29:27.113425  5003 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.079525  5702 raft_consensus.cc:3060] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.084569  5708 raft_consensus.cc:493] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.116906  5708 raft_consensus.cc:515] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.117753  5608 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.119714  5708 leader_election.cc:290] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.120427  5616 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.138803  5616 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.113898  5603 raft_consensus.cc:493] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.126498  5712 raft_consensus.cc:493] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.142666  5616 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.142587  5712 raft_consensus.cc:515] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.143357  5603 raft_consensus.cc:515] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.144997  5712 leader_election.cc:290] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.145532  5603 leader_election.cc:290] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.132840  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.134184  5703 raft_consensus.cc:515] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.135123  5003 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:27.147768  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.135397  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb55bf7ad4704d18bb564eeb5320fee8" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.136221  5610 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.141008  4889 raft_consensus.cc:2468] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.130872  5707 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.150308  5001 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.148571  4890 raft_consensus.cc:2393] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.126243  5696 raft_consensus.cc:515] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.153231  5708 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.159356  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.159754  5708 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.159296  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.161163  5030 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.160212  5032 raft_consensus.cc:3060] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.151494  5603 raft_consensus.cc:2804] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.166548  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:27.167192  5034 raft_consensus.cc:2468] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.155572  5616 raft_consensus.cc:2804] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.168165  4887 raft_consensus.cc:2393] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.168928  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.169580  4960 raft_consensus.cc:2468] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:27.159497  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.159323  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.173341  4958 raft_consensus.cc:2468] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:27.173341  4886 raft_consensus.cc:2393] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.175223  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
W20260709 12:29:27.176033  4929 leader_election.cc:343] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.176265  4960 raft_consensus.cc:2468] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
W20260709 12:29:27.177433  4929 leader_election.cc:343] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.178151  5607 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.178699  5607 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.180989  5708 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.153439  5707 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.180909  5607 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.183846  5707 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.191023  5471 raft_consensus.cc:493] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.165602  5603 raft_consensus.cc:493] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.192660  5603 raft_consensus.cc:3060] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.191622  5471 raft_consensus.cc:515] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.201292  5032 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.200356  5003 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.176136  5610 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.197886  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.205633  5656 raft_consensus.cc:3060] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.206446  5703 leader_election.cc:290] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.210258  5471 leader_election.cc:290] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.211356  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.211932  5696 leader_election.cc:290] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.212179  5605 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:27.212116  4958 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.213604  5605 tablet_bootstrap.cc:492] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:27.213500  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.214340  4955 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.214471  5605 ts_tablet_manager.cc:1403] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.297s	user 0.045s	sys 0.024s
I20260709 12:29:27.219035  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.219177  5605 raft_consensus.cc:359] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.220207  5605 raft_consensus.cc:740] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.221161  5605 consensus_queue.cc:260] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.221774  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.222453  4955 raft_consensus.cc:2468] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.202216  5616 raft_consensus.cc:697] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:27.219746  4960 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.225207  5030 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.227581  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.223855  5605 ts_tablet_manager.cc:1434] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:29:27.230660  4960 raft_consensus.cc:2393] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.232199  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.232463  5003 leader_election.cc:304] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.232909  4890 raft_consensus.cc:3060] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.233753  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.228909  5472 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.234328  4889 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.236166  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.236655  5003 leader_election.cc:304] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.236809  4889 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.237911  5003 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:27.229117  5708 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.246495  5708 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.248567  5713 raft_consensus.cc:515] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.258435  4883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.240567  5719 raft_consensus.cc:2804] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.259123  5708 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.250567  4889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.238982  5716 raft_consensus.cc:2749] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.259164  5713 leader_election.cc:290] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.252553  4855 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.259883  4889 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.251979  4885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.224350  5702 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.260833  4885 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.261075  4855 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:27.261440  5653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e359e160bb414d9099cff45b96f6b40d" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.261950  5712 raft_consensus.cc:2804] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.262392  5712 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.262466  5541 raft_consensus.cc:2749] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.262916  5712 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.251744  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.254083  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:27.264027  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.264722  4886 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.265275  5702 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.223855  5616 consensus_queue.cc:237] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.265815  4855 leader_election.cc:304] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.266299  5605 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:27.267745  5717 raft_consensus.cc:2804] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.268164  5717 raft_consensus.cc:493] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.268570  5717 raft_consensus.cc:3060] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.256415  5610 raft_consensus.cc:2804] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.262250  5653 raft_consensus.cc:2393] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.249722  5649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.264106  4888 raft_consensus.cc:2468] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.270524  5649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.271678  5649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.259357  5719 raft_consensus.cc:493] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.272423  5719 raft_consensus.cc:3060] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.255549  5656 raft_consensus.cc:2468] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.269305  5610 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.269022  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.280273  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.280385  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.280965  4960 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.281222  5647 raft_consensus.cc:2468] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.282524  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.262926  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.283723  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03b26e75deab440d913747aece108666" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.285475  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.285781  5647 raft_consensus.cc:2393] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.264786  5655 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.289661  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:27.289880  5651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.290305  4960 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.291013  5651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.278812  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.292593  5648 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.286749  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.293674  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.287137  5653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.297011  5651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.297013  5610 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.294595  5652 raft_consensus.cc:1275] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.305315  5472 raft_consensus.cc:2749] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.288964  5646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.287936  5649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.310322  5719 raft_consensus.cc:515] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.311830  5712 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.307544  5001 leader_election.cc:304] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.312631  5719 leader_election.cc:290] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.312661  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.313516  5652 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.314391  5649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:27.315126  5649 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:27.315587  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.316215  5652 raft_consensus.cc:3060] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.316277  5646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.275815  4890 raft_consensus.cc:2468] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.319049  5001 leader_election.cc:304] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.320123  5703 raft_consensus.cc:2804] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.320729  5703 raft_consensus.cc:697] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
W20260709 12:29:27.321265  4929 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.321952  4929 leader_election.cc:343] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.322202  5712 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.321810  5703 consensus_queue.cc:237] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:27.322680  4929 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.313447  5719 raft_consensus.cc:3060] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:27.323359  4929 leader_election.cc:343] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.305652  5717 raft_consensus.cc:515] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.328761  5717 leader_election.cc:290] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.328936  4711 catalog_manager.cc:5697] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.330797  4711 catalog_manager.cc:5697] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.331983  5610 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:27.323957  4929 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.333519  5655 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.306702  4927 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.334585  5610 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.335628  4927 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.336674  5713 raft_consensus.cc:2804] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.337028  5713 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.337267  5608 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.337399  5713 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.337754  5608 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.338179  5608 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.339401  5719 raft_consensus.cc:2749] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.341424  5603 raft_consensus.cc:515] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.343688  5603 leader_election.cc:290] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:27.347321  5003 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.131:41655, user_credentials={real_user=slave}} blocked reactor thread for 109450us
I20260709 12:29:27.344519  4927 leader_election.cc:304] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.347824  5608 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.348620  4927 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.346482  5713 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.349397  5708 raft_consensus.cc:3060] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.349510  4927 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.350435  5709 raft_consensus.cc:2804] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.350924  5713 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.351014  5709 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.351552  5709 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.351661  5713 raft_consensus.cc:2804] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.354923  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:27.355683  5030 raft_consensus.cc:2468] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:27.357231  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.358002  5030 raft_consensus.cc:2393] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
W20260709 12:29:27.359179  4929 leader_election.cc:343] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.361379  5034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fab4a18b1e4d2faa9f58d62f7ecd89" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.362725  5034 raft_consensus.cc:3060] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.352087  5713 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.363596  5713 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.360190  5608 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.361687  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:27.367475  5708 raft_consensus.cc:2749] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.367681  5032 raft_consensus.cc:2393] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:27.362785  5652 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:27.372335  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e5720333ba432baadd844189c9c80b" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.374155  5034 raft_consensus.cc:2468] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:27.360613  5030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" is_pre_election: true
I20260709 12:29:27.376209  5030 raft_consensus.cc:2393] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:27.360527  5709 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.361999  4927 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.377818  4927 leader_election.cc:304] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.377381  4712 catalog_manager.cc:5697] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.378664  5709 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.379364  5709 raft_consensus.cc:2804] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.379654  4712 catalog_manager.cc:5697] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:27.382016  4929 leader_election.cc:343] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.382678  5031 raft_consensus.cc:2468] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.383368  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e99cb592cbcf46d3bbf20936ff9ec873" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.384028  5608 raft_consensus.cc:2804] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.385987  5649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.386082  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.383936  5646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.386900  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.381105  5709 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.407444  5608 raft_consensus.cc:493] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.409423  5709 raft_consensus.cc:3060] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.409646  5608 raft_consensus.cc:3060] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.410674  5646 raft_consensus.cc:3060] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.389851  5647 raft_consensus.cc:1275] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.390650  5708 consensus_queue.cc:1048] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:27.393100  4929 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.414407  4929 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:27.415261  4929 leader_election.cc:304] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:27.416041  5449 raft_consensus.cc:2804] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.416409  5449 raft_consensus.cc:493] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.416711  5449 raft_consensus.cc:3060] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:27.310704  4855 leader_election.cc:343] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.393425  5003 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.417521  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 106843us
I20260709 12:29:27.391909  5713 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:27.418082  4855 leader_election.cc:343] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.418500  5654 raft_consensus.cc:3060] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:27.418717  4855 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.419725  5713 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.403262  4890 raft_consensus.cc:1275] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.420292  5708 raft_consensus.cc:2804] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.420822  5708 raft_consensus.cc:697] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:27.418089  5003 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.421672  5708 consensus_queue.cc:237] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
W20260709 12:29:27.422704  5003 leader_election.cc:343] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.423132  5003 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.424211  5003 leader_election.cc:304] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.430223  4929 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.425362  5616 raft_consensus.cc:2749] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.426976  5656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.427282  5652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.433113  5655 raft_consensus.cc:3060] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.429550  5610 consensus_queue.cc:1048] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 12:29:27.419734  4855 leader_election.cc:343] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.424466  5720 raft_consensus.cc:2749] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.431384  5471 raft_consensus.cc:2804] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:27.435369  4855 leader_election.cc:343] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.435817  5719 raft_consensus.cc:2749] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.430045  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.432070  4929 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:27.444329  5713 raft_consensus.cc:2749] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.436944  4855 leader_election.cc:304] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.445326  5712 raft_consensus.cc:2804] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.445845  5712 raft_consensus.cc:697] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:27.445950  4855 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.446748  5696 raft_consensus.cc:2804] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.447356  5696 raft_consensus.cc:493] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.446785  5712 consensus_queue.cc:237] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.447754  5696 raft_consensus.cc:3060] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.437412  4712 catalog_manager.cc:5697] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:27.438125  5003 leader_election.cc:343] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.451022  5003 leader_election.cc:343] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.452670  5719 consensus_queue.cc:1048] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:27.453567  5605 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:27.454777  5605 tablet_bootstrap.cc:492] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:27.455438  5605 ts_tablet_manager.cc:1403] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.189s	user 0.013s	sys 0.004s
I20260709 12:29:27.442272  5608 raft_consensus.cc:515] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.441254  5709 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.459247  4929 leader_election.cc:304] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:27.459604  5031 raft_consensus.cc:1275] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.435774  5471 raft_consensus.cc:493] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.460703  5471 raft_consensus.cc:3060] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:27.446842  4855 leader_election.cc:343] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.461977  4855 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.462764  5717 raft_consensus.cc:2804] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.463166  4855 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.463327  5717 raft_consensus.cc:697] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:27.464010  5609 raft_consensus.cc:2804] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.464409  5609 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.464314  5717 consensus_queue.cc:237] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.464955  5609 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.465600  5709 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.467967  5030 raft_consensus.cc:1275] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.476508  5699 raft_consensus.cc:3060] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.455801  5696 raft_consensus.cc:515] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.478952  5471 raft_consensus.cc:515] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.436143  5649 raft_consensus.cc:1275] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.494676  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f3921f98794b60a18335cd330faa86" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.495358  4958 raft_consensus.cc:3060] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.496555  5471 leader_election.cc:290] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.497104  5608 leader_election.cc:290] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:27.500367  5655 raft_consensus.cc:2468] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.503355  4958 raft_consensus.cc:2468] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.482317  5696 leader_election.cc:290] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.483083  4960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.483204  5646 raft_consensus.cc:2468] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.484859  5654 raft_consensus.cc:2468] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.503736  5605 raft_consensus.cc:359] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.505436  5605 raft_consensus.cc:740] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.508567  4958 raft_consensus.cc:1275] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.506553  5605 consensus_queue.cc:260] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.510715  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
W20260709 12:29:27.513644  5003 consensus_peers.cc:597] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:27.514635  5605 ts_tablet_manager.cc:1434] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.001s
I20260709 12:29:27.514951  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.486873  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0783f64f319148009af4133e89e944ee" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.491017  5609 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.516300  5647 raft_consensus.cc:3060] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.518203  5609 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.491472  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.492753  4855 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.428894  5449 raft_consensus.cc:515] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.518987  4962 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.520226  5449 leader_election.cc:290] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:27.485847  5699 raft_consensus.cc:2749] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.521062  4960 raft_consensus.cc:2393] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.515578  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.525060  4713 catalog_manager.cc:5697] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.526731  5647 raft_consensus.cc:2468] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:27.527659  5609 raft_consensus.cc:2804] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.528124  4855 leader_election.cc:304] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.530164  5717 raft_consensus.cc:2804] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.530611  5717 raft_consensus.cc:493] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.530992  5717 raft_consensus.cc:3060] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.528141  5609 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.531723  5609 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.526803  4713 catalog_manager.cc:5697] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.534937  5003 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:27.536934  4855 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.537787  5703 raft_consensus.cc:2804] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.537781  5712 raft_consensus.cc:2804] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.538403  4955 raft_consensus.cc:2468] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.538511  5703 raft_consensus.cc:697] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:27.545344  5703 consensus_queue.cc:237] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.546605  5717 raft_consensus.cc:515] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.549160  5717 leader_election.cc:290] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.540575  5712 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.550038  5712 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.515583  5605 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:27.538388  4855 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.561868  5717 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:27.562332  5717 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:27.562727  5717 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.536948  5003 leader_election.cc:304] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:27.565891  5610 raft_consensus.cc:2804] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.566452  5610 raft_consensus.cc:697] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:27.543610  4962 raft_consensus.cc:2468] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:27.567335  5610 consensus_queue.cc:237] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.572413  4713 catalog_manager.cc:5697] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.576113  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.578037  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.578706  5712 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.581171  5712 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.581663  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.582335  5648 raft_consensus.cc:3060] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.583335  5031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29feba7c56da402abe958d088341ea39" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.583945  5031 raft_consensus.cc:3060] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.584779  5038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e4db1a6dac4e348f16817448c5005e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.585430  5038 raft_consensus.cc:3060] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.588673  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.589391  5035 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.590947  5717 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.593008  5038 raft_consensus.cc:2468] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.594762  5609 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:27.596226  4929 leader_election.cc:343] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.597141  4929 leader_election.cc:343] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.598440  4929 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.599335  4929 leader_election.cc:343] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.599237  5708 consensus_queue.cc:1048] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:27.600277  4929 leader_election.cc:343] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.600921  4929 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:27.601831  5709 raft_consensus.cc:2804] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:29:27.601989  4929 leader_election.cc:343] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.602404  5709 raft_consensus.cc:697] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:27.603235  5031 raft_consensus.cc:2468] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:27.603354  5709 consensus_queue.cc:237] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.609194  5449 consensus_queue.cc:1048] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:27.603873  5699 consensus_queue.cc:1048] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:27.603883  5035 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.615957  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.616870  5654 raft_consensus.cc:2393] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.619976  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.620838  5655 raft_consensus.cc:2393] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in this term.
I20260709 12:29:27.622221  5616 consensus_queue.cc:1048] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.622535  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.627998  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.628165  4713 catalog_manager.cc:5697] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.633929  5003 leader_election.cc:304] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.634268  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ef746273b574779b4dcb59a0d239aa2" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:27.635213  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.636817  4955 raft_consensus.cc:2393] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.637780  5717 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.638864  5616 raft_consensus.cc:2804] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.640355  5003 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.643268  5646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.643885  5703 raft_consensus.cc:2804] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.643255  5616 raft_consensus.cc:697] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:27.644562  5703 raft_consensus.cc:697] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:27.645368  5616 consensus_queue.cc:237] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.645869  5609 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:27.645529  5703 consensus_queue.cc:237] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.641219  5647 raft_consensus.cc:1275] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.649959  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30e68af9cb68410b9539fea5e2191450" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.650617  4962 raft_consensus.cc:3060] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.654325  5648 raft_consensus.cc:2468] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.655457  4712 catalog_manager.cc:5697] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.669415  4887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.669627  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.670228  4887 raft_consensus.cc:2393] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.670855  4888 raft_consensus.cc:2393] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.671502  4962 raft_consensus.cc:2468] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:27.673035  4888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.674194  4888 raft_consensus.cc:2393] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.676082  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.676795  4890 raft_consensus.cc:2393] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.682303  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd4abb7c334f02ba9c4672289410f4" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:27.683470  4886 raft_consensus.cc:2393] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa1ec70394d466ab14173b48b67b8ed in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:27.685742  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d35c974e4cb84cf49a6143cc50956e93" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
W20260709 12:29:27.687496  4929 consensus_peers.cc:597] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:27.692147  4929 leader_election.cc:304] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.694515  5449 raft_consensus.cc:2804] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.696483  5449 raft_consensus.cc:697] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:27.697669  5449 consensus_queue.cc:237] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.686745  4886 raft_consensus.cc:3060] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.699138  4929 leader_election.cc:304] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:27.701741  5471 raft_consensus.cc:2749] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.705256  4888 raft_consensus.cc:1275] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:27.725405  5003 consensus_peers.cc:597] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:27.731734  4855 leader_election.cc:304] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.733911  5717 raft_consensus.cc:2804] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.734155  5605 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:27.734310  4886 raft_consensus.cc:2468] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:27.734493  5717 raft_consensus.cc:697] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:27.735196  5605 tablet_bootstrap.cc:492] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:27.735672  5717 consensus_queue.cc:237] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.735949  5605 ts_tablet_manager.cc:1403] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.221s	user 0.026s	sys 0.012s
I20260709 12:29:27.739101  5605 raft_consensus.cc:359] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.739780  5605 raft_consensus.cc:740] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.747349  5605 consensus_queue.cc:260] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.748976  5605 ts_tablet_manager.cc:1434] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:29:27.749990  5605 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:27.758865  4855 leader_election.cc:304] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.760133  5712 raft_consensus.cc:2804] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.760751  5712 raft_consensus.cc:697] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:27.761562  5712 consensus_queue.cc:237] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.739806  4710 catalog_manager.cc:5697] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.764102  4713 catalog_manager.cc:5697] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.765904  4855 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:27.767910  5717 raft_consensus.cc:2804] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.768483  5717 raft_consensus.cc:697] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:27.768092  4712 catalog_manager.cc:5697] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.769411  5717 consensus_queue.cc:237] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:27.776312  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.777308  4955 raft_consensus.cc:2393] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.778504  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.777509  4710 catalog_manager.cc:5697] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.781029  4886 raft_consensus.cc:1275] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.782694  5616 consensus_queue.cc:1048] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.785697  4710 catalog_manager.cc:5697] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.789551  5655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.792122  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.819027  4955 raft_consensus.cc:2393] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.798633  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:27.800664  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:27.820287  4962 raft_consensus.cc:2393] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:27.802186  4927 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.823408  4927 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.824245  5471 raft_consensus.cc:2749] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.824971  5471 raft_consensus.cc:2749] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.828871  4927 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.829675  5449 raft_consensus.cc:2749] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.830859  4927 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:27.831547  5449 raft_consensus.cc:2749] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.835593  4888 raft_consensus.cc:1275] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.837342  5719 consensus_queue.cc:1048] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.837056  4712 catalog_manager.cc:5697] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.839073  4712 catalog_manager.cc:5697] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.857312  5699 consensus_queue.cc:1048] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:27.857378  4927 leader_election.cc:304] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.862211  5708 raft_consensus.cc:2804] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.862829  5708 raft_consensus.cc:697] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:27.863819  5708 consensus_queue.cc:237] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:27.817449  4855 leader_election.cc:343] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.867915  5648 raft_consensus.cc:1275] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.870499  5610 consensus_queue.cc:1048] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:27.867961  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 50541.5us
W20260709 12:29:27.872872  4855 leader_election.cc:343] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.875146  4855 leader_election.cc:343] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.888695  5712 consensus_queue.cc:1048] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:27.900902  4855 leader_election.cc:343] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.907528  5605 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:27.908599  5605 tablet_bootstrap.cc:492] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:27.909282  5605 ts_tablet_manager.cc:1403] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.160s	user 0.070s	sys 0.071s
W20260709 12:29:27.912595  4927 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:33187, user_credentials={real_user=slave}} blocked reactor thread for 55293.8us
I20260709 12:29:27.912747  5605 raft_consensus.cc:359] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.914068  5605 raft_consensus.cc:740] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
W20260709 12:29:27.916774  4855 leader_election.cc:343] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.918000  4855 leader_election.cc:343] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.917330  5605 consensus_queue.cc:260] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.919894  4712 catalog_manager.cc:5697] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.921465  5605 ts_tablet_manager.cc:1434] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:29:27.922438  5605 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:27.925036  4855 leader_election.cc:304] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.925882  5712 raft_consensus.cc:2749] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.926352  4855 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.927466  4855 leader_election.cc:304] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:27.927994  5712 raft_consensus.cc:2749] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.928139  5609 raft_consensus.cc:2804] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.928886  5609 raft_consensus.cc:697] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:27.929956  5609 consensus_queue.cc:237] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:27.935674  4855 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.949175  5038 raft_consensus.cc:1275] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.962314  5609 raft_consensus.cc:2749] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.963132  5708 consensus_queue.cc:1048] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.965718  4855 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.966920  4855 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:27.967463  5609 raft_consensus.cc:2749] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.971107  5609 raft_consensus.cc:2749] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:27.976204  4710 catalog_manager.cc:5697] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 2, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.981515  5605 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:27.987232  5605 tablet_bootstrap.cc:492] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:27.988752  5605 ts_tablet_manager.cc:1403] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.028s
I20260709 12:29:27.989737  5648 raft_consensus.cc:1275] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.991796  5605 raft_consensus.cc:359] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.992311  5708 consensus_queue.cc:1048] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.992620  5605 raft_consensus.cc:740] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.993746  5605 consensus_queue.cc:260] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:27.996443  5605 ts_tablet_manager.cc:1434] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:28.003860  5605 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.005693  4960 raft_consensus.cc:1275] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.018723  5654 raft_consensus.cc:1275] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.029009  5038 raft_consensus.cc:1275] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.040405  4960 raft_consensus.cc:1275] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.045588  5038 raft_consensus.cc:1275] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:28.048954  4855 consensus_peers.cc:597] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:28.051503  5712 consensus_queue.cc:1048] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.053082  5719 consensus_queue.cc:1048] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.058096  4960 raft_consensus.cc:1275] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.059893  5720 consensus_queue.cc:1048] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.068712  5712 consensus_queue.cc:1048] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:28.082110  5003 consensus_peers.cc:597] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.088224  5605 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.089425  5605 tablet_bootstrap.cc:492] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.090981  5648 raft_consensus.cc:1275] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.090131  5605 ts_tablet_manager.cc:1403] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.087s	user 0.043s	sys 0.035s
I20260709 12:29:28.094118  5605 raft_consensus.cc:359] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.094324  5609 consensus_queue.cc:1048] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.094847  5605 raft_consensus.cc:740] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.098877  5609 consensus_queue.cc:1048] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.099943  5648 raft_consensus.cc:1275] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.102921  5720 consensus_queue.cc:1048] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.110106  5696 consensus_queue.cc:1048] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.095685  5605 consensus_queue.cc:260] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.120549  5605 ts_tablet_manager.cc:1434] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:29:28.121383  5605 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:28.122110  5003 consensus_peers.cc:597] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.135771  5654 raft_consensus.cc:1275] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.137751  5720 consensus_queue.cc:1048] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.153679  4960 raft_consensus.cc:1275] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.157289  4887 raft_consensus.cc:1275] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.161751  5720 consensus_queue.cc:1048] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:29:28.178458  4960 raft_consensus.cc:1275] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.190637  5719 consensus_queue.cc:1048] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:29:28.249241  5654 raft_consensus.cc:1275] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:28.279541  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.132:42487, user_credentials={real_user=slave}} blocked reactor thread for 114500us
I20260709 12:29:28.287710  5605 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.281270  5648 raft_consensus.cc:3060] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.296447  5605 tablet_bootstrap.cc:492] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.297307  5605 ts_tablet_manager.cc:1403] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.176s	user 0.027s	sys 0.008s
I20260709 12:29:28.298177  5699 consensus_queue.cc:1048] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:29:28.310017  5038 raft_consensus.cc:1275] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.329635  5648 raft_consensus.cc:1275] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:28.339147  4855 consensus_peers.cc:597] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:28.340811  5449 consensus_queue.cc:1048] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.340718  5471 consensus_queue.cc:1048] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.352552  5605 raft_consensus.cc:359] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.353981  5605 raft_consensus.cc:740] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.356904  4886 raft_consensus.cc:1275] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.359131  5605 consensus_queue.cc:260] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
W20260709 12:29:28.374655  5003 consensus_peers.cc:597] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:28.374863  5030 raft_consensus.cc:1275] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.375974  5032 raft_consensus.cc:1275] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.376788  5034 raft_consensus.cc:1275] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.379781  5605 ts_tablet_manager.cc:1434] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.082s	user 0.011s	sys 0.000s
I20260709 12:29:28.386662  5605 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:28.412662  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 73592.2us
I20260709 12:29:28.412883  5708 consensus_queue.cc:1048] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.423079  5706 consensus_queue.cc:1048] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.424433  5782 consensus_queue.cc:1048] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.425688  5696 consensus_queue.cc:1048] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:28.433571  4855 consensus_peers.cc:597] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.435278  5708 consensus_queue.cc:1048] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.467593  5605 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.469803  5605 tablet_bootstrap.cc:492] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.471154  5605 ts_tablet_manager.cc:1403] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.090s	user 0.037s	sys 0.005s
I20260709 12:29:28.473745  5605 raft_consensus.cc:359] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.474470  5605 raft_consensus.cc:740] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.477449  5605 consensus_queue.cc:260] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.479856  5605 ts_tablet_manager.cc:1434] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:29:28.480746  5605 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:28.491094  4929 consensus_peers.cc:597] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:28.503086  5605 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.504140  5605 tablet_bootstrap.cc:492] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.505028  5605 ts_tablet_manager.cc:1403] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 12:29:28.507470  5605 raft_consensus.cc:359] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.508129  5605 raft_consensus.cc:740] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.508837  5605 consensus_queue.cc:260] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.511698  5605 ts_tablet_manager.cc:1434] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:28.512559  5605 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:28.518544  4929 consensus_peers.cc:597] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:28.519299  4929 consensus_peers.cc:597] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:28.556965  4964 raft_consensus.cc:1275] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:28.597579  4855 consensus_peers.cc:597] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:28.593238  5609 consensus_queue.cc:1048] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:29:28.607555  5605 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.609234  5605 tablet_bootstrap.cc:492] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.610186  5605 ts_tablet_manager.cc:1403] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.024s
I20260709 12:29:28.613127  5654 raft_consensus.cc:1275] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.622411  5605 raft_consensus.cc:359] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.625241  5605 raft_consensus.cc:740] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
W20260709 12:29:28.627480  4929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 43636.3us
I20260709 12:29:28.628525  5605 consensus_queue.cc:260] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
W20260709 12:29:28.630544  4929 consensus_peers.cc:597] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.631168  5605 ts_tablet_manager.cc:1434] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 12:29:28.633486  5605 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.686592  5605 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.687665  5605 tablet_bootstrap.cc:492] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.688413  5605 ts_tablet_manager.cc:1403] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.055s	user 0.032s	sys 0.021s
I20260709 12:29:28.691182  5605 raft_consensus.cc:359] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.692127  5605 raft_consensus.cc:740] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.693763  5605 consensus_queue.cc:260] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.702068  5605 ts_tablet_manager.cc:1434] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:29:28.703665  5605 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
W20260709 12:29:28.725042  4855 consensus_peers.cc:597] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.726899  5609 consensus_queue.cc:1048] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:28.728693  4855 consensus_peers.cc:597] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.742910  5605 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.744480  5605 tablet_bootstrap.cc:492] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.745409  5605 ts_tablet_manager.cc:1403] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.042s	user 0.032s	sys 0.003s
I20260709 12:29:28.758867  5605 raft_consensus.cc:359] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.759562  5605 raft_consensus.cc:740] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.760216  5605 consensus_queue.cc:260] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.762794  5605 ts_tablet_manager.cc:1434] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 12:29:28.763677  5605 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.780936  5605 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.781709  5605 tablet_bootstrap.cc:492] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.782271  5605 ts_tablet_manager.cc:1403] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 12:29:28.784514  5605 raft_consensus.cc:359] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.785192  5605 raft_consensus.cc:740] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.786051  5605 consensus_queue.cc:260] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.788419  5605 ts_tablet_manager.cc:1434] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:28.789480  5605 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.804837  5605 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.805827  5605 tablet_bootstrap.cc:492] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.806504  5605 ts_tablet_manager.cc:1403] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:29:28.809161  5605 raft_consensus.cc:359] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.809866  5605 raft_consensus.cc:740] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.810624  5605 consensus_queue.cc:260] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.812400  5605 ts_tablet_manager.cc:1434] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:28.813136  5605 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.833809  5605 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.834864  5605 tablet_bootstrap.cc:492] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.835564  5605 ts_tablet_manager.cc:1403] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.007s
I20260709 12:29:28.838228  5605 raft_consensus.cc:359] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.838922  5605 raft_consensus.cc:740] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.839612  5605 consensus_queue.cc:260] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.841689  5605 ts_tablet_manager.cc:1434] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:28.843389  5605 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.860416  5605 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.861310  5605 tablet_bootstrap.cc:492] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.861887  5605 ts_tablet_manager.cc:1403] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 12:29:28.864189  5605 raft_consensus.cc:359] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.864895  5605 raft_consensus.cc:740] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.865640  5605 consensus_queue.cc:260] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.867733  5605 ts_tablet_manager.cc:1434] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:28.868795  5605 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.884752  5605 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.885854  5605 tablet_bootstrap.cc:492] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.886662  5605 ts_tablet_manager.cc:1403] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:29:28.889266  5605 raft_consensus.cc:359] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.889940  5605 raft_consensus.cc:740] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.890622  5605 consensus_queue.cc:260] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:28.892656  5605 ts_tablet_manager.cc:1434] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:28.893918  5605 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.907855  5605 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.909327  5605 tablet_bootstrap.cc:492] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.910094  5605 ts_tablet_manager.cc:1403] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 12:29:28.914149  5605 raft_consensus.cc:359] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.914937  5605 raft_consensus.cc:740] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.916759  5605 consensus_queue.cc:260] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:28.918439  5605 ts_tablet_manager.cc:1434] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:29:28.919494  5605 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.941458  5605 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:28.942452  5605 tablet_bootstrap.cc:492] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:28.943068  5605 ts_tablet_manager.cc:1403] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 12:29:28.945106  5605 raft_consensus.cc:359] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.945597  5605 raft_consensus.cc:740] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.946262  5605 consensus_queue.cc:260] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.948523  5605 ts_tablet_manager.cc:1434] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:29:28.949877  5605 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:28.950440  5654 raft_consensus.cc:3060] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.956431  5720 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.957062  5720 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.962248  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:28.962373  5720 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:28.963078  4958 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:28.964057  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:28.964893  5648 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 1.
I20260709 12:29:28.966336  5003 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:28.967231  5720 raft_consensus.cc:2804] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:28.967648  5720 raft_consensus.cc:493] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.968029  5720 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.981173  5720 raft_consensus.cc:515] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:28.983911  5720 leader_election.cc:290] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:28.985306  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:28.986193  4958 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.986037  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4831fc84d91e4eafbc37bf86cd29ad5e" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:28.987210  5648 raft_consensus.cc:3060] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.001844  4958 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 3.
I20260709 12:29:29.003948  5003 leader_election.cc:304] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:29.004918  5720 raft_consensus.cc:2804] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.005447  5720 raft_consensus.cc:697] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:29.006402  5720 consensus_queue.cc:237] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:29.007534  5648 raft_consensus.cc:2468] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 3.
I20260709 12:29:29.008203  5605 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.009233  5605 tablet_bootstrap.cc:492] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.009910  5605 ts_tablet_manager.cc:1403] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.060s	user 0.020s	sys 0.032s
I20260709 12:29:29.012266  5605 raft_consensus.cc:359] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.012952  5605 raft_consensus.cc:740] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.013607  5605 consensus_queue.cc:260] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.016152  5605 ts_tablet_manager.cc:1434] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:29.023850  5605 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.025254  4710 catalog_manager.cc:5697] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 3, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 3 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:29.050836  5605 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.051818  5605 tablet_bootstrap.cc:492] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.052536  5605 ts_tablet_manager.cc:1403] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.006s
I20260709 12:29:29.054648  5605 raft_consensus.cc:359] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.055341  5605 raft_consensus.cc:740] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.056038  5605 consensus_queue.cc:260] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.058144  5605 ts_tablet_manager.cc:1434] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:29.059271  5605 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.073212  5605 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.074183  5605 tablet_bootstrap.cc:492] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.074880  5605 ts_tablet_manager.cc:1403] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.003s
I20260709 12:29:29.077095  5605 raft_consensus.cc:359] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.077644  5605 raft_consensus.cc:740] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.078289  5605 consensus_queue.cc:260] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.079896  5654 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.081220  5605 ts_tablet_manager.cc:1434] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:29:29.082089  5605 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.099221  5605 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.100865  5605 tablet_bootstrap.cc:492] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.101229  5655 raft_consensus.cc:3060] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.101624  5605 ts_tablet_manager.cc:1403] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 12:29:29.104060  5605 raft_consensus.cc:359] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:29.107537  5605 raft_consensus.cc:740] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.108318  5605 consensus_queue.cc:260] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:29.111327  5605 ts_tablet_manager.cc:1434] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:29:29.112177  5647 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.112170  5605 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.121006  5648 raft_consensus.cc:3060] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.141544  5605 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.142702  5605 tablet_bootstrap.cc:492] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.143827  5605 ts_tablet_manager.cc:1403] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.012s
I20260709 12:29:29.145788  5605 raft_consensus.cc:359] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:29.146253  5605 raft_consensus.cc:740] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.147009  5605 consensus_queue.cc:260] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:29.149117  5605 ts_tablet_manager.cc:1434] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:29:29.149897  5605 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.163074  5605 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.164333  5605 tablet_bootstrap.cc:492] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.165004  5605 ts_tablet_manager.cc:1403] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:29:29.166935  5605 raft_consensus.cc:359] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:29.167591  5605 raft_consensus.cc:740] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.168354  5605 consensus_queue.cc:260] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:29.170544  5605 ts_tablet_manager.cc:1434] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:29:29.171809  5605 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.198884  5605 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.199880  5605 tablet_bootstrap.cc:492] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.200631  5605 ts_tablet_manager.cc:1403] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.000s
I20260709 12:29:29.202838  5605 raft_consensus.cc:359] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.203466  5605 raft_consensus.cc:740] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.204013  5605 consensus_queue.cc:260] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.205312  5605 ts_tablet_manager.cc:1434] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:29.206646  5605 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap starting.
I20260709 12:29:29.220728  5605 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:29.221589  5605 tablet_bootstrap.cc:492] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Bootstrap complete.
I20260709 12:29:29.222204  5605 ts_tablet_manager.cc:1403] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:29:29.224350  5605 raft_consensus.cc:359] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.224877  5605 raft_consensus.cc:740] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Initialized, Role: FOLLOWER
I20260709 12:29:29.225489  5605 consensus_queue.cc:260] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.227422  5605 ts_tablet_manager.cc:1434] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:29.288599  5647 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.345798  5772 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.346381  5772 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.350538  5772 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:29.405303  5648 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.419493  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:29.420450  4890 raft_consensus.cc:2393] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 2: Already voted for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in this term.
I20260709 12:29:29.425418  4964 raft_consensus.cc:1275] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:29.427533  5616 consensus_queue.cc:1048] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.447242  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:29.448125  4958 raft_consensus.cc:2393] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ec932ee892248cda1f4e99e4ba7887e in current term 2: Already voted for candidate 0fa1ec70394d466ab14173b48b67b8ed in this term.
I20260709 12:29:29.449116  5647 raft_consensus.cc:3060] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.449399  5600 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e; no voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83
I20260709 12:29:29.450850  5772 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.460736  5797 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.461392  5797 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.463325  5772 raft_consensus.cc:2749] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:29.469959  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:29.470903  5654 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:29.472222  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:29.475309  4890 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:29.472944  4929 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:29.473065  5797 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:29.479526  5797 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:29.480207  5797 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:29.488281  5655 raft_consensus.cc:1275] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:29.490135  5616 consensus_queue.cc:1048] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.497323  5785 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.498075  5785 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.498446  5646 raft_consensus.cc:3060] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.502702  5785 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:29.480746  5797 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.505249  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:29.505988  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" is_pre_election: true
I20260709 12:29:29.506068  5654 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 1.
I20260709 12:29:29.506785  4890 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:29.507550  4929 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:29.508512  5785 raft_consensus.cc:2804] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:29.508926  5785 raft_consensus.cc:493] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:29.509308  5785 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.520594  5797 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.523319  5797 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:29.524638  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:29.525580  4890 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.539414  5648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:29.540578  5648 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.568699  5648 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 3.
I20260709 12:29:29.573248  4890 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 3.
I20260709 12:29:29.575351  4927 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:29.576208  5797 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.576802  5797 raft_consensus.cc:697] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:29.577816  5797 consensus_queue.cc:237] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.594367  5785 raft_consensus.cc:515] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.599208  5785 leader_election.cc:290] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:29.601737  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:29.602488  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce6101a3bd546fe9ae5ba5767af6277" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:29.603053  4890 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.602523  5647 raft_consensus.cc:3060] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.603582  4711 catalog_manager.cc:5697] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 3, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 3 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:29.611912  5647 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 3.
I20260709 12:29:29.614607  4890 raft_consensus.cc:2468] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 3.
I20260709 12:29:29.615569  4929 leader_election.cc:304] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:29.616952  5785 raft_consensus.cc:2804] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.617477  5785 raft_consensus.cc:697] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:29.618386  5785 consensus_queue.cc:237] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.637542  4711 catalog_manager.cc:5697] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 3, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 3 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:29.638772  5774 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.639331  5774 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.641530  5774 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:29.642460  5647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:29.642861  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:29.643318  5647 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 1.
I20260709 12:29:29.643551  4958 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:29.644884  4855 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:29.646426  5609 raft_consensus.cc:2804] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:29.646858  5609 raft_consensus.cc:493] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:29.647215  5609 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.654788  5609 raft_consensus.cc:515] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.657126  5609 leader_election.cc:290] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:29.658087  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:29.658787  5654 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.658084  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f743d73272948abbd7fc320e050742c" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:29.663480  4958 raft_consensus.cc:3060] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.667632  5654 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 3.
I20260709 12:29:29.668874  4855 leader_election.cc:304] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:29.670787  5847 raft_consensus.cc:2804] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.671273  5847 raft_consensus.cc:697] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:29.672266  4958 raft_consensus.cc:2468] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 3.
I20260709 12:29:29.672215  5847 consensus_queue.cc:237] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.751101  5775 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.775261  5775 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.779093  5775 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:29.837738  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e" is_pre_election: true
I20260709 12:29:29.839165  5654 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:29.841092  4710 catalog_manager.cc:5697] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 3, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:29.841081  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed" is_pre_election: true
I20260709 12:29:29.845340  4958 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 2.
I20260709 12:29:29.893409  4855 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:29.894472  5775 raft_consensus.cc:2804] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:29.894953  5775 raft_consensus.cc:493] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:29.895308  5775 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.915990  5775 raft_consensus.cc:515] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.918356  5775 leader_election.cc:290] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 election: Requested vote from peers 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:29.929435  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:29.929838  5654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "322e052ba6ca4c2989a3fd9aea3efc60" candidate_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
I20260709 12:29:29.930158  4962 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.930536  5654 raft_consensus.cc:3060] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.944713  5654 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 3.
I20260709 12:29:29.952554  4855 leader_election.cc:304] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:29.953537  5851 raft_consensus.cc:2804] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.953620  4962 raft_consensus.cc:2468] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3fc3aedb46404a8f0c54ac0bd04f83 in term 3.
I20260709 12:29:29.954202  5851 raft_consensus.cc:697] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 LEADER]: Becoming Leader. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:29.955444  5851 consensus_queue.cc:237] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:29.973047  4710 catalog_manager.cc:5697] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 reported cstate change: term changed from 1 to 3, leader changed from 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129) to 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:29.976048  5654 raft_consensus.cc:1275] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:29.984102  5838 consensus_queue.cc:1048] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:30.004452  4886 raft_consensus.cc:1275] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.009099  5797 consensus_queue.cc:1048] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:30.081243  4890 raft_consensus.cc:1275] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.082504  5838 consensus_queue.cc:1048] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.097970  5646 raft_consensus.cc:1275] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.101364  5838 consensus_queue.cc:1048] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.166568  5654 raft_consensus.cc:1275] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.168879  5851 consensus_queue.cc:1048] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.193797  4962 raft_consensus.cc:1275] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.195616  5851 consensus_queue.cc:1048] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.491076  5654 raft_consensus.cc:1275] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.492641  5852 consensus_queue.cc:1048] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.504040  4962 raft_consensus.cc:1275] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 8c3fc3aedb46404a8f0c54ac0bd04f83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.506282  5775 consensus_queue.cc:1048] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:36.625886 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:36.684180  4886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:36.685035  4886 raft_consensus.cc:606] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:36.701900  4962 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3340b333db824849a2af865d2658fd89"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:35318
I20260709 12:29:36.702430  4962 raft_consensus.cc:606] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:36.715636  5035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "351e55d393df4e159f251e3253097ed7"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
mode: GRACEFUL
new_leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
 from {username='slave'} at 127.0.0.1:39950
I20260709 12:29:36.716349  5035 raft_consensus.cc:606] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Received request to transfer leadership to TS 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:36.719225  5035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5149b4ff21da4751975490772301b992"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:39950
I20260709 12:29:36.719703  5035 raft_consensus.cc:606] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:36.722427  4962 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
mode: GRACEFUL
new_leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
 from {username='slave'} at 127.0.0.1:35318
I20260709 12:29:36.722946  4962 raft_consensus.cc:606] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Received request to transfer leadership to TS 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:36.725590  4962 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:35318
I20260709 12:29:36.726114  4962 raft_consensus.cc:606] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:36.727388 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:29:36.728852 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:36.729766 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:36.755707  5868 raft_consensus.cc:993] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.756356  5868 raft_consensus.cc:1081] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.758394  5655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5149b4ff21da4751975490772301b992"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53782
I20260709 12:29:36.758999  5655 raft_consensus.cc:493] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:36.759356  5655 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.772081  5655 raft_consensus.cc:515] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:36.780167  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:36.780813  4962 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.787385  5655 leader_election.cc:290] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:36.802374  4962 raft_consensus.cc:2468] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:36.803862  5600 leader_election.cc:304] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:36.813807  5875 raft_consensus.cc:2804] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:36.814931  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5149b4ff21da4751975490772301b992" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:36.815554  5032 raft_consensus.cc:3055] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:36.815800  5032 raft_consensus.cc:740] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:36.816596  5032 consensus_queue.cc:260] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:36.817905  5032 raft_consensus.cc:3060] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.827080  5032 raft_consensus.cc:2468] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:36.832526  5838 raft_consensus.cc:993] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.833173  5838 raft_consensus.cc:1081] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.835561  5655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53768
I20260709 12:29:36.836328  5655 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:36.836833  5655 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.841712  5875 raft_consensus.cc:697] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:36.843323  5875 consensus_queue.cc:237] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:36.850919  5655 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:36.853979  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:36.854575  4962 raft_consensus.cc:3055] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:36.854854  4962 raft_consensus.cc:740] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:36.855538  5655 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:36.855626  4962 consensus_queue.cc:260] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:36.856185  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc367942ca4feb9a1fd830092d860f" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:36.856878  5032 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.856056  4711 catalog_manager.cc:5697] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:36.857225  4962 raft_consensus.cc:3060] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.860423  5852 raft_consensus.cc:993] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.860913  5852 raft_consensus.cc:1081] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.862546  5655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53762
I20260709 12:29:36.863265  5655 raft_consensus.cc:493] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:36.863670  5655 raft_consensus.cc:3060] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:36.864691  5032 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:36.866847  4962 raft_consensus.cc:2468] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:36.867316  5600 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:36.868186  5875 raft_consensus.cc:2804] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:36.868748  5875 raft_consensus.cc:697] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:36.869943  5875 consensus_queue.cc:237] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:36.871120  5655 raft_consensus.cc:515] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:36.873783  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:36.873940  5655 leader_election.cc:290] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:36.874352  4886 raft_consensus.cc:3055] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:36.874662  4886 raft_consensus.cc:740] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:36.875306  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0165ba3321bd42768963b30a87b715e5" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:36.875411  4886 consensus_queue.cc:260] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:36.875958  4962 raft_consensus.cc:3060] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:36.883634  4962 raft_consensus.cc:2468] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:36.883832  4886 raft_consensus.cc:3060] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:36.885017  5600 leader_election.cc:304] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:36.885934  5875 raft_consensus.cc:2804] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:36.886425  5875 raft_consensus.cc:697] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:36.887322  5875 consensus_queue.cc:237] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:36.894767  4886 raft_consensus.cc:2468] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:36.912945  4711 catalog_manager.cc:5697] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:36.914600  4711 catalog_manager.cc:5697] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 2 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:36.947454  5854 raft_consensus.cc:993] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.948020  5838 raft_consensus.cc:1081] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:36.949810  5655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3340b333db824849a2af865d2658fd89"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53768
I20260709 12:29:36.950518  5655 raft_consensus.cc:493] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:36.951270  5655 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.957758  5655 raft_consensus.cc:515] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:36.959643  5655 leader_election.cc:290] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:36.960613  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:36.961220  4886 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.962783  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3340b333db824849a2af865d2658fd89" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:36.963416  4962 raft_consensus.cc:3055] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:36.963725  4962 raft_consensus.cc:740] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:36.964591  4962 consensus_queue.cc:260] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:36.965845  4962 raft_consensus.cc:3060] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.967938  4886 raft_consensus.cc:2468] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:36.969457  5598 leader_election.cc:304] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:36.970232  5875 raft_consensus.cc:2804] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:36.970670  5875 raft_consensus.cc:697] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:36.971531  5875 consensus_queue.cc:237] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:36.971781  4962 raft_consensus.cc:2468] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:36.983922  4711 catalog_manager.cc:5697] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:37.052708  5779 raft_consensus.cc:993] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: : Instructing follower 0fa1ec70394d466ab14173b48b67b8ed to start an election
I20260709 12:29:37.053239  5868 raft_consensus.cc:1081] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Signalling peer 0fa1ec70394d466ab14173b48b67b8ed to start an election
I20260709 12:29:37.054955  4962 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "351e55d393df4e159f251e3253097ed7"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:37.055414  4962 raft_consensus.cc:493] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:37.055687  4962 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:37.065694  4962 raft_consensus.cc:515] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:37.068058  4962 leader_election.cc:290] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:37.069680  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:37.070590  4886 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:37.072284  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:37.072886  5032 raft_consensus.cc:3055] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:37.073179  5032 raft_consensus.cc:740] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:37.073868  5032 consensus_queue.cc:260] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:37.074853  5032 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:37.078590  4886 raft_consensus.cc:2468] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:37.080385  4927 leader_election.cc:304] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:37.081256  5854 raft_consensus.cc:2804] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:37.081655  5854 raft_consensus.cc:697] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:37.082356  5032 raft_consensus.cc:2468] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:37.082406  5854 consensus_queue.cc:237] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:37.093665  4711 catalog_manager.cc:5697] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:37.129884  5838 raft_consensus.cc:993] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: : Instructing follower 3f742efcbd8e4dea9c62e531ccaf31d6 to start an election
I20260709 12:29:37.130513  5854 raft_consensus.cc:1081] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Signalling peer 3f742efcbd8e4dea9c62e531ccaf31d6 to start an election
I20260709 12:29:37.132257  5032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
 from {username='slave'} at 127.0.0.1:60254
I20260709 12:29:37.132970  5032 raft_consensus.cc:493] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:37.133344  5032 raft_consensus.cc:3060] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:37.141048  5032 raft_consensus.cc:515] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:37.144277  5032 leader_election.cc:290] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:37.144435  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:37.145107  4886 raft_consensus.cc:3060] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:37.148185  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc20ff0e7f54f64a936b32f513913fc" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:37.148892  4962 raft_consensus.cc:3055] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:37.149161  4962 raft_consensus.cc:740] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:37.149935  4962 consensus_queue.cc:260] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:37.151242  4962 raft_consensus.cc:3060] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:37.153692  4886 raft_consensus.cc:2468] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:37.155014  5001 leader_election.cc:304] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:37.155861  5779 raft_consensus.cc:2804] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:37.156394  5779 raft_consensus.cc:697] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:37.157223  5779 consensus_queue.cc:237] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:37.158142  4962 raft_consensus.cc:2468] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:37.169364  4711 catalog_manager.cc:5697] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:37.309038  5032 raft_consensus.cc:1275] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.310526  5875 consensus_queue.cc:1048] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.326203  4964 raft_consensus.cc:1275] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.328171  5876 consensus_queue.cc:1048] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:37.358637  5032 raft_consensus.cc:1275] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.360117  5876 consensus_queue.cc:1048] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.373864  4964 raft_consensus.cc:1275] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:37.382987  4962 raft_consensus.cc:1275] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.386128  5875 consensus_queue.cc:1048] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:37.388072  5875 consensus_queue.cc:1048] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.406687  4886 raft_consensus.cc:1275] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:37.408977  5876 consensus_queue.cc:1048] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:37.434295  4890 raft_consensus.cc:1275] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.436089  5876 consensus_queue.cc:1048] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.447244  4962 raft_consensus.cc:1275] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.451228  5876 consensus_queue.cc:1048] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.596177  4886 raft_consensus.cc:1275] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:37.597879  5779 consensus_queue.cc:1048] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:37.611171  4958 raft_consensus.cc:1275] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:37.612546  5868 consensus_queue.cc:1048] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:37.665258  4890 raft_consensus.cc:1275] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:37.666625  5854 consensus_queue.cc:1048] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:37.681088  5035 raft_consensus.cc:1275] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:37.682680  5854 consensus_queue.cc:1048] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:37.774243 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:37.821585  4886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:37.822194  4886 raft_consensus.cc:606] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:37.824973  4886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:37.825599  4886 raft_consensus.cc:606] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:37.827893  4958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b790753ff8784fb4821657345fa7353e"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:35318
I20260709 12:29:37.828575  4958 raft_consensus.cc:606] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:37.831458  5032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:39950
I20260709 12:29:37.831925  5032 raft_consensus.cc:606] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:37.840961  4886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:37.841647  4886 raft_consensus.cc:606] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:37.842842 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:29:37.844621 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:37.845413 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:38.023749  5902 raft_consensus.cc:993] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.024343  5779 raft_consensus.cc:1081] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.026300  5655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53782
I20260709 12:29:38.026793  5655 raft_consensus.cc:493] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:38.027093  5655 raft_consensus.cc:3060] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.032377  5655 raft_consensus.cc:515] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.035233  5655 leader_election.cc:290] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:38.036142  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:38.036518  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c079e12498924bdaa8ba38c6c2c4dd6c" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:38.036851  4886 raft_consensus.cc:3060] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.037089  5035 raft_consensus.cc:3055] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:38.037446  5035 raft_consensus.cc:740] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:38.038211  5035 consensus_queue.cc:260] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.039451  5035 raft_consensus.cc:3060] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.043272  4886 raft_consensus.cc:2468] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:38.045558  5598 leader_election.cc:304] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:38.046262  5876 raft_consensus.cc:2804] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:38.046304  5035 raft_consensus.cc:2468] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:38.046794  5876 raft_consensus.cc:697] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:38.047720  5876 consensus_queue.cc:237] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.057978  4710 catalog_manager.cc:5697] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 1 to 2, leader changed from 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 2 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:38.117890  5865 raft_consensus.cc:993] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.118428  5852 raft_consensus.cc:1081] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.120100  5655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53762
I20260709 12:29:38.120676  5655 raft_consensus.cc:493] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:38.121068  5655 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:38.129132  5655 raft_consensus.cc:515] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.131510  5655 leader_election.cc:290] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:38.132350  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:38.132958  4886 raft_consensus.cc:3055] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:38.133260  4886 raft_consensus.cc:740] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:38.133989  4886 consensus_queue.cc:260] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.135281  4886 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:38.131685  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f790808b2394d2fbe8a000bb3f62ea8" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:38.136644  5032 raft_consensus.cc:3060] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:38.139971  4886 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:38.141091  5598 leader_election.cc:304] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:38.141912  5032 raft_consensus.cc:2468] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:38.141920  5876 raft_consensus.cc:2804] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:38.142742  5876 raft_consensus.cc:697] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:38.143625  5876 consensus_queue.cc:237] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.153795  4710 catalog_manager.cc:5697] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:38.172916  5865 raft_consensus.cc:993] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.173432  5852 raft_consensus.cc:1081] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.175032  5647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53762
I20260709 12:29:38.175654  5647 raft_consensus.cc:493] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:38.175959  5647 raft_consensus.cc:3060] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.182309  5647 raft_consensus.cc:515] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.184342  5647 leader_election.cc:290] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:38.184571  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:38.185024  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa2320b4fabb4defa55ae5284ad3f4e4" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:38.185308  5032 raft_consensus.cc:3060] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.185603  4886 raft_consensus.cc:3055] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:38.185913  4886 raft_consensus.cc:740] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:38.186674  4886 consensus_queue.cc:260] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.187942  4886 raft_consensus.cc:3060] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.192310  5032 raft_consensus.cc:2468] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:38.193660  5600 leader_election.cc:304] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:38.194361  4886 raft_consensus.cc:2468] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:38.194428  5876 raft_consensus.cc:2804] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:38.195941  5876 raft_consensus.cc:697] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:38.196846  5876 consensus_queue.cc:237] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:38.207543  4710 catalog_manager.cc:5697] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 2 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:38.257898  5905 raft_consensus.cc:993] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.258492  5838 raft_consensus.cc:1081] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:38.260334  5647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b790753ff8784fb4821657345fa7353e"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53768
I20260709 12:29:38.260818  5647 raft_consensus.cc:493] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:38.261139  5647 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:38.266484  5647 raft_consensus.cc:515] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:38.268646  5647 leader_election.cc:290] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 4 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:38.269793  4958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:38.270373  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b790753ff8784fb4821657345fa7353e" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:38.270493  4958 raft_consensus.cc:3055] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:38.271471  4958 raft_consensus.cc:740] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:38.271131  4886 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:38.272336  4958 consensus_queue.cc:260] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:38.273589  4958 raft_consensus.cc:3060] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:38.278782  4886 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 4.
I20260709 12:29:38.280002  5598 leader_election.cc:304] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:38.280884  5876 raft_consensus.cc:2804] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:38.281057  4958 raft_consensus.cc:2468] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 4.
I20260709 12:29:38.281379  5876 raft_consensus.cc:697] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 4 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:38.282480  5876 consensus_queue.cc:237] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:38.292796  4710 catalog_manager.cc:5697] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 3 to 4, leader changed from 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 4 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:38.349491  5865 raft_consensus.cc:993] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0fa1ec70394d466ab14173b48b67b8ed to start an election
I20260709 12:29:38.349946  5865 raft_consensus.cc:1081] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 0fa1ec70394d466ab14173b48b67b8ed to start an election
I20260709 12:29:38.351336  4962 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
 from {username='slave'} at 127.0.0.1:54570
I20260709 12:29:38.351783  4962 raft_consensus.cc:493] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:38.352293  4962 raft_consensus.cc:3060] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.358194  4962 raft_consensus.cc:515] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:38.359781  4962 leader_election.cc:290] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:38.360695  4886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:38.360878  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a10ed8c982e41c2b060892645dd7992" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:38.361334  4886 raft_consensus.cc:3055] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:38.361479  5032 raft_consensus.cc:3060] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.368710  5032 raft_consensus.cc:2468] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:38.361721  4886 raft_consensus.cc:740] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:38.370158  4929 leader_election.cc:304] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:38.371098  5916 raft_consensus.cc:2804] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:38.371840  5916 raft_consensus.cc:697] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:38.371027  4886 consensus_queue.cc:260] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:38.372960  5916 consensus_queue.cc:237] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:38.373389  4886 raft_consensus.cc:3060] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:38.382059  4886 raft_consensus.cc:2468] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:38.386237  4711 catalog_manager.cc:5697] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:38.621060  5032 raft_consensus.cc:1275] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:38.622320  5875 consensus_queue.cc:1048] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:38.641573  4886 raft_consensus.cc:1275] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:38.644018  5876 consensus_queue.cc:1048] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:38.689507  5032 raft_consensus.cc:1275] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:38.691532  5875 consensus_queue.cc:1048] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:38.704453  4886 raft_consensus.cc:1275] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:38.706002  5924 consensus_queue.cc:1048] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:38.725253  5032 raft_consensus.cc:1275] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:38.726838  5924 consensus_queue.cc:1048] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:38.757918  4958 raft_consensus.cc:1275] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 4 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:38.757918  4886 raft_consensus.cc:1275] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:38.759411  5875 consensus_queue.cc:1048] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:38.761051  5927 consensus_queue.cc:1048] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:38.768357  5032 raft_consensus.cc:1275] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:38.769955  5905 consensus_queue.cc:1048] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:38.796787  4890 raft_consensus.cc:1275] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:38.798700  5905 consensus_queue.cc:1048] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:38.814481  4890 raft_consensus.cc:1275] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 4 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:38.816342  5876 consensus_queue.cc:1048] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:38.882913 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:38.925861  4958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:35318
I20260709 12:29:38.926492  4958 raft_consensus.cc:606] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:38.929626  4890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23b4122ef56040c887e2e7481f657288"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:38.930121  4890 raft_consensus.cc:606] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:38.932672  4890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:38.933395  4890 raft_consensus.cc:606] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:38.935936  4890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:38.936631  4890 raft_consensus.cc:606] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:38.939270  4890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:38.939788  4890 raft_consensus.cc:606] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:38.941072 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:29:38.942575 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:38.943207 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:39.058274  5932 raft_consensus.cc:993] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0fa1ec70394d466ab14173b48b67b8ed to start an election
I20260709 12:29:39.058796  5852 raft_consensus.cc:1081] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 0fa1ec70394d466ab14173b48b67b8ed to start an election
I20260709 12:29:39.060335  4958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
 from {username='slave'} at 127.0.0.1:54570
I20260709 12:29:39.060897  4958 raft_consensus.cc:493] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.061259  4958 raft_consensus.cc:3060] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.067942  4958 raft_consensus.cc:515] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:39.069546  4958 leader_election.cc:290] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:39.070391  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:39.070545  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52764aff47af4493a2f25ca5f8b3387c" candidate_uuid: "0fa1ec70394d466ab14173b48b67b8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:39.071099  4890 raft_consensus.cc:3055] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.071202  5032 raft_consensus.cc:3060] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.071394  4890 raft_consensus.cc:740] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:39.072283  4890 consensus_queue.cc:260] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:39.073787  4890 raft_consensus.cc:3060] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.078006  5032 raft_consensus.cc:2468] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:39.079113  4929 leader_election.cc:304] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa1ec70394d466ab14173b48b67b8ed, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:39.079841  5916 raft_consensus.cc:2804] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.080336  5916 raft_consensus.cc:697] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Leader. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:39.081277  4890 raft_consensus.cc:2468] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa1ec70394d466ab14173b48b67b8ed in term 2.
I20260709 12:29:39.081192  5916 consensus_queue.cc:237] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:39.092537  4711 catalog_manager.cc:5697] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131). New cstate: current_term: 2 leader_uuid: "0fa1ec70394d466ab14173b48b67b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.138542  5852 raft_consensus.cc:993] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:39.139106  5932 raft_consensus.cc:1081] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:39.140784  5647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53762
I20260709 12:29:39.141363  5647 raft_consensus.cc:493] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.141672  5647 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:39.147668  5647 raft_consensus.cc:515] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:39.149899  5647 leader_election.cc:290] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:39.150234  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:39.150466  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64da30821dfb420183c42da517b4a6e1" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:39.151373  5032 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:39.151698  4890 raft_consensus.cc:3055] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:39.152035  4890 raft_consensus.cc:740] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:39.152840  4890 consensus_queue.cc:260] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:39.154110  4890 raft_consensus.cc:3060] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:39.158633  5032 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:39.161836  4890 raft_consensus.cc:2468] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:39.162910  5598 leader_election.cc:304] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:39.164142  5927 raft_consensus.cc:2804] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:39.164681  5927 raft_consensus.cc:697] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:39.165601  5927 consensus_queue.cc:237] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:39.177044  4711 catalog_manager.cc:5697] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.297739  5838 raft_consensus.cc:993] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:39.298182  5916 raft_consensus.cc:1081] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:39.299978  5647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53768
I20260709 12:29:39.300719  5647 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.301014  5647 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:39.306784  5647 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:39.309132  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:39.309746  4962 raft_consensus.cc:3055] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:39.310079  4962 raft_consensus.cc:740] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:39.310165  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07913583dcd7461b82e4125d7bf9d9f0" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:39.310765  5032 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:39.310974  4962 consensus_queue.cc:260] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:39.311900  5647 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:39.312978  5865 raft_consensus.cc:993] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 3f742efcbd8e4dea9c62e531ccaf31d6 to start an election
I20260709 12:29:39.313503  5865 raft_consensus.cc:1081] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 3f742efcbd8e4dea9c62e531ccaf31d6 to start an election
I20260709 12:29:39.314450  4962 raft_consensus.cc:3060] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:39.316457  5035 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23b4122ef56040c887e2e7481f657288"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
 from {username='slave'} at 127.0.0.1:60264
I20260709 12:29:39.317073  5035 raft_consensus.cc:493] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.317423  5035 raft_consensus.cc:3060] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.318835  5032 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:39.320132  5600 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 
I20260709 12:29:39.321055  5927 raft_consensus.cc:2804] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:39.322196  5927 raft_consensus.cc:697] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:39.322835  4962 raft_consensus.cc:2468] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:39.323052  5927 consensus_queue.cc:237] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:39.323904  5932 raft_consensus.cc:993] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:39.324416  5865 raft_consensus.cc:1081] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:39.326138  5035 raft_consensus.cc:515] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:39.326598  5647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53762
I20260709 12:29:39.327149  5647 raft_consensus.cc:493] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.327551  5647 raft_consensus.cc:3060] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.329181  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:39.329821  4890 raft_consensus.cc:3055] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.330103  4890 raft_consensus.cc:740] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:39.330852  4890 consensus_queue.cc:260] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:39.331867  4890 raft_consensus.cc:3060] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.333922  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4122ef56040c887e2e7481f657288" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:39.333721  5035 leader_election.cc:290] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:39.334551  4962 raft_consensus.cc:3060] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.335047  4711 catalog_manager.cc:5697] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:39.338657  4890 raft_consensus.cc:2468] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:39.340000  5001 leader_election.cc:304] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:39.340868  5902 raft_consensus.cc:2804] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.341343  5902 raft_consensus.cc:697] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:39.342092  5902 consensus_queue.cc:237] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:39.344161  4962 raft_consensus.cc:2468] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 2.
I20260709 12:29:39.346091  5647 raft_consensus.cc:515] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:39.348999  4890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:39.350500  4890 raft_consensus.cc:3055] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.350862  4890 raft_consensus.cc:740] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:39.351675  4890 consensus_queue.cc:260] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:39.353264  5647 leader_election.cc:290] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:39.352707  4890 raft_consensus.cc:3060] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.354337  4962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ceef50c6394676adfa8f58903d2b90" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:39.355913  4962 raft_consensus.cc:3060] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.362788  4962 raft_consensus.cc:2468] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:39.364107  5600 leader_election.cc:304] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:39.365109  5927 raft_consensus.cc:2804] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.365461  4890 raft_consensus.cc:2468] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:39.366307  5927 raft_consensus.cc:697] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:39.367334  5927 consensus_queue.cc:237] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:39.372088  4711 catalog_manager.cc:5697] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 2 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:39.386518  4711 catalog_manager.cc:5697] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 2 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.476197  4890 raft_consensus.cc:1275] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:39.477557  5916 consensus_queue.cc:1048] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:39.492283  5035 raft_consensus.cc:1275] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0fa1ec70394d466ab14173b48b67b8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:39.493579  5916 consensus_queue.cc:1048] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:39.635646  4890 raft_consensus.cc:1275] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:39.636917  5927 consensus_queue.cc:1048] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:39.659951  5035 raft_consensus.cc:1275] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:39.661684  5927 consensus_queue.cc:1048] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:39.728848  4958 raft_consensus.cc:1275] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:39.730018  5902 consensus_queue.cc:1048] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:39.741231  4890 raft_consensus.cc:1275] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:39.743968  5952 consensus_queue.cc:1048] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:39.742430  5035 raft_consensus.cc:1275] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:39.746070  5927 consensus_queue.cc:1048] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:39.772819  4958 raft_consensus.cc:1275] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:39.774256  5927 consensus_queue.cc:1048] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:39.821807  4886 raft_consensus.cc:1275] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:39.870244  5924 consensus_queue.cc:1048] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:29:39.994349 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:40.007627  4958 raft_consensus.cc:1275] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.028851  5966 consensus_queue.cc:1048] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:29:40.102749  4855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:44881, user_credentials={real_user=slave}} blocked reactor thread for 73736.9us
I20260709 12:29:40.272595  4961 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "351e55d393df4e159f251e3253097ed7"
dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
mode: GRACEFUL
new_leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
 from {username='slave'} at 127.0.0.1:35318
I20260709 12:29:40.273681  4961 raft_consensus.cc:606] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Received request to transfer leadership to TS 3f742efcbd8e4dea9c62e531ccaf31d6
I20260709 12:29:40.281600  5032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:39950
I20260709 12:29:40.282280  5032 raft_consensus.cc:606] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:40.285625  4885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b65cf7fce855449990ccd780e435d919"
dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:54568
I20260709 12:29:40.286257  4885 raft_consensus.cc:606] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:40.287803 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:40.289546 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:40.290758 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:40.325835  5970 raft_consensus.cc:993] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:40.326592  5932 raft_consensus.cc:1081] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:40.328212  5654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b65cf7fce855449990ccd780e435d919"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53762
I20260709 12:29:40.328886  5654 raft_consensus.cc:493] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.329375  5654 raft_consensus.cc:3060] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.336798  5654 raft_consensus.cc:515] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:40.338930  5654 leader_election.cc:290] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:40.340993  4885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:40.342403  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65cf7fce855449990ccd780e435d919" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:40.342893  4885 raft_consensus.cc:3055] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:40.343001  4961 raft_consensus.cc:3060] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.343405  4885 raft_consensus.cc:740] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3fc3aedb46404a8f0c54ac0bd04f83, State: Running, Role: LEADER
I20260709 12:29:40.344475  4885 consensus_queue.cc:260] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:40.347303  4885 raft_consensus.cc:3060] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.349848  5912 raft_consensus.cc:993] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:40.350574  5952 raft_consensus.cc:1081] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:40.351985  4961 raft_consensus.cc:2468] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:40.353477  5647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53782
I20260709 12:29:40.354125  5647 raft_consensus.cc:493] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.354174  5600 leader_election.cc:304] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:40.354523  5647 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:40.355296  5963 raft_consensus.cc:2804] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:40.355795  5963 raft_consensus.cc:697] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:40.356443  4885 raft_consensus.cc:2468] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:40.356940  5963 consensus_queue.cc:237] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } }
I20260709 12:29:40.364795  5647 raft_consensus.cc:515] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:40.367784  5032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:40.368435  5032 raft_consensus.cc:3055] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:40.368716  5032 raft_consensus.cc:740] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:40.369690  5647 leader_election.cc:290] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Requested vote from peers 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487), 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187)
I20260709 12:29:40.369451  5032 consensus_queue.cc:260] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:40.370766  4885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729ed090b704e7197d4f2df2b07eb29" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:40.371335  4885 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:40.371603  5032 raft_consensus.cc:3060] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:40.373401  4712 catalog_manager.cc:5697] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 1 to 2, leader changed from 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 2 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:40.377835  4885 raft_consensus.cc:2468] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:40.380159  5598 leader_election.cc:304] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:40.381273  5963 raft_consensus.cc:2804] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:40.381798  5963 raft_consensus.cc:697] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:40.382968  5963 consensus_queue.cc:237] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } }
I20260709 12:29:40.383754  5032 raft_consensus.cc:2468] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 3.
I20260709 12:29:40.408042  4712 catalog_manager.cc:5697] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 2 to 3, leader changed from 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 3 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:40.739269  5938 raft_consensus.cc:993] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: : Instructing follower 3f742efcbd8e4dea9c62e531ccaf31d6 to start an election
I20260709 12:29:40.739782  5938 raft_consensus.cc:1081] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Signalling peer 3f742efcbd8e4dea9c62e531ccaf31d6 to start an election
I20260709 12:29:40.741389  5032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "351e55d393df4e159f251e3253097ed7"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
 from {username='slave'} at 127.0.0.1:60254
I20260709 12:29:40.741957  5032 raft_consensus.cc:493] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.742476  5032 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:40.748035  5032 raft_consensus.cc:515] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:40.749532  5032 leader_election.cc:290] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 election: Requested vote from peers 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187), 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655)
I20260709 12:29:40.750605  4961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:40.751230  4961 raft_consensus.cc:3055] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:40.751621  4961 raft_consensus.cc:740] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0fa1ec70394d466ab14173b48b67b8ed, State: Running, Role: LEADER
I20260709 12:29:40.750512  4885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351e55d393df4e159f251e3253097ed7" candidate_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83"
I20260709 12:29:40.752590  4885 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:40.752704  4961 consensus_queue.cc:260] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:40.753854  4961 raft_consensus.cc:3060] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:40.758800  4885 raft_consensus.cc:2468] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 3.
I20260709 12:29:40.760222  5001 leader_election.cc:304] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6, 8c3fc3aedb46404a8f0c54ac0bd04f83; no voters: 
I20260709 12:29:40.760895  4961 raft_consensus.cc:2468] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f742efcbd8e4dea9c62e531ccaf31d6 in term 3.
I20260709 12:29:40.761174  5912 raft_consensus.cc:2804] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:40.761850  5912 raft_consensus.cc:697] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 LEADER]: Becoming Leader. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:40.762794  5912 consensus_queue.cc:237] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } }
I20260709 12:29:40.766556  4961 raft_consensus.cc:1275] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.768219  5967 consensus_queue.cc:1048] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:29:40.781351  4712 catalog_manager.cc:5697] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 reported cstate change: term changed from 2 to 3, leader changed from 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131) to 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132). New cstate: current_term: 3 leader_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:40.794009  4885 raft_consensus.cc:1275] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.795993  5967 consensus_queue.cc:1048] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.896962  5032 raft_consensus.cc:1275] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:40.898054  5967 consensus_queue.cc:1048] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:40.909436  4885 raft_consensus.cc:1275] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:40.911795  5966 consensus_queue.cc:1048] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:41.146343  4964 raft_consensus.cc:1275] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:41.148021  5912 consensus_queue.cc:1048] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:41.159436  4890 raft_consensus.cc:1275] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Refusing update from remote peer 3f742efcbd8e4dea9c62e531ccaf31d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:41.161816  5968 consensus_queue.cc:1048] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3fc3aedb46404a8f0c54ac0bd04f83" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:41.336472 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:41.375193  5035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919"
dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
mode: GRACEFUL
new_leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:39950
I20260709 12:29:41.375881  5035 raft_consensus.cc:606] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Received request to transfer leadership to TS 0ec932ee892248cda1f4e99e4ba7887e
I20260709 12:29:41.377385 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:29:41.379179 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:41.379976 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:41.599907  5912 raft_consensus.cc:993] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: : Instructing follower 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:41.600445  5968 raft_consensus.cc:1081] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Signalling peer 0ec932ee892248cda1f4e99e4ba7887e to start an election
I20260709 12:29:41.602103  5654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919"
dest_uuid: "0ec932ee892248cda1f4e99e4ba7887e"
 from {username='slave'} at 127.0.0.1:53782
I20260709 12:29:41.602665  5654 raft_consensus.cc:493] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:41.603080  5654 raft_consensus.cc:3060] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:41.610544  5654 raft_consensus.cc:515] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:41.612495  5654 leader_election.cc:290] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Requested vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132:42487)
I20260709 12:29:41.613284  4955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0fa1ec70394d466ab14173b48b67b8ed"
I20260709 12:29:41.613996  4955 raft_consensus.cc:3060] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:41.614234  5035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72db5336d073433fa404fdf28bdfa919" candidate_uuid: "0ec932ee892248cda1f4e99e4ba7887e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6"
I20260709 12:29:41.616326  5035 raft_consensus.cc:3055] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:41.616714  5035 raft_consensus.cc:740] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f742efcbd8e4dea9c62e531ccaf31d6, State: Running, Role: LEADER
I20260709 12:29:41.617633  5035 consensus_queue.cc:260] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:41.621577  5035 raft_consensus.cc:3060] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:41.623467  4955 raft_consensus.cc:2468] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:41.624727  5600 leader_election.cc:304] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed; no voters: 
I20260709 12:29:41.625448  5989 raft_consensus.cc:2804] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:41.625963  5989 raft_consensus.cc:697] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Becoming Leader. State: Replica: 0ec932ee892248cda1f4e99e4ba7887e, State: Running, Role: LEADER
I20260709 12:29:41.627015  5989 consensus_queue.cc:237] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:41.629419  5035 raft_consensus.cc:2468] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec932ee892248cda1f4e99e4ba7887e in term 2.
I20260709 12:29:41.637137  4712 catalog_manager.cc:5697] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e reported cstate change: term changed from 1 to 2, leader changed from 3f742efcbd8e4dea9c62e531ccaf31d6 (127.24.71.132) to 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129). New cstate: current_term: 2 leader_uuid: "0ec932ee892248cda1f4e99e4ba7887e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:42.157460  5035 raft_consensus.cc:1275] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:42.159700  5966 consensus_queue.cc:1048] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:42.172879  4955 raft_consensus.cc:1275] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Refusing update from remote peer 0ec932ee892248cda1f4e99e4ba7887e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:42.174835  5989 consensus_queue.cc:1048] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:42.422758 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:42.461172 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:42.464324 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:42.465061 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:42.507198 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:42.546721 24862 tablet_server.cc:179] TabletServer@127.24.71.129:44881 shutting down...
W20260709 12:29:42.578850  4929 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 41 similar messages]
I20260709 12:29:42.588557 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:29:42.588814  4929 consensus_peers.cc:597] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.589741 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.590891 24862 raft_consensus.cc:2243] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.591531 24862 raft_consensus.cc:2272] T 5f743d73272948abbd7fc320e050742c P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.594521 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.595248 24862 raft_consensus.cc:2243] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.595693 24862 raft_consensus.cc:2272] T d35c974e4cb84cf49a6143cc50956e93 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.598196 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.598814 24862 raft_consensus.cc:2243] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.599372 24862 raft_consensus.cc:2272] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.602918 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.603708 24862 raft_consensus.cc:2243] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:29:42.604909  4929 consensus_peers.cc:597] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.604933  4855 consensus_peers.cc:597] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.604980 24862 raft_consensus.cc:2272] T 5149b4ff21da4751975490772301b992 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.609054 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.610075 24862 raft_consensus.cc:2243] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.610919 24862 raft_consensus.cc:2272] T b65cf7fce855449990ccd780e435d919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.613564 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.614698 24862 raft_consensus.cc:2243] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.615409 24862 raft_consensus.cc:2272] T 827a3f43a1924ae880f566f2f314143f P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.617987  4929 consensus_peers.cc:597] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.618729 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.619400 24862 raft_consensus.cc:2243] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.619791 24862 raft_consensus.cc:2272] T 638da9762ab2493e85b9205f6c2679f9 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.623111 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.623965 24862 raft_consensus.cc:2243] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.624481 24862 raft_consensus.cc:2272] T 0783f64f319148009af4133e89e944ee P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.625181  4855 consensus_peers.cc:597] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.626489  4855 consensus_peers.cc:597] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.627146 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.627889 24862 raft_consensus.cc:2243] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.628317 24862 raft_consensus.cc:2272] T cc4a1cd832b5450f8efa34e455eeead0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.630569 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.631209 24862 raft_consensus.cc:2243] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.631668 24862 raft_consensus.cc:2272] T bea31c506847448087d1d664687c1d57 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.634083 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.634837 24862 raft_consensus.cc:2243] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.635344 24862 raft_consensus.cc:2272] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.638422 24862 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:42.638437  4855 consensus_peers.cc:597] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.639139 24862 raft_consensus.cc:2243] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.639573 24862 raft_consensus.cc:2272] T 57c5a10b9463451aaf68ce652335418b P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.642016 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.642652 24862 raft_consensus.cc:2243] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.643092 24862 raft_consensus.cc:2272] T 30e68af9cb68410b9539fea5e2191450 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.645387 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.646011 24862 raft_consensus.cc:2243] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.647316 24862 raft_consensus.cc:2272] T 4f790808b2394d2fbe8a000bb3f62ea8 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.650862 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.651821 24862 raft_consensus.cc:2243] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.653030 24862 raft_consensus.cc:2272] T 0165ba3321bd42768963b30a87b715e5 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.656821 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.658074 24862 raft_consensus.cc:2243] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.659442 24862 raft_consensus.cc:2272] T 8729ed090b704e7197d4f2df2b07eb29 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.663056 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.664021 24862 raft_consensus.cc:2243] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.665288 24862 raft_consensus.cc:2272] T 72db5336d073433fa404fdf28bdfa919 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.669164 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.670037 24862 raft_consensus.cc:2243] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.670701 24862 raft_consensus.cc:2272] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.671306  4855 consensus_peers.cc:597] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.673955 24862 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:42.673995  4855 consensus_peers.cc:597] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.675036 24862 raft_consensus.cc:2243] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.675588 24862 raft_consensus.cc:2272] T 31a7ec1dbc9448afaffb4f4291fbabed P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.678427 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.679186 24862 raft_consensus.cc:2243] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.679752 24862 raft_consensus.cc:2272] T e99cb592cbcf46d3bbf20936ff9ec873 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.683013 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.683720 24862 raft_consensus.cc:2243] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.684163 24862 raft_consensus.cc:2272] T 95c3d8ad3cb14b11aef810d11bc59682 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.686451 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.687043 24862 raft_consensus.cc:2243] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.687510 24862 raft_consensus.cc:2272] T 15e4db1a6dac4e348f16817448c5005e P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.689836 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.690593 24862 raft_consensus.cc:2243] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.691642 24862 raft_consensus.cc:2272] T fa2320b4fabb4defa55ae5284ad3f4e4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.694919 24862 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:42.695449  5003 consensus_peers.cc:597] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.695513 24862 raft_consensus.cc:2243] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.696305 24862 raft_consensus.cc:2272] T 34f3921f98794b60a18335cd330faa86 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.698523 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.699111 24862 raft_consensus.cc:2243] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.700022 24862 raft_consensus.cc:2272] T c079e12498924bdaa8ba38c6c2c4dd6c P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.702260 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.702914 24862 raft_consensus.cc:2243] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.703805 24862 raft_consensus.cc:2272] T b790753ff8784fb4821657345fa7353e P 0ec932ee892248cda1f4e99e4ba7887e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.706838 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.707616 24862 raft_consensus.cc:2243] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.708105 24862 raft_consensus.cc:2272] T a0d5e87b68e14a5580328547bf386316 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.711136 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.711951 24862 raft_consensus.cc:2243] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.712604 24862 raft_consensus.cc:2272] T 7abd4abb7c334f02ba9c4672289410f4 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.715998 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.716863 24862 raft_consensus.cc:2243] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.717913 24862 raft_consensus.cc:2272] T 45ceef50c6394676adfa8f58903d2b90 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.720345 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.720896 24862 raft_consensus.cc:2243] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.721302 24862 raft_consensus.cc:2272] T 3ef746273b574779b4dcb59a0d239aa2 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.724082 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.724864 24862 raft_consensus.cc:2243] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.725387 24862 raft_consensus.cc:2272] T 03b26e75deab440d913747aece108666 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.727551 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.728130 24862 raft_consensus.cc:2243] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.728955 24862 raft_consensus.cc:2272] T 66dc367942ca4feb9a1fd830092d860f P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.730962 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.731516 24862 raft_consensus.cc:2243] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.731914 24862 raft_consensus.cc:2272] T 1c15fed052da4a3392865a529df5eda0 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.734014 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.734656 24862 raft_consensus.cc:2243] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.735069 24862 raft_consensus.cc:2272] T a47bf01353704752b89641cdf6e88e53 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.737808 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.738338 24862 raft_consensus.cc:2243] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.738797 24862 raft_consensus.cc:2272] T 29feba7c56da402abe958d088341ea39 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.741779 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.742516 24862 raft_consensus.cc:2243] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:29:42.743392  5003 consensus_peers.cc:597] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.743527 24862 raft_consensus.cc:2272] T 07913583dcd7461b82e4125d7bf9d9f0 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.746004 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.746632 24862 raft_consensus.cc:2243] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.747128 24862 raft_consensus.cc:2272] T e359e160bb414d9099cff45b96f6b40d P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.750718 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.751480 24862 raft_consensus.cc:2243] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.752038 24862 raft_consensus.cc:2272] T 845a033bb8104895847b349dcb5eeccb P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.752434  5003 consensus_peers.cc:597] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.754909 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.755506 24862 raft_consensus.cc:2243] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.755889 24862 raft_consensus.cc:2272] T 747d7984e025401681cfe15c121b8a64 P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.758517 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.759177 24862 raft_consensus.cc:2243] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.760035 24862 raft_consensus.cc:2272] T 64da30821dfb420183c42da517b4a6e1 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.763101  4929 consensus_peers.cc:597] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.763301 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.764431 24862 raft_consensus.cc:2243] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.764988 24862 raft_consensus.cc:2272] T 92e5720333ba432baadd844189c9c80b P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.767661 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.768442 24862 raft_consensus.cc:2243] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.769001 24862 raft_consensus.cc:2272] T 26d588e01b9943119c0c0427f2bc7b6f P 0ec932ee892248cda1f4e99e4ba7887e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.772441 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.773171 24862 raft_consensus.cc:2243] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:42.773610  4929 consensus_peers.cc:597] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.773752 24862 raft_consensus.cc:2272] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.777325 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.778029 24862 raft_consensus.cc:2243] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:29:42.778067  4855 consensus_peers.cc:597] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.779335 24862 raft_consensus.cc:2272] T 3340b333db824849a2af865d2658fd89 P 0ec932ee892248cda1f4e99e4ba7887e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.782485 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:42.783023 24862 raft_consensus.cc:2243] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:42.783157  5003 consensus_peers.cc:597] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.783644 24862 raft_consensus.cc:2272] T bb55bf7ad4704d18bb564eeb5320fee8 P 0ec932ee892248cda1f4e99e4ba7887e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.801141  4929 consensus_peers.cc:597] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.809650  4855 consensus_peers.cc:597] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.821310  5003 consensus_peers.cc:597] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.823402  5003 consensus_peers.cc:597] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.825901  4929 consensus_peers.cc:597] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.853956  5003 consensus_peers.cc:597] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.877560 24862 tablet_server.cc:196] TabletServer@127.24.71.129:44881 shutdown complete.
W20260709 12:29:42.895493  4855 consensus_peers.cc:597] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.956164  4929 consensus_peers.cc:597] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.967557 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
W20260709 12:29:42.986068  5003 consensus_peers.cc:597] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.018628  4929 consensus_peers.cc:597] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.021852  5003 consensus_peers.cc:597] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.022686  4929 consensus_peers.cc:597] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.025381 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:43.026511 24862 tablet_replica.cc:333] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.027650 24862 raft_consensus.cc:2243] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.028326 24862 raft_consensus.cc:2272] T 45ceef50c6394676adfa8f58903d2b90 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.028616  5003 consensus_peers.cc:597] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.030480 24862 tablet_replica.cc:333] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.031209 24862 raft_consensus.cc:2243] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.031673 24862 raft_consensus.cc:2272] T fa2320b4fabb4defa55ae5284ad3f4e4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.033864 24862 tablet_replica.cc:333] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.034533 24862 raft_consensus.cc:2243] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.035459 24862 raft_consensus.cc:2272] T e359e160bb414d9099cff45b96f6b40d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.038070 24862 tablet_replica.cc:333] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.038962 24862 raft_consensus.cc:2243] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.039992 24862 raft_consensus.cc:2272] T 80aa6fe92f51449ab69485480db26075 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.042649 24862 tablet_replica.cc:333] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.043411 24862 raft_consensus.cc:2243] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.044488 24862 raft_consensus.cc:2272] T 322e052ba6ca4c2989a3fd9aea3efc60 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.046775 24862 tablet_replica.cc:333] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.047406 24862 raft_consensus.cc:2243] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.047900 24862 raft_consensus.cc:2272] T 845a033bb8104895847b349dcb5eeccb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.050477 24862 tablet_replica.cc:333] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.051364 24862 raft_consensus.cc:2243] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.051946 24862 raft_consensus.cc:2272] T 638da9762ab2493e85b9205f6c2679f9 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.055034 24862 tablet_replica.cc:333] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.055647 24862 raft_consensus.cc:2243] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.056352 24862 raft_consensus.cc:2272] T 3a10ed8c982e41c2b060892645dd7992 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.058496 24862 tablet_replica.cc:333] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.059118 24862 raft_consensus.cc:2243] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.059623 24862 raft_consensus.cc:2272] T 52764aff47af4493a2f25ca5f8b3387c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.061882 24862 tablet_replica.cc:333] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.062474 24862 raft_consensus.cc:2243] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.062920 24862 raft_consensus.cc:2272] T d35c974e4cb84cf49a6143cc50956e93 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.065657 24862 tablet_replica.cc:333] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.066272 24862 raft_consensus.cc:2243] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.067206 24862 raft_consensus.cc:2272] T 92e5720333ba432baadd844189c9c80b P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.069806 24862 tablet_replica.cc:333] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.070556 24862 raft_consensus.cc:2243] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.070972 24862 raft_consensus.cc:2272] T b65cf7fce855449990ccd780e435d919 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.073822 24862 tablet_replica.cc:333] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
W20260709 12:29:43.074129  5003 consensus_peers.cc:597] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Couldn't send request to peer 0ec932ee892248cda1f4e99e4ba7887e. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.074697 24862 raft_consensus.cc:2243] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.075177 24862 raft_consensus.cc:2272] T 23b4122ef56040c887e2e7481f657288 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.077569 24862 tablet_replica.cc:333] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.078228 24862 raft_consensus.cc:2243] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.078684 24862 raft_consensus.cc:2272] T 9ce6101a3bd546fe9ae5ba5767af6277 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.080029  5001 consensus_peers.cc:597] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.081631 24862 tablet_replica.cc:333] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.082383 24862 raft_consensus.cc:2243] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.082882 24862 raft_consensus.cc:2272] T 3340b333db824849a2af865d2658fd89 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.084125  5001 consensus_peers.cc:597] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.085453 24862 tablet_replica.cc:333] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.086117 24862 raft_consensus.cc:2243] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.086544 24862 raft_consensus.cc:2272] T 7909fe384cf84b089c40d28a15fd8869 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.088829 24862 tablet_replica.cc:333] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.089434 24862 raft_consensus.cc:2243] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.090227 24862 raft_consensus.cc:2272] T 15e4db1a6dac4e348f16817448c5005e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.092392 24862 tablet_replica.cc:333] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.092943 24862 raft_consensus.cc:2243] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.093991 24862 raft_consensus.cc:2272] T 07d989f2ffa44c5480a996303eb45fbb P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.096100 24862 tablet_replica.cc:333] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.096680 24862 raft_consensus.cc:2243] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.097133 24862 raft_consensus.cc:2272] T e99cb592cbcf46d3bbf20936ff9ec873 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.099903 24862 tablet_replica.cc:333] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.100772 24862 raft_consensus.cc:2243] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.101382 24862 raft_consensus.cc:2272] T 95c3d8ad3cb14b11aef810d11bc59682 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.104213 24862 tablet_replica.cc:333] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.104856 24862 raft_consensus.cc:2243] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.105350 24862 raft_consensus.cc:2272] T 4f790808b2394d2fbe8a000bb3f62ea8 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.108179 24862 tablet_replica.cc:333] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.109009 24862 raft_consensus.cc:2243] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.110083 24862 raft_consensus.cc:2272] T 30e68af9cb68410b9539fea5e2191450 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.112613 24862 tablet_replica.cc:333] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.113411 24862 raft_consensus.cc:2243] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.114076 24862 raft_consensus.cc:2272] T 0165ba3321bd42768963b30a87b715e5 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.116827 24862 tablet_replica.cc:333] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.117688 24862 raft_consensus.cc:2243] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.123250 24862 raft_consensus.cc:2272] T 31a7ec1dbc9448afaffb4f4291fbabed P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.127136 24862 tablet_replica.cc:333] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.128080 24862 raft_consensus.cc:2243] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.128813 24862 raft_consensus.cc:2272] T b790753ff8784fb4821657345fa7353e P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.132117 24862 tablet_replica.cc:333] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.133003 24862 raft_consensus.cc:2243] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.133654 24862 raft_consensus.cc:2272] T c079e12498924bdaa8ba38c6c2c4dd6c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.134680  5001 consensus_peers.cc:597] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.136498 24862 tablet_replica.cc:333] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.137467 24862 raft_consensus.cc:2243] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.138036 24862 raft_consensus.cc:2272] T a0d5e87b68e14a5580328547bf386316 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.141750 24862 tablet_replica.cc:333] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.142402 24862 raft_consensus.cc:2243] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.142963 24862 raft_consensus.cc:2272] T 64b05cbdc6f14118b098b8dd06d859c3 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.145100 24862 tablet_replica.cc:333] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.145696 24862 raft_consensus.cc:2243] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.146139 24862 raft_consensus.cc:2272] T 7abd4abb7c334f02ba9c4672289410f4 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.148715 24862 tablet_replica.cc:333] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.149485 24862 raft_consensus.cc:2243] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.150087 24862 raft_consensus.cc:2272] T 64da30821dfb420183c42da517b4a6e1 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.153234 24862 tablet_replica.cc:333] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.154066 24862 raft_consensus.cc:2243] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.155107 24862 raft_consensus.cc:2272] T 747d7984e025401681cfe15c121b8a64 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.155246  4927 consensus_peers.cc:597] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.156035  4927 consensus_peers.cc:597] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.157639 24862 tablet_replica.cc:333] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.158394 24862 raft_consensus.cc:2243] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.158933 24862 raft_consensus.cc:2272] T 351e55d393df4e159f251e3253097ed7 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.161373 24862 tablet_replica.cc:333] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.161989 24862 raft_consensus.cc:2243] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.163026 24862 raft_consensus.cc:2272] T 83e9766db6054724a5055336fa6baa91 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.166479 24862 tablet_replica.cc:333] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
W20260709 12:29:43.166551  4927 consensus_peers.cc:597] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.167196  4927 consensus_peers.cc:597] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.167505 24862 raft_consensus.cc:2243] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.167982 24862 raft_consensus.cc:2272] T 999b4d2234f74d80ad85ddf37ec99b01 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.170157 24862 tablet_replica.cc:333] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.170853 24862 raft_consensus.cc:2243] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.171633 24862 raft_consensus.cc:2272] T 03b26e75deab440d913747aece108666 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.174794 24862 tablet_replica.cc:333] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.175628 24862 raft_consensus.cc:2243] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.176332 24862 raft_consensus.cc:2272] T cc4a1cd832b5450f8efa34e455eeead0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.176512  5001 consensus_peers.cc:597] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.179205 24862 tablet_replica.cc:333] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.179893 24862 raft_consensus.cc:2243] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.180462 24862 raft_consensus.cc:2272] T 8729ed090b704e7197d4f2df2b07eb29 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.182452 24862 tablet_replica.cc:333] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.183106 24862 raft_consensus.cc:2243] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.183584 24862 raft_consensus.cc:2272] T 5dc20ff0e7f54f64a936b32f513913fc P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.187346 24862 tablet_replica.cc:333] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.188316 24862 raft_consensus.cc:2243] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.188887 24862 raft_consensus.cc:2272] T 1c15fed052da4a3392865a529df5eda0 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.191670 24862 tablet_replica.cc:333] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.192700 24862 raft_consensus.cc:2243] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.193833 24862 raft_consensus.cc:2272] T 5f743d73272948abbd7fc320e050742c P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.197402 24862 tablet_replica.cc:333] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.197964 24862 raft_consensus.cc:2243] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.198890 24862 raft_consensus.cc:2272] T a47bf01353704752b89641cdf6e88e53 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.202139 24862 tablet_replica.cc:333] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.202987 24862 raft_consensus.cc:2243] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.203682 24862 raft_consensus.cc:2272] T 54153631246c4cc68f62a6d2724ab306 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.203835  4927 consensus_peers.cc:597] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.206533  5001 consensus_peers.cc:597] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.206889 24862 tablet_replica.cc:333] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.207576 24862 raft_consensus.cc:2243] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.208611 24862 raft_consensus.cc:2272] T 79100fd1ff6942f5b408d20820466ef2 P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.211443 24862 tablet_replica.cc:333] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83: stopping tablet replica
I20260709 12:29:43.212039 24862 raft_consensus.cc:2243] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.213088 24862 raft_consensus.cc:2272] T 0799f16a2c4d4001826b14d3b2f5f42d P 8c3fc3aedb46404a8f0c54ac0bd04f83 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.245723  4927 consensus_peers.cc:597] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.273780  5001 consensus_peers.cc:597] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.282788  4927 consensus_peers.cc:597] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.294461  5001 consensus_peers.cc:597] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:33187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.304557 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
I20260709 12:29:43.385329 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
W20260709 12:29:43.415625  5003 consensus_peers.cc:597] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.429960 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:43.430974 24862 tablet_replica.cc:333] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.432014 24862 raft_consensus.cc:2243] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.432703 24862 raft_consensus.cc:2272] T 45ceef50c6394676adfa8f58903d2b90 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.435765 24862 tablet_replica.cc:333] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
W20260709 12:29:43.436488  4927 consensus_peers.cc:597] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:33187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.436682 24862 raft_consensus.cc:2243] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.437667 24862 raft_consensus.cc:2272] T bea31c506847448087d1d664687c1d57 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.440062 24862 tablet_replica.cc:333] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.440723 24862 raft_consensus.cc:2243] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.441179 24862 raft_consensus.cc:2272] T 827a3f43a1924ae880f566f2f314143f P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.443240 24862 tablet_replica.cc:333] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.443847 24862 raft_consensus.cc:2243] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.444346 24862 raft_consensus.cc:2272] T 07d989f2ffa44c5480a996303eb45fbb P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.447482 24862 tablet_replica.cc:333] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.448335 24862 raft_consensus.cc:2243] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.448805 24862 raft_consensus.cc:2272] T 34f3921f98794b60a18335cd330faa86 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.450973  4927 consensus_peers.cc:597] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:33187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.451942 24862 tablet_replica.cc:333] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.452881 24862 raft_consensus.cc:2243] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.453480 24862 raft_consensus.cc:2272] T 322e052ba6ca4c2989a3fd9aea3efc60 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.453403  4927 consensus_peers.cc:597] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:33187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.455928 24862 tablet_replica.cc:333] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.456760 24862 raft_consensus.cc:2243] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.457765 24862 raft_consensus.cc:2272] T 52764aff47af4493a2f25ca5f8b3387c P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.460907 24862 tablet_replica.cc:333] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.461896 24862 raft_consensus.cc:2243] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.462883 24862 raft_consensus.cc:2272] T 638da9762ab2493e85b9205f6c2679f9 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.465978 24862 tablet_replica.cc:333] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.466843 24862 raft_consensus.cc:2243] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.467948 24862 raft_consensus.cc:2272] T 3a10ed8c982e41c2b060892645dd7992 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.470983 24862 tablet_replica.cc:333] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
W20260709 12:29:43.471028  5001 consensus_peers.cc:597] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:33187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.471948 24862 raft_consensus.cc:2243] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.472551 24862 raft_consensus.cc:2272] T 0783f64f319148009af4133e89e944ee P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.475335 24862 tablet_replica.cc:333] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.476116 24862 raft_consensus.cc:2243] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.477190 24862 raft_consensus.cc:2272] T d35c974e4cb84cf49a6143cc50956e93 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.480055 24862 tablet_replica.cc:333] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.480683 24862 raft_consensus.cc:2243] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.481124 24862 raft_consensus.cc:2272] T 4831fc84d91e4eafbc37bf86cd29ad5e P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.483556 24862 tablet_replica.cc:333] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.484297 24862 raft_consensus.cc:2243] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.484753 24862 raft_consensus.cc:2272] T b65cf7fce855449990ccd780e435d919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.486750 24862 tablet_replica.cc:333] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.487342 24862 raft_consensus.cc:2243] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.487799 24862 raft_consensus.cc:2272] T 23b4122ef56040c887e2e7481f657288 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.489941 24862 tablet_replica.cc:333] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.490724 24862 raft_consensus.cc:2243] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.491752 24862 raft_consensus.cc:2272] T bb55bf7ad4704d18bb564eeb5320fee8 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.492672  5001 consensus_peers.cc:597] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 8c3fc3aedb46404a8f0c54ac0bd04f83 (127.24.71.130:33187): Couldn't send request to peer 8c3fc3aedb46404a8f0c54ac0bd04f83. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:33187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.494127 24862 tablet_replica.cc:333] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.494809 24862 raft_consensus.cc:2243] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.495586 24862 raft_consensus.cc:2272] T 9ce6101a3bd546fe9ae5ba5767af6277 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.497767 24862 tablet_replica.cc:333] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.498339 24862 raft_consensus.cc:2243] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.498780 24862 raft_consensus.cc:2272] T 3340b333db824849a2af865d2658fd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.500885 24862 tablet_replica.cc:333] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.501461 24862 raft_consensus.cc:2243] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.501878 24862 raft_consensus.cc:2272] T 30e68af9cb68410b9539fea5e2191450 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.503804 24862 tablet_replica.cc:333] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.504526 24862 raft_consensus.cc:2243] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.505342 24862 raft_consensus.cc:2272] T 57c5a10b9463451aaf68ce652335418b P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.507361 24862 tablet_replica.cc:333] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.507937 24862 raft_consensus.cc:2243] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.508459 24862 raft_consensus.cc:2272] T 0165ba3321bd42768963b30a87b715e5 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.510546 24862 tablet_replica.cc:333] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.511087 24862 raft_consensus.cc:2243] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.511504 24862 raft_consensus.cc:2272] T 5149b4ff21da4751975490772301b992 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.513464 24862 tablet_replica.cc:333] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.514060 24862 raft_consensus.cc:2243] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.514500 24862 raft_consensus.cc:2272] T 26d588e01b9943119c0c0427f2bc7b6f P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.516486 24862 tablet_replica.cc:333] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.517024 24862 raft_consensus.cc:2243] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.517774 24862 raft_consensus.cc:2272] T 7abd4abb7c334f02ba9c4672289410f4 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.519621 24862 tablet_replica.cc:333] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.520105 24862 raft_consensus.cc:2243] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.520569 24862 raft_consensus.cc:2272] T 64b05cbdc6f14118b098b8dd06d859c3 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.522444 24862 tablet_replica.cc:333] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.522920 24862 raft_consensus.cc:2243] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.523367 24862 raft_consensus.cc:2272] T b790753ff8784fb4821657345fa7353e P 0fa1ec70394d466ab14173b48b67b8ed [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.525420 24862 tablet_replica.cc:333] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.525897 24862 raft_consensus.cc:2243] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.526760 24862 raft_consensus.cc:2272] T 7909fe384cf84b089c40d28a15fd8869 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.528847 24862 tablet_replica.cc:333] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.529457 24862 raft_consensus.cc:2243] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.529990 24862 raft_consensus.cc:2272] T e359e160bb414d9099cff45b96f6b40d P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.532051 24862 tablet_replica.cc:333] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.532629 24862 raft_consensus.cc:2243] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.533016 24862 raft_consensus.cc:2272] T 80aa6fe92f51449ab69485480db26075 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.534904 24862 tablet_replica.cc:333] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.535429 24862 raft_consensus.cc:2243] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.535858 24862 raft_consensus.cc:2272] T 07913583dcd7461b82e4125d7bf9d9f0 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.537832 24862 tablet_replica.cc:333] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.538532 24862 raft_consensus.cc:2243] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.539254 24862 raft_consensus.cc:2272] T 66dc367942ca4feb9a1fd830092d860f P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.542191 24862 tablet_replica.cc:333] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.543056 24862 raft_consensus.cc:2243] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.543681 24862 raft_consensus.cc:2272] T 83e9766db6054724a5055336fa6baa91 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.546272 24862 tablet_replica.cc:333] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.546962 24862 raft_consensus.cc:2243] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.547976 24862 raft_consensus.cc:2272] T 999b4d2234f74d80ad85ddf37ec99b01 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.550621 24862 tablet_replica.cc:333] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.551210 24862 raft_consensus.cc:2243] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.551641 24862 raft_consensus.cc:2272] T 72db5336d073433fa404fdf28bdfa919 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.553685 24862 tablet_replica.cc:333] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.554216 24862 raft_consensus.cc:2243] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.554674 24862 raft_consensus.cc:2272] T 5dc20ff0e7f54f64a936b32f513913fc P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.556706 24862 tablet_replica.cc:333] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.557242 24862 raft_consensus.cc:2243] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.557668 24862 raft_consensus.cc:2272] T 79100fd1ff6942f5b408d20820466ef2 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.559638 24862 tablet_replica.cc:333] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.560158 24862 raft_consensus.cc:2243] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.560709 24862 raft_consensus.cc:2272] T 0799f16a2c4d4001826b14d3b2f5f42d P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.562672 24862 tablet_replica.cc:333] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.563293 24862 raft_consensus.cc:2243] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.564224 24862 raft_consensus.cc:2272] T 1c15fed052da4a3392865a529df5eda0 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.566306 24862 tablet_replica.cc:333] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.566883 24862 raft_consensus.cc:2243] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.567485 24862 raft_consensus.cc:2272] T 5f743d73272948abbd7fc320e050742c P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.569881 24862 tablet_replica.cc:333] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.570398 24862 raft_consensus.cc:2243] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.570838 24862 raft_consensus.cc:2272] T a47bf01353704752b89641cdf6e88e53 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.572908 24862 tablet_replica.cc:333] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.573562 24862 raft_consensus.cc:2243] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.574383 24862 raft_consensus.cc:2272] T 29feba7c56da402abe958d088341ea39 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.576661 24862 tablet_replica.cc:333] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.577315 24862 raft_consensus.cc:2243] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.578382 24862 raft_consensus.cc:2272] T 54153631246c4cc68f62a6d2724ab306 P 0fa1ec70394d466ab14173b48b67b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.581647 24862 tablet_replica.cc:333] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.582405 24862 raft_consensus.cc:2243] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.582940 24862 raft_consensus.cc:2272] T 351e55d393df4e159f251e3253097ed7 P 0fa1ec70394d466ab14173b48b67b8ed [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.583374  5003 consensus_peers.cc:597] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.585393 24862 tablet_replica.cc:333] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.586153 24862 raft_consensus.cc:2243] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.586688 24862 raft_consensus.cc:2272] T 3ef746273b574779b4dcb59a0d239aa2 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.589586 24862 tablet_replica.cc:333] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed: stopping tablet replica
I20260709 12:29:43.590245 24862 raft_consensus.cc:2243] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.591092 24862 raft_consensus.cc:2272] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 0fa1ec70394d466ab14173b48b67b8ed [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.595234  5003 consensus_peers.cc:597] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.617295  5003 consensus_peers.cc:597] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.641399  5003 consensus_peers.cc:597] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.663003 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:29:43.674981  5968 raft_consensus.cc:493] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:43.675539  5968 raft_consensus.cc:515] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:43.677554  5968 leader_election.cc:290] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
W20260709 12:29:43.682080  5003 leader_election.cc:336] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Network error: Client connection negotiation failed: client connection to 127.24.71.131:41655: connect: Connection refused (error 111)
W20260709 12:29:43.682794  5003 leader_election.cc:336] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111)
I20260709 12:29:43.683142  5003 leader_election.cc:304] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:43.683909  5968 raft_consensus.cc:2749] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:29:43.723434  5003 consensus_peers.cc:597] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.131:41655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.742765 24862 tablet_server.cc:179] TabletServer@127.24.71.132:0 shutting down...
W20260709 12:29:43.752614  5003 consensus_peers.cc:597] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.131:41655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.793074 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:43.793931 24862 tablet_replica.cc:333] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.794525 24862 raft_consensus.cc:2243] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.794966 24862 raft_consensus.cc:2272] T bb55bf7ad4704d18bb564eeb5320fee8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.797159 24862 tablet_replica.cc:333] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.797827 24862 raft_consensus.cc:2243] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.798219 24862 raft_consensus.cc:2272] T fa2320b4fabb4defa55ae5284ad3f4e4 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.798854  5912 raft_consensus.cc:493] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:43.799360  5912 raft_consensus.cc:515] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:43.800935 24862 tablet_replica.cc:333] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.801435  5912 leader_election.cc:290] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:43.801849 24862 raft_consensus.cc:2243] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.802515 24862 raft_consensus.cc:2272] T bea31c506847448087d1d664687c1d57 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.805495 24862 tablet_replica.cc:333] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
W20260709 12:29:43.805665  5003 leader_election.cc:336] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Network error: Client connection negotiation failed: client connection to 127.24.71.131:41655: connect: Connection refused (error 111)
I20260709 12:29:43.806341 24862 raft_consensus.cc:2243] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:29:43.806529  5003 leader_election.cc:336] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111)
I20260709 12:29:43.807015  5003 leader_election.cc:304] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:43.807389 24862 raft_consensus.cc:2272] T 827a3f43a1924ae880f566f2f314143f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.807574  5912 raft_consensus.cc:2749] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:43.810077 24862 tablet_replica.cc:333] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.811127 24862 raft_consensus.cc:2243] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.812395 24862 raft_consensus.cc:2272] T 34f3921f98794b60a18335cd330faa86 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.815351 24862 tablet_replica.cc:333] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.815990 24862 raft_consensus.cc:2243] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.816541 24862 raft_consensus.cc:2272] T 3a10ed8c982e41c2b060892645dd7992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.818513 24862 tablet_replica.cc:333] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.819095 24862 raft_consensus.cc:2243] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.819840 24862 raft_consensus.cc:2272] T 845a033bb8104895847b349dcb5eeccb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.822579 24862 tablet_replica.cc:333] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.823478 24862 raft_consensus.cc:2243] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.824018 24862 raft_consensus.cc:2272] T 52764aff47af4493a2f25ca5f8b3387c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:43.825923  5003 consensus_peers.cc:597] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 -> Peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Couldn't send request to peer 0fa1ec70394d466ab14173b48b67b8ed. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.131:41655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:43.826665 24862 tablet_replica.cc:333] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.827296 24862 raft_consensus.cc:2243] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.828078 24862 raft_consensus.cc:2272] T 4831fc84d91e4eafbc37bf86cd29ad5e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.830084 24862 tablet_replica.cc:333] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.830662 24862 raft_consensus.cc:2243] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.831393 24862 raft_consensus.cc:2272] T 23b4122ef56040c887e2e7481f657288 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.833328 24862 tablet_replica.cc:333] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.833895 24862 raft_consensus.cc:2243] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.834342 24862 raft_consensus.cc:2272] T 07d989f2ffa44c5480a996303eb45fbb P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.836400 24862 tablet_replica.cc:333] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.836969 24862 raft_consensus.cc:2243] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.837380 24862 raft_consensus.cc:2272] T 15e4db1a6dac4e348f16817448c5005e P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.839306 24862 tablet_replica.cc:333] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.839880 24862 raft_consensus.cc:2243] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.840724 24862 raft_consensus.cc:2272] T e99cb592cbcf46d3bbf20936ff9ec873 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.842724 24862 tablet_replica.cc:333] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.843472 24862 raft_consensus.cc:2243] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.844451 24862 raft_consensus.cc:2272] T 95c3d8ad3cb14b11aef810d11bc59682 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.846966 24862 tablet_replica.cc:333] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.847653 24862 raft_consensus.cc:2243] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.848124 24862 raft_consensus.cc:2272] T 5149b4ff21da4751975490772301b992 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.850513 24862 tablet_replica.cc:333] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.851234 24862 raft_consensus.cc:2243] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.851748 24862 raft_consensus.cc:2272] T 92e5720333ba432baadd844189c9c80b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.854614 24862 tablet_replica.cc:333] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.855252 24862 raft_consensus.cc:2243] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.856299 24862 raft_consensus.cc:2272] T 26d588e01b9943119c0c0427f2bc7b6f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.857950  5912 raft_consensus.cc:493] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ec932ee892248cda1f4e99e4ba7887e)
I20260709 12:29:43.858414  5912 raft_consensus.cc:515] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa1ec70394d466ab14173b48b67b8ed" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41655 } } peers { permanent_uuid: "3f742efcbd8e4dea9c62e531ccaf31d6" member_type: VOTER last_known_addr { host: "127.24.71.132" port: 42487 } } peers { permanent_uuid: "0ec932ee892248cda1f4e99e4ba7887e" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 44881 } }
I20260709 12:29:43.859112 24862 tablet_replica.cc:333] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.860018 24862 raft_consensus.cc:2243] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.860438  5912 leader_election.cc:290] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655), 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881)
I20260709 12:29:43.860574 24862 raft_consensus.cc:2272] T 7909fe384cf84b089c40d28a15fd8869 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.862756 24862 tablet_replica.cc:333] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.863544 24862 raft_consensus.cc:2243] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.864084 24862 raft_consensus.cc:2272] T c079e12498924bdaa8ba38c6c2c4dd6c P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.866943 24862 tablet_replica.cc:333] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
W20260709 12:29:43.867449  5003 leader_election.cc:336] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0fa1ec70394d466ab14173b48b67b8ed (127.24.71.131:41655): Network error: Client connection negotiation failed: client connection to 127.24.71.131:41655: connect: Connection refused (error 111)
I20260709 12:29:43.867745 24862 raft_consensus.cc:2243] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:29:43.868151  5003 leader_election.cc:336] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ec932ee892248cda1f4e99e4ba7887e (127.24.71.129:44881): Network error: Client connection negotiation failed: client connection to 127.24.71.129:44881: connect: Connection refused (error 111)
I20260709 12:29:43.868528  5003 leader_election.cc:304] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f742efcbd8e4dea9c62e531ccaf31d6; no voters: 0ec932ee892248cda1f4e99e4ba7887e, 0fa1ec70394d466ab14173b48b67b8ed
I20260709 12:29:43.868866 24862 raft_consensus.cc:2272] T a0d5e87b68e14a5580328547bf386316 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.869306  5912 raft_consensus.cc:2749] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:43.871284 24862 tablet_replica.cc:333] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.871977 24862 raft_consensus.cc:2243] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.872816 24862 raft_consensus.cc:2272] T 64b05cbdc6f14118b098b8dd06d859c3 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.874799 24862 tablet_replica.cc:333] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.875352 24862 raft_consensus.cc:2243] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.875777 24862 raft_consensus.cc:2272] T 29feba7c56da402abe958d088341ea39 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.877846 24862 tablet_replica.cc:333] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.878547 24862 raft_consensus.cc:2243] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.879038 24862 raft_consensus.cc:2272] T 54153631246c4cc68f62a6d2724ab306 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.881125 24862 tablet_replica.cc:333] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.881704 24862 raft_consensus.cc:2243] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.882154 24862 raft_consensus.cc:2272] T 64da30821dfb420183c42da517b4a6e1 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.884363 24862 tablet_replica.cc:333] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.885027 24862 raft_consensus.cc:2243] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.885465 24862 raft_consensus.cc:2272] T 747d7984e025401681cfe15c121b8a64 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.887331 24862 tablet_replica.cc:333] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.887873 24862 raft_consensus.cc:2243] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.888695 24862 raft_consensus.cc:2272] T 351e55d393df4e159f251e3253097ed7 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.890609 24862 tablet_replica.cc:333] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.891134 24862 raft_consensus.cc:2243] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.891908 24862 raft_consensus.cc:2272] T 3ef746273b574779b4dcb59a0d239aa2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.894023 24862 tablet_replica.cc:333] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.894743 24862 raft_consensus.cc:2243] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.895241 24862 raft_consensus.cc:2272] T 07fab4a18b1e4d2faa9f58d62f7ecd89 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.898057 24862 tablet_replica.cc:333] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.898645 24862 raft_consensus.cc:2243] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.899117 24862 raft_consensus.cc:2272] T 31a7ec1dbc9448afaffb4f4291fbabed P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.901418 24862 tablet_replica.cc:333] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.901963 24862 raft_consensus.cc:2243] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.902431 24862 raft_consensus.cc:2272] T 03b26e75deab440d913747aece108666 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.904467 24862 tablet_replica.cc:333] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.904989 24862 raft_consensus.cc:2243] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.905419 24862 raft_consensus.cc:2272] T 66dc367942ca4feb9a1fd830092d860f P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.907251 24862 tablet_replica.cc:333] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.907785 24862 raft_consensus.cc:2243] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.908623 24862 raft_consensus.cc:2272] T 0783f64f319148009af4133e89e944ee P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.910750 24862 tablet_replica.cc:333] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.911288 24862 raft_consensus.cc:2243] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.912014 24862 raft_consensus.cc:2272] T cc4a1cd832b5450f8efa34e455eeead0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.914032 24862 tablet_replica.cc:333] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.914636 24862 raft_consensus.cc:2243] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.915063 24862 raft_consensus.cc:2272] T 8729ed090b704e7197d4f2df2b07eb29 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.917033 24862 tablet_replica.cc:333] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.917578 24862 raft_consensus.cc:2243] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.917990 24862 raft_consensus.cc:2272] T 72db5336d073433fa404fdf28bdfa919 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.919839 24862 tablet_replica.cc:333] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.920420 24862 raft_consensus.cc:2243] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:43.921258 24862 raft_consensus.cc:2272] T 5dc20ff0e7f54f64a936b32f513913fc P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.923197 24862 tablet_replica.cc:333] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.923789 24862 raft_consensus.cc:2243] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.924302 24862 raft_consensus.cc:2272] T 4f790808b2394d2fbe8a000bb3f62ea8 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.926234 24862 tablet_replica.cc:333] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.926782 24862 raft_consensus.cc:2243] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.927187 24862 raft_consensus.cc:2272] T 57c5a10b9463451aaf68ce652335418b P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.929128 24862 tablet_replica.cc:333] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.929644 24862 raft_consensus.cc:2243] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.930052 24862 raft_consensus.cc:2272] T 80aa6fe92f51449ab69485480db26075 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.932018 24862 tablet_replica.cc:333] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.932783 24862 raft_consensus.cc:2243] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.933292 24862 raft_consensus.cc:2272] T 07913583dcd7461b82e4125d7bf9d9f0 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.935241 24862 tablet_replica.cc:333] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.935767 24862 raft_consensus.cc:2243] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.936208 24862 raft_consensus.cc:2272] T 79100fd1ff6942f5b408d20820466ef2 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.938246 24862 tablet_replica.cc:333] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.938750 24862 raft_consensus.cc:2243] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.939154 24862 raft_consensus.cc:2272] T 0799f16a2c4d4001826b14d3b2f5f42d P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.941120 24862 tablet_replica.cc:333] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.941614 24862 raft_consensus.cc:2243] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.942015 24862 raft_consensus.cc:2272] T 83e9766db6054724a5055336fa6baa91 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:43.943898 24862 tablet_replica.cc:333] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6: stopping tablet replica
I20260709 12:29:43.944406 24862 raft_consensus.cc:2243] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:43.944808 24862 raft_consensus.cc:2272] T 999b4d2234f74d80ad85ddf37ec99b01 P 3f742efcbd8e4dea9c62e531ccaf31d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.014843 24862 tablet_server.cc:196] TabletServer@127.24.71.132:0 shutdown complete.
I20260709 12:29:44.080035 24862 master.cc:562] Master@127.24.71.190:43079 shutting down...
I20260709 12:29:44.107692 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.108356 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.108706 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P a45175607d2c4b6c8f4c87e11ac28642: stopping tablet replica
I20260709 12:29:44.131645 24862 master.cc:584] Master@127.24.71.190:43079 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (31785 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:29:44.229174 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:33429
I20260709 12:29:44.230412 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:44.237962  6014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.238469  6017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.238667  6015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.239831 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:44.241205 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:44.241389 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:44.241587 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600184241569 us; error 0 us; skew 500 ppm
I20260709 12:29:44.242216 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:44.251463 24862 webserver.cc:533] Webserver started at http://127.24.71.190:36229/ using document root <none> and password file <none>
I20260709 12:29:44.252045 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:44.252274 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:44.252605 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:44.253822 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "8c8d2ad8e3a04631a3260fcb14a176f3"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.258652 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:44.262399  6022 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.263232 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:29:44.263553 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "8c8d2ad8e3a04631a3260fcb14a176f3"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.263859 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:44.290308 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:44.291636 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:44.336170 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:33429
I20260709 12:29:44.336383  6073 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:33429 every 8 connection(s)
I20260709 12:29:44.342645  6074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.360317  6074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3: Bootstrap starting.
I20260709 12:29:44.366906  6074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:44.372895  6074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3: No bootstrap required, opened a new log
I20260709 12:29:44.375667  6074 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" member_type: VOTER }
I20260709 12:29:44.376209  6074 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:44.376504  6074 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8d2ad8e3a04631a3260fcb14a176f3, State: Initialized, Role: FOLLOWER
I20260709 12:29:44.377234  6074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" member_type: VOTER }
I20260709 12:29:44.377938  6074 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:44.378214  6074 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:44.378506  6074 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:44.385622  6074 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" member_type: VOTER }
I20260709 12:29:44.386334  6074 leader_election.cc:304] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8d2ad8e3a04631a3260fcb14a176f3; no voters: 
I20260709 12:29:44.387966  6074 leader_election.cc:290] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:44.388547  6077 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:44.390188  6077 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8d2ad8e3a04631a3260fcb14a176f3, State: Running, Role: LEADER
I20260709 12:29:44.391238  6074 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:44.391150  6077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" member_type: VOTER }
I20260709 12:29:44.397498  6079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8d2ad8e3a04631a3260fcb14a176f3. Latest consensus state: current_term: 1 leader_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" member_type: VOTER } }
I20260709 12:29:44.398360  6079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:44.399776  6078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d2ad8e3a04631a3260fcb14a176f3" member_type: VOTER } }
I20260709 12:29:44.400552  6078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:44.400868  6083 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:44.406482  6083 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:44.411856 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:44.416829  6083 catalog_manager.cc:1383] Generated new cluster ID: 5057ebbedddb4185ba2a69216e2dbcc9
I20260709 12:29:44.417313  6083 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:44.436918  6083 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:44.438360  6083 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:44.452944  6083 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3: Generated new TSK 0
I20260709 12:29:44.453593  6083 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:44.482016 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:44.489151  6095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.490101  6096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.492498  6098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.492676 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:44.494002 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:44.494411 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:44.494590 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600184494571 us; error 0 us; skew 500 ppm
I20260709 12:29:44.495151 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:44.497865 24862 webserver.cc:533] Webserver started at http://127.24.71.129:37343/ using document root <none> and password file <none>
I20260709 12:29:44.498404 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:44.498615 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:44.498888 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:44.500017 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "3ed410022a0347998e98296e2ded5313"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.504971 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:44.508997  6103 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.509814 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:29:44.510250 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "3ed410022a0347998e98296e2ded5313"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.510561 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:44.520033 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:44.521349 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:44.522871 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:44.525360 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:44.525599 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.525859 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:44.526062 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.573215 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:46371
I20260709 12:29:44.573309  6165 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:46371 every 8 connection(s)
I20260709 12:29:44.578688 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:44.587733  6170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.588681  6171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.594075  6174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.594170 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:44.595227  6166 heartbeater.cc:344] Connected to a master server at 127.24.71.190:33429
I20260709 12:29:44.595626  6166 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:44.595871 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:44.596199 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:44.596477 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600184596458 us; error 0 us; skew 500 ppm
I20260709 12:29:44.596518  6166 heartbeater.cc:507] Master 127.24.71.190:33429 requested a full tablet report, sending...
I20260709 12:29:44.597275 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:44.599166  6039 ts_manager.cc:194] Registered new tserver with Master: 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371)
I20260709 12:29:44.600457 24862 webserver.cc:533] Webserver started at http://127.24.71.130:37675/ using document root <none> and password file <none>
I20260709 12:29:44.601198 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:44.601220  6039 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34130
I20260709 12:29:44.601496 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:44.601958 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:44.603387 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "bab20726a5e647358591df3262751012"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.608657 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:29:44.612728  6178 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.613806 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:44.614091 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "bab20726a5e647358591df3262751012"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.614362 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:44.627562 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:44.628871 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:44.630528 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:44.633116 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:44.633320 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.633503 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:44.633741 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.682145 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:38441
I20260709 12:29:44.682229  6240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:38441 every 8 connection(s)
I20260709 12:29:44.687806 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:44.696831  6244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.698211  6245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.701606 24862 server_base.cc:1061] running on GCE node
W20260709 12:29:44.702203  6247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.703301 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:44.703497 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:44.703644  6241 heartbeater.cc:344] Connected to a master server at 127.24.71.190:33429
I20260709 12:29:44.703716 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600184703696 us; error 0 us; skew 500 ppm
I20260709 12:29:44.704198  6241 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:44.704708 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:44.705135  6241 heartbeater.cc:507] Master 127.24.71.190:33429 requested a full tablet report, sending...
I20260709 12:29:44.707710  6039 ts_manager.cc:194] Registered new tserver with Master: bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:44.708017 24862 webserver.cc:533] Webserver started at http://127.24.71.131:43283/ using document root <none> and password file <none>
I20260709 12:29:44.708822 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:44.709118 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:44.709491 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:44.709658  6039 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34138
I20260709 12:29:44.711009 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "3f5da4a94b64457bad9c235b50ab9817"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.715840 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:29:44.719990  6252 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.721050 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:44.721524 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "3f5da4a94b64457bad9c235b50ab9817"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:44.721855 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:44.738554 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:44.739825 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:44.741673 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:44.744398 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:44.744637 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.744907 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:44.745083 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.792268 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:35819
I20260709 12:29:44.792464  6314 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:35819 every 8 connection(s)
I20260709 12:29:44.809147  6315 heartbeater.cc:344] Connected to a master server at 127.24.71.190:33429
I20260709 12:29:44.809566  6315 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:44.810336  6315 heartbeater.cc:507] Master 127.24.71.190:33429 requested a full tablet report, sending...
I20260709 12:29:44.812661  6039 ts_manager.cc:194] Registered new tserver with Master: 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:44.813297 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017361272s
I20260709 12:29:44.814139  6039 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34152
I20260709 12:29:44.840731  6039 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34164:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:44.843662  6039 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:44.941188  6131 tablet_service.cc:1511] Processing CreateTablet for tablet 7863463b701e46ee99e63b74fd272495 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:44.943118  6131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7863463b701e46ee99e63b74fd272495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.943792  6276 tablet_service.cc:1511] Processing CreateTablet for tablet ea1f12226fbc4fe1b4386433458ca641 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:44.944193  6127 tablet_service.cc:1511] Processing CreateTablet for tablet ea1f12226fbc4fe1b4386433458ca641 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:44.944964  6280 tablet_service.cc:1511] Processing CreateTablet for tablet 7863463b701e46ee99e63b74fd272495 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:44.945374  6276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1f12226fbc4fe1b4386433458ca641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.945839  6127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1f12226fbc4fe1b4386433458ca641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.945556  6130 tablet_service.cc:1511] Processing CreateTablet for tablet 16e097ef621f4c629caecf2dab3cf4fa (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:44.946319  6280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7863463b701e46ee99e63b74fd272495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.947127  6130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e097ef621f4c629caecf2dab3cf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.949481  6279 tablet_service.cc:1511] Processing CreateTablet for tablet 16e097ef621f4c629caecf2dab3cf4fa (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:44.949599  6129 tablet_service.cc:1511] Processing CreateTablet for tablet 77170d07974647869c088b9050034fdf (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:44.950933  6279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e097ef621f4c629caecf2dab3cf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.952334  6128 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa4bb002c2842b78da14a6de8d0da89 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:44.953516  6277 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa4bb002c2842b78da14a6de8d0da89 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:44.954972  6126 tablet_service.cc:1511] Processing CreateTablet for tablet abb57f67ef0f4a8f92d48b137fedfc22 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:44.955931  6278 tablet_service.cc:1511] Processing CreateTablet for tablet 77170d07974647869c088b9050034fdf (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:44.957564  6129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77170d07974647869c088b9050034fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.958290  6206 tablet_service.cc:1511] Processing CreateTablet for tablet 7863463b701e46ee99e63b74fd272495 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:44.959241  6278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77170d07974647869c088b9050034fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.959590  6206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7863463b701e46ee99e63b74fd272495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.960789  6277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa4bb002c2842b78da14a6de8d0da89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.960939  6128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa4bb002c2842b78da14a6de8d0da89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.964097  6126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb57f67ef0f4a8f92d48b137fedfc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.967633  6205 tablet_service.cc:1511] Processing CreateTablet for tablet 16e097ef621f4c629caecf2dab3cf4fa (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:44.969188  6205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e097ef621f4c629caecf2dab3cf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.973857  6204 tablet_service.cc:1511] Processing CreateTablet for tablet 77170d07974647869c088b9050034fdf (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:44.975445  6204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77170d07974647869c088b9050034fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.986076  6203 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa4bb002c2842b78da14a6de8d0da89 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:44.987624  6203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa4bb002c2842b78da14a6de8d0da89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.996629  6202 tablet_service.cc:1511] Processing CreateTablet for tablet ea1f12226fbc4fe1b4386433458ca641 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:44.998162  6202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1f12226fbc4fe1b4386433458ca641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:44.999604  6275 tablet_service.cc:1511] Processing CreateTablet for tablet abb57f67ef0f4a8f92d48b137fedfc22 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:45.002429  6125 tablet_service.cc:1511] Processing CreateTablet for tablet c2403e18c0044be299916191c9728356 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:45.003897  6275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb57f67ef0f4a8f92d48b137fedfc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.004088  6125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2403e18c0044be299916191c9728356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.017910  6201 tablet_service.cc:1511] Processing CreateTablet for tablet abb57f67ef0f4a8f92d48b137fedfc22 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:45.019488  6201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb57f67ef0f4a8f92d48b137fedfc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.029964  6200 tablet_service.cc:1511] Processing CreateTablet for tablet c2403e18c0044be299916191c9728356 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:45.031442  6200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2403e18c0044be299916191c9728356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.034479  6274 tablet_service.cc:1511] Processing CreateTablet for tablet c2403e18c0044be299916191c9728356 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:45.036065  6274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2403e18c0044be299916191c9728356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.058413  6124 tablet_service.cc:1511] Processing CreateTablet for tablet 0848ca0813aa4fb48fc7089056db5f83 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:45.059931  6124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0848ca0813aa4fb48fc7089056db5f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.079452  6331 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.104449  6330 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.107041  6199 tablet_service.cc:1511] Processing CreateTablet for tablet 0848ca0813aa4fb48fc7089056db5f83 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:45.108642  6199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0848ca0813aa4fb48fc7089056db5f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.113584  6330 tablet_bootstrap.cc:654] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.121918  6201 tablet_service.cc:1511] Processing CreateTablet for tablet 373cff2e13c443f8b363fe366dddd076 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:45.123267  6331 tablet_bootstrap.cc:654] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.123425  6201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373cff2e13c443f8b363fe366dddd076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.123857  6274 tablet_service.cc:1511] Processing CreateTablet for tablet 373cff2e13c443f8b363fe366dddd076 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:45.125193  6274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373cff2e13c443f8b363fe366dddd076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.111418  6275 tablet_service.cc:1511] Processing CreateTablet for tablet 0848ca0813aa4fb48fc7089056db5f83 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:45.130367  6275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0848ca0813aa4fb48fc7089056db5f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.133285  6332 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.117268  6129 tablet_service.cc:1511] Processing CreateTablet for tablet 373cff2e13c443f8b363fe366dddd076 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:45.138672  6129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373cff2e13c443f8b363fe366dddd076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.154623  6332 tablet_bootstrap.cc:654] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.155339  6330 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.155934  6330 ts_tablet_manager.cc:1403] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.052s	user 0.007s	sys 0.013s
I20260709 12:29:45.159402  6330 raft_consensus.cc:359] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.160318  6330 raft_consensus.cc:385] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.160714  6330 raft_consensus.cc:740] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.161545  6330 consensus_queue.cc:260] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.162174  6331 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.162760  6331 ts_tablet_manager.cc:1403] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.012s
I20260709 12:29:45.166074  6331 raft_consensus.cc:359] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.166846  6331 raft_consensus.cc:385] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.167153  6131 tablet_service.cc:1511] Processing CreateTablet for tablet 1c5fcd86214e4cefa4a243936b949014 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:45.169246  6131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5fcd86214e4cefa4a243936b949014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.167155  6331 raft_consensus.cc:740] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.170837  6331 consensus_queue.cc:260] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.175374  6201 tablet_service.cc:1511] Processing CreateTablet for tablet 1c5fcd86214e4cefa4a243936b949014 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:45.176949  6201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5fcd86214e4cefa4a243936b949014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.182071  6331 ts_tablet_manager.cc:1434] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:29:45.181877  6275 tablet_service.cc:1511] Processing CreateTablet for tablet 1c5fcd86214e4cefa4a243936b949014 (DEFAULT_TABLE table=test-workload [id=4aa5f69b80a849d7a7d03af57dae2bcc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:45.183334  6275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5fcd86214e4cefa4a243936b949014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:45.183620  6166 heartbeater.cc:499] Master 127.24.71.190:33429 was elected leader, sending a full tablet report...
I20260709 12:29:45.184614  6315 heartbeater.cc:499] Master 127.24.71.190:33429 was elected leader, sending a full tablet report...
I20260709 12:29:45.190696  6331 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.198875  6331 tablet_bootstrap.cc:654] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.206103  6330 ts_tablet_manager.cc:1434] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.050s	user 0.016s	sys 0.011s
I20260709 12:29:45.207360  6330 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.216087  6330 tablet_bootstrap.cc:654] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.216954  6332 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.217552  6332 ts_tablet_manager.cc:1403] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.014s
I20260709 12:29:45.221483  6332 raft_consensus.cc:359] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.222235  6332 raft_consensus.cc:385] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.222529  6332 raft_consensus.cc:740] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.222584  6331 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.222919  6330 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.223212  6331 ts_tablet_manager.cc:1403] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.008s
I20260709 12:29:45.223519  6330 ts_tablet_manager.cc:1403] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.002s
I20260709 12:29:45.223496  6332 consensus_queue.cc:260] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.226783  6330 raft_consensus.cc:359] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.227578  6330 raft_consensus.cc:385] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.227902  6330 raft_consensus.cc:740] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.227792  6331 raft_consensus.cc:359] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.228890  6331 raft_consensus.cc:385] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.229308  6331 raft_consensus.cc:740] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.228902  6330 consensus_queue.cc:260] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.230299  6331 consensus_queue.cc:260] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.231323  6330 ts_tablet_manager.cc:1434] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:29:45.232419  6330 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.232741  6331 ts_tablet_manager.cc:1434] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:29:45.233798  6331 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.239145  6330 tablet_bootstrap.cc:654] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.240203  6331 tablet_bootstrap.cc:654] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.252157  6331 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.252732  6330 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.252736  6331 ts_tablet_manager.cc:1403] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:29:45.253361  6330 ts_tablet_manager.cc:1403] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.002s
I20260709 12:29:45.256036  6331 raft_consensus.cc:359] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.256359  6330 raft_consensus.cc:359] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.256917  6331 raft_consensus.cc:385] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.257114  6330 raft_consensus.cc:385] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.257290  6331 raft_consensus.cc:740] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.257462  6330 raft_consensus.cc:740] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.258261  6330 consensus_queue.cc:260] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.258261  6331 consensus_queue.cc:260] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.260766  6330 ts_tablet_manager.cc:1434] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:45.261119  6331 ts_tablet_manager.cc:1434] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:45.261878  6330 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.262283  6331 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.269455  6330 tablet_bootstrap.cc:654] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.269455  6331 tablet_bootstrap.cc:654] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.284149  6330 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.284263  6331 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.284803  6330 ts_tablet_manager.cc:1403] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.023s	user 0.022s	sys 0.000s
I20260709 12:29:45.284919  6331 ts_tablet_manager.cc:1403] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 12:29:45.288076  6331 raft_consensus.cc:359] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.288322  6330 raft_consensus.cc:359] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.289116  6331 raft_consensus.cc:385] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.289146  6330 raft_consensus.cc:385] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.289759  6330 raft_consensus.cc:740] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.290772  6330 consensus_queue.cc:260] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.289752  6331 raft_consensus.cc:740] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.292989  6330 ts_tablet_manager.cc:1434] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:45.294862  6330 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
W20260709 12:29:45.296926  6316 tablet.cc:2406] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:45.293696  6331 consensus_queue.cc:260] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.300839  6241 heartbeater.cc:499] Master 127.24.71.190:33429 was elected leader, sending a full tablet report...
I20260709 12:29:45.301227  6332 ts_tablet_manager.cc:1434] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.083s	user 0.031s	sys 0.050s
I20260709 12:29:45.301975  6331 ts_tablet_manager.cc:1434] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:29:45.302301  6332 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.303042  6331 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.308935  6331 tablet_bootstrap.cc:654] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.309676  6332 tablet_bootstrap.cc:654] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.314584  6330 tablet_bootstrap.cc:654] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.321700  6331 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.322466  6331 ts_tablet_manager.cc:1403] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 12:29:45.327072  6332 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.327379  6330 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.327234  6331 raft_consensus.cc:359] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.327944  6332 ts_tablet_manager.cc:1403] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.010s
I20260709 12:29:45.328115  6330 ts_tablet_manager.cc:1403] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.000s
I20260709 12:29:45.328328  6331 raft_consensus.cc:385] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.330996  6331 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.331596  6332 raft_consensus.cc:359] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.332649  6330 raft_consensus.cc:359] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.332830  6331 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.333855  6332 raft_consensus.cc:385] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.334079  6330 raft_consensus.cc:385] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.334444  6332 raft_consensus.cc:740] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.334681  6330 raft_consensus.cc:740] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.335728  6330 consensus_queue.cc:260] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.335671  6332 consensus_queue.cc:260] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.337337  6331 ts_tablet_manager.cc:1434] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:29:45.337251  6340 raft_consensus.cc:493] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.338018  6340 raft_consensus.cc:515] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.338652  6331 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.342180  6330 ts_tablet_manager.cc:1434] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:29:45.343537  6330 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.343829  6340 leader_election.cc:290] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.344625  6336 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.345409  6336 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.347559  6331 tablet_bootstrap.cc:654] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.351050  6330 tablet_bootstrap.cc:654] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.360459  6336 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.373155  6336 raft_consensus.cc:493] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.379536  6336 raft_consensus.cc:515] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.387284  6336 leader_election.cc:290] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.399662  6331 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.400336  6331 ts_tablet_manager.cc:1403] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.000s
I20260709 12:29:45.400691  6332 ts_tablet_manager.cc:1434] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.072s	user 0.011s	sys 0.003s
I20260709 12:29:45.401868  6332 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.401863  6330 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.405472  6330 ts_tablet_manager.cc:1403] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.013s
I20260709 12:29:45.403767  6331 raft_consensus.cc:359] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.406315  6331 raft_consensus.cc:385] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.406677  6331 raft_consensus.cc:740] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
W20260709 12:29:45.406715  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:45.408824  6332 tablet_bootstrap.cc:654] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.415632  6331 consensus_queue.cc:260] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.413275  6330 raft_consensus.cc:359] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.420674  6340 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.421276  6330 raft_consensus.cc:385] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.421399  6340 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.421720  6330 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.423067  6330 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.421720  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.424633  6340 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.425527  6330 ts_tablet_manager.cc:1434] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.004s
I20260709 12:29:45.425933  6331 ts_tablet_manager.cc:1434] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:29:45.427428  6331 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.429536  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77170d07974647869c088b9050034fdf" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.430298  6330 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.430387  6140 raft_consensus.cc:2468] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 0.
I20260709 12:29:45.431838  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.432797  6290 raft_consensus.cc:2468] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 0.
I20260709 12:29:45.433000  6215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77170d07974647869c088b9050034fdf" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.434653  6141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.431959  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.436141  6216 raft_consensus.cc:2468] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 0.
W20260709 12:29:45.437999  6181 leader_election.cc:343] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.438421  6255 leader_election.cc:304] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
W20260709 12:29:45.439682  6255 leader_election.cc:343] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.440305  6353 raft_consensus.cc:2804] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.440829  6141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.448648  6353 raft_consensus.cc:493] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.448865  6330 tablet_bootstrap.cc:654] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:45.446048  6253 leader_election.cc:343] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.449055  6141 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 0.
I20260709 12:29:45.450510  6253 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
I20260709 12:29:45.451545  6336 raft_consensus.cc:2804] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.452067  6336 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.452556  6336 raft_consensus.cc:3060] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.441394  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.442500  6335 raft_consensus.cc:493] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.455555  6335 raft_consensus.cc:515] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.446358  6181 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:45.458127  6181 leader_election.cc:304] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:45.459290  6340 raft_consensus.cc:2804] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.459705  6340 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.460176  6340 raft_consensus.cc:3060] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.463522  6335 leader_election.cc:290] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.464408  6335 raft_consensus.cc:493] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.463550  6359 raft_consensus.cc:2804] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.465233  6359 raft_consensus.cc:493] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.465062  6335 raft_consensus.cc:515] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.467473  6335 leader_election.cc:290] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.465613  6359 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.449115  6353 raft_consensus.cc:3060] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:45.473294  6181 leader_election.cc:343] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.440938  6332 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.477344  6332 ts_tablet_manager.cc:1403] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.006s
I20260709 12:29:45.447995  6331 tablet_bootstrap.cc:654] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.480024  6336 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.491578  6357 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.492460  6357 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.494133  6340 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.495637  6357 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.498353  6362 raft_consensus.cc:493] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.499027  6336 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.498914  6362 raft_consensus.cc:515] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.500423  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.501305  6216 raft_consensus.cc:2393] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f5da4a94b64457bad9c235b50ab9817 in current term 1: Already voted for candidate bab20726a5e647358591df3262751012 in this term.
I20260709 12:29:45.502017  6340 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.502676  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
W20260709 12:29:45.504055  6181 leader_election.cc:343] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.504974  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.505877  6290 raft_consensus.cc:2393] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bab20726a5e647358591df3262751012 in current term 1: Already voted for candidate 3f5da4a94b64457bad9c235b50ab9817 in this term.
I20260709 12:29:45.506966  6181 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:45.507155  6332 raft_consensus.cc:359] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.507939  6332 raft_consensus.cc:385] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.508303  6332 raft_consensus.cc:740] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.508487  6141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.509253  6141 raft_consensus.cc:2468] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 0.
I20260709 12:29:45.510751  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:45.510361  6332 consensus_queue.cc:260] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.512147  6330 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.512851  6255 leader_election.cc:304] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:45.515132  6330 ts_tablet_manager.cc:1403] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.085s	user 0.035s	sys 0.027s
W20260709 12:29:45.515468  6255 leader_election.cc:343] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.515990  6255 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817; no voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012
I20260709 12:29:45.517714  6336 raft_consensus.cc:2804] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.518136  6336 raft_consensus.cc:493] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.517807  6353 raft_consensus.cc:515] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.518539  6336 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.518982  6330 raft_consensus.cc:359] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.519796  6330 raft_consensus.cc:385] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.520098  6330 raft_consensus.cc:740] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.520615  6353 leader_election.cc:290] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.521327  6353 raft_consensus.cc:2749] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.521184  6330 consensus_queue.cc:260] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.523972  6330 ts_tablet_manager.cc:1434] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:29:45.513381  6359 raft_consensus.cc:515] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.524590  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.525102  6330 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.525468  6216 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.526052  6359 leader_election.cc:290] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.526829  6362 leader_election.cc:290] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.527501  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:45.528429  6140 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.514407  6340 raft_consensus.cc:2749] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.530025  6332 ts_tablet_manager.cc:1434] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:29:45.531806  6331 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.531700  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.532406  6331 ts_tablet_manager.cc:1403] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.003s
I20260709 12:29:45.533406  6330 tablet_bootstrap.cc:654] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.534231  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.535702  6331 raft_consensus.cc:359] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.528453  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.536732  6331 raft_consensus.cc:385] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.537096  6331 raft_consensus.cc:740] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.537120  6290 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.537997  6104 leader_election.cc:304] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:45.538146  6331 consensus_queue.cc:260] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.539341  6357 raft_consensus.cc:2804] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:29:45.539520  6104 leader_election.cc:343] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.539852  6357 raft_consensus.cc:493] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.540436  6357 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:45.540637  6104 leader_election.cc:343] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.544696  6141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77170d07974647869c088b9050034fdf" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:45.544742  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.545320  6141 raft_consensus.cc:3060] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.546861  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.548085  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77170d07974647869c088b9050034fdf" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.549875  6290 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.553179  6332 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.555720  6331 ts_tablet_manager.cc:1434] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:29:45.558560  6106 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
I20260709 12:29:45.559957  6335 raft_consensus.cc:2804] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.560693  6335 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.560814  6331 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.561153  6335 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.563664  6141 raft_consensus.cc:2468] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:45.565583  6140 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 1.
W20260709 12:29:45.565699  6253 leader_election.cc:343] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.567209  6181 leader_election.cc:304] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:45.568323  6359 raft_consensus.cc:2804] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.569312  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.569837  6359 raft_consensus.cc:697] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:45.570909  6359 consensus_queue.cc:237] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.572793  6336 raft_consensus.cc:515] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.575814  6289 raft_consensus.cc:2393] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ed410022a0347998e98296e2ded5313 in current term 1: Already voted for candidate 3f5da4a94b64457bad9c235b50ab9817 in this term.
I20260709 12:29:45.576807  6330 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.587184  6330 ts_tablet_manager.cc:1403] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.012s
I20260709 12:29:45.580192  6357 raft_consensus.cc:515] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.587862  6106 leader_election.cc:304] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ed410022a0347998e98296e2ded5313; no voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012
I20260709 12:29:45.589599  6357 leader_election.cc:290] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Requested vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.590204  6357 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.590240  6330 raft_consensus.cc:359] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.591022  6330 raft_consensus.cc:385] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.591305  6330 raft_consensus.cc:740] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.591972  6330 consensus_queue.cc:260] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.594201  6330 ts_tablet_manager.cc:1434] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:45.595207  6330 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.595456  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.583020  6331 tablet_bootstrap.cc:654] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.596511  6216 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.579507  6255 leader_election.cc:304] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
I20260709 12:29:45.601138  6330 tablet_bootstrap.cc:654] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.601089  6364 raft_consensus.cc:2804] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.601732  6364 raft_consensus.cc:697] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:45.602963  6364 consensus_queue.cc:237] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.579906  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.608677  6290 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.609956  6215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
W20260709 12:29:45.612875  6253 leader_election.cc:343] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:45.617416  6253 leader_election.cc:343] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.580997  6336 leader_election.cc:290] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.622426  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:45.633558  6332 tablet_bootstrap.cc:654] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.634913  6331 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.635672  6331 ts_tablet_manager.cc:1403] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.075s	user 0.008s	sys 0.011s
I20260709 12:29:45.635785  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.636662  6289 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.642390  6367 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.643102  6367 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.647478  6357 raft_consensus.cc:2749] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.647559  6215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.651917  6216 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
W20260709 12:29:45.651221  6104 leader_election.cc:343] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.654023  6367 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.654171  6104 leader_election.cc:304] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:45.659921  6331 raft_consensus.cc:359] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.661042  6331 raft_consensus.cc:385] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.661397  6331 raft_consensus.cc:740] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.662878  6140 raft_consensus.cc:2468] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:45.665267  6255 leader_election.cc:304] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
I20260709 12:29:45.667207  6364 raft_consensus.cc:2804] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.667883  6364 raft_consensus.cc:697] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:45.669035  6364 consensus_queue.cc:237] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.671903  6039 catalog_manager.cc:5697] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 0 to 1, leader changed from <none> to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:45.673568  6290 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 1.
I20260709 12:29:45.675168  6288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.675788  6331 consensus_queue.cc:260] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.634433  6335 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.689683  6288 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.689826  6362 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.690352  6362 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.677286  6357 raft_consensus.cc:2804] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.691310  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.692940  6330 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
W20260709 12:29:45.692986  6104 leader_election.cc:343] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.693509  6362 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.693744  6330 ts_tablet_manager.cc:1403] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.099s	user 0.049s	sys 0.044s
I20260709 12:29:45.691835  6357 raft_consensus.cc:697] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Running, Role: LEADER
I20260709 12:29:45.695135  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.695132  6357 consensus_queue.cc:237] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.695478  6106 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
W20260709 12:29:45.696712  6255 leader_election.cc:343] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.697067  6367 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.697597  6367 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.697314  6330 raft_consensus.cc:359] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.698011  6367 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.698112  6330 raft_consensus.cc:385] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.698487  6330 raft_consensus.cc:740] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.699219  6331 ts_tablet_manager.cc:1434] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 12:29:45.699241  6330 consensus_queue.cc:260] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.700784  6331 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.701406  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.702776  6290 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.680867  6289 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
I20260709 12:29:45.702140  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.701879  6330 ts_tablet_manager.cc:1434] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:45.705700  6335 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Requested vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
W20260709 12:29:45.708444  6253 leader_election.cc:343] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.709640  6253 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817; no voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012
I20260709 12:29:45.710551  6364 raft_consensus.cc:2749] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.711529  6331 tablet_bootstrap.cc:654] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.709074  6332 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.713539  6332 ts_tablet_manager.cc:1403] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.161s	user 0.024s	sys 0.003s
I20260709 12:29:45.718185  6367 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.722345  6367 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Requested vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.726835  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.727523  6289 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.728613  6330 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:45.722419  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
W20260709 12:29:45.734282  6104 leader_election.cc:343] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.735745  6290 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
I20260709 12:29:45.736637  6331 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.737172  6331 ts_tablet_manager.cc:1403] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.002s
I20260709 12:29:45.737231  6106 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
I20260709 12:29:45.738701  6367 raft_consensus.cc:2804] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.739274  6367 raft_consensus.cc:697] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Running, Role: LEADER
I20260709 12:29:45.738883  6038 catalog_manager.cc:5697] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed410022a0347998e98296e2ded5313 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "3ed410022a0347998e98296e2ded5313" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:45.740128  6367 consensus_queue.cc:237] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.740330  6331 raft_consensus.cc:359] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.741093  6331 raft_consensus.cc:385] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.741391  6331 raft_consensus.cc:740] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.742127  6331 consensus_queue.cc:260] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.743140  6289 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
I20260709 12:29:45.745057  6331 ts_tablet_manager.cc:1434] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:45.748035  6331 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817: Bootstrap starting.
I20260709 12:29:45.748116  6106 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
I20260709 12:29:45.749276  6367 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.749821  6367 raft_consensus.cc:697] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Running, Role: LEADER
I20260709 12:29:45.742242  6332 raft_consensus.cc:359] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.750443  6332 raft_consensus.cc:385] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.750802  6332 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.750888  6367 consensus_queue.cc:237] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.751920  6330 tablet_bootstrap.cc:654] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.751857  6332 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.755872  6332 ts_tablet_manager.cc:1434] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:29:45.756361  6331 tablet_bootstrap.cc:654] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.757947  6332 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.759443  6364 raft_consensus.cc:493] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.760003  6364 raft_consensus.cc:515] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.762729  6037 catalog_manager.cc:5697] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.765255  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.766435  6140 raft_consensus.cc:2468] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 0.
I20260709 12:29:45.767304  6364 leader_election.cc:290] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.768741  6255 leader_election.cc:304] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:45.768828  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
W20260709 12:29:45.772567  6253 leader_election.cc:343] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.776103  6364 raft_consensus.cc:2804] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.776916  6364 raft_consensus.cc:493] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.777441  6364 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.783347  6332 tablet_bootstrap.cc:654] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.790432  6331 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817: No bootstrap required, opened a new log
I20260709 12:29:45.791093  6331 ts_tablet_manager.cc:1403] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.022s
I20260709 12:29:45.794369  6331 raft_consensus.cc:359] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.795238  6331 raft_consensus.cc:385] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.795101  6364 raft_consensus.cc:515] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.796025  6331 raft_consensus.cc:740] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.797302  6331 consensus_queue.cc:260] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.799168  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:45.799918  6140 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.800175  6331 ts_tablet_manager.cc:1434] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:29:45.803795  6038 catalog_manager.cc:5697] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed410022a0347998e98296e2ded5313 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "3ed410022a0347998e98296e2ded5313" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:45.806169  6038 catalog_manager.cc:5697] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed410022a0347998e98296e2ded5313 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "3ed410022a0347998e98296e2ded5313" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:45.814507  6332 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.815142  6332 ts_tablet_manager.cc:1403] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.009s
I20260709 12:29:45.816560  6140 raft_consensus.cc:2468] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:45.818326  6364 leader_election.cc:290] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.818277  6255 leader_election.cc:304] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:45.818738  6332 raft_consensus.cc:359] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.819288  6364 raft_consensus.cc:2804] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.820046  6332 raft_consensus.cc:385] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.820549  6332 raft_consensus.cc:740] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.820896  6364 raft_consensus.cc:697] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:45.821475  6332 consensus_queue.cc:260] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.821911  6364 consensus_queue.cc:237] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.819731  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.824150  6332 ts_tablet_manager.cc:1434] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
W20260709 12:29:45.824556  6253 leader_election.cc:343] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.825357  6332 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.828415  6037 catalog_manager.cc:5697] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.832609  6332 tablet_bootstrap.cc:654] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.834198  6330 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: No bootstrap required, opened a new log
I20260709 12:29:45.834807  6330 ts_tablet_manager.cc:1403] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.109s	user 0.015s	sys 0.029s
I20260709 12:29:45.838514  6330 raft_consensus.cc:359] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.839553  6330 raft_consensus.cc:385] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.839932  6330 raft_consensus.cc:740] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.840744  6330 consensus_queue.cc:260] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.842299  6332 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.842741  6330 ts_tablet_manager.cc:1434] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:45.842936  6332 ts_tablet_manager.cc:1403] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.001s
I20260709 12:29:45.846325  6332 raft_consensus.cc:359] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.847121  6332 raft_consensus.cc:385] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.847440  6332 raft_consensus.cc:740] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.848183  6332 consensus_queue.cc:260] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.850752  6332 ts_tablet_manager.cc:1434] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:45.851692  6332 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.852680  6367 raft_consensus.cc:493] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.853272  6367 raft_consensus.cc:515] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.854933  6037 catalog_manager.cc:5697] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.858721  6332 tablet_bootstrap.cc:654] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.859799  6367 leader_election.cc:290] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.860392  6357 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.860901  6357 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:45.859699  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.859701  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.864060  6289 raft_consensus.cc:2468] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.866225  6357 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:45.865732  6106 leader_election.cc:304] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
W20260709 12:29:45.868530  6104 leader_election.cc:343] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.869246  6332 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.869771  6332 ts_tablet_manager.cc:1403] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.004s
I20260709 12:29:45.869666  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.870684  6289 raft_consensus.cc:2393] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ed410022a0347998e98296e2ded5313 in current term 1: Already voted for candidate 3f5da4a94b64457bad9c235b50ab9817 in this term.
I20260709 12:29:45.871371  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.872210  6216 raft_consensus.cc:2393] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ed410022a0347998e98296e2ded5313 in current term 1: Already voted for candidate bab20726a5e647358591df3262751012 in this term.
I20260709 12:29:45.872809  6332 raft_consensus.cc:359] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.873299  6104 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ed410022a0347998e98296e2ded5313; no voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012
I20260709 12:29:45.873618  6332 raft_consensus.cc:385] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.873930  6332 raft_consensus.cc:740] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.874197  6367 raft_consensus.cc:2804] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.874585  6357 raft_consensus.cc:3060] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.874812  6367 raft_consensus.cc:493] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.874745  6332 consensus_queue.cc:260] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.875392  6367 raft_consensus.cc:3060] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.880100  6332 ts_tablet_manager.cc:1434] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:29:45.881552  6332 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.882275  6367 raft_consensus.cc:515] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.883867  6357 raft_consensus.cc:2749] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.885681  6359 raft_consensus.cc:493] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.885778  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.886397  6359 raft_consensus.cc:515] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.887315  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.887976  6289 raft_consensus.cc:3060] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:45.888355  6104 leader_election.cc:343] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:45.888638  6359 leader_election.cc:290] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.890431  6332 tablet_bootstrap.cc:654] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.893368  6367 leader_election.cc:290] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Requested vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.893463  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.892151  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.895377  6140 raft_consensus.cc:2393] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bab20726a5e647358591df3262751012 in current term 1: Already voted for candidate 3f5da4a94b64457bad9c235b50ab9817 in this term.
I20260709 12:29:45.896474  6181 leader_election.cc:304] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:45.897328  6359 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.897480  6289 raft_consensus.cc:2468] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
I20260709 12:29:45.898756  6106 leader_election.cc:304] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: bab20726a5e647358591df3262751012
I20260709 12:29:45.900547  6367 raft_consensus.cc:2804] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.901106  6367 raft_consensus.cc:697] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Running, Role: LEADER
I20260709 12:29:45.902053  6367 consensus_queue.cc:237] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.906035  6332 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.905978  6359 raft_consensus.cc:2749] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.906978  6332 ts_tablet_manager.cc:1403] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.004s
I20260709 12:29:45.909687  6332 raft_consensus.cc:359] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.910356  6332 raft_consensus.cc:385] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.910646  6332 raft_consensus.cc:740] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.911309  6332 consensus_queue.cc:260] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.911832  6038 catalog_manager.cc:5697] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed410022a0347998e98296e2ded5313 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "3ed410022a0347998e98296e2ded5313" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:45.913473  6332 ts_tablet_manager.cc:1434] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:29:45.914454  6332 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.919636  6332 tablet_bootstrap.cc:654] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.924278  6332 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.924889  6332 ts_tablet_manager.cc:1403] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:29:45.927656  6332 raft_consensus.cc:359] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.928293  6332 raft_consensus.cc:385] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.928545  6332 raft_consensus.cc:740] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.929000  6332 consensus_queue.cc:260] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.930714  6332 ts_tablet_manager.cc:1434] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:45.931902  6332 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012: Bootstrap starting.
I20260709 12:29:45.932797  6359 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.933326  6359 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.935675  6359 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.936259  6367 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.936765  6332 tablet_bootstrap.cc:654] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:45.936869  6367 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.938128  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.939529  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.940038  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:45.940376  6289 raft_consensus.cc:2393] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bab20726a5e647358591df3262751012 in current term 1: Already voted for candidate 3ed410022a0347998e98296e2ded5313 in this term.
I20260709 12:29:45.940723  6216 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.942027  6104 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:45.942066  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.943116  6289 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 0.
I20260709 12:29:45.944273  6357 raft_consensus.cc:2804] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:45.942042  6181 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:45.946004  6359 raft_consensus.cc:493] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:45.946244  6340 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.946486  6359 raft_consensus.cc:515] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.949502  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:45.950275  6140 raft_consensus.cc:2393] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bab20726a5e647358591df3262751012 in current term 1: Already voted for candidate 3f5da4a94b64457bad9c235b50ab9817 in this term.
I20260709 12:29:45.944826  6357 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:45.951575  6357 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.952117  6359 leader_election.cc:290] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.954424  6340 raft_consensus.cc:2749] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.956602  6367 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.957464  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:45.957677  6332 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012: No bootstrap required, opened a new log
I20260709 12:29:45.958295  6332 ts_tablet_manager.cc:1403] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.012s
I20260709 12:29:45.958946  6181 leader_election.cc:304] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:45.959810  6340 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.960084  6357 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.960929  6332 raft_consensus.cc:359] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.961476  6332 raft_consensus.cc:385] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:45.961700  6332 raft_consensus.cc:740] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.962316  6332 consensus_queue.cc:260] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.962733  6357 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Requested vote from peers bab20726a5e647358591df3262751012 (127.24.71.130:38441), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:45.963271  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:45.963869  6216 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.964128  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3ed410022a0347998e98296e2ded5313" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:45.964744  6332 ts_tablet_manager.cc:1434] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:45.964763  6289 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:45.967760  6340 raft_consensus.cc:2749] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:45.970780  6216 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
I20260709 12:29:45.971931  6104 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:45.972787  6357 raft_consensus.cc:2804] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:45.973225  6357 raft_consensus.cc:697] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Running, Role: LEADER
I20260709 12:29:45.973645  6289 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed410022a0347998e98296e2ded5313 in term 1.
I20260709 12:29:45.974102  6357 consensus_queue.cc:237] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:45.982643  6037 catalog_manager.cc:5697] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed410022a0347998e98296e2ded5313 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "3ed410022a0347998e98296e2ded5313" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:46.003187  6364 consensus_queue.cc:1048] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.014617  6216 raft_consensus.cc:3060] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:46.023772  6362 consensus_queue.cc:1048] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.065990  6340 consensus_queue.cc:1048] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.078402  6340 consensus_queue.cc:1048] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.195741  6357 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.208935  6357 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.229244  6362 consensus_queue.cc:1048] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.232399  6340 raft_consensus.cc:493] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:46.233059  6340 raft_consensus.cc:515] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:46.233800  6336 consensus_queue.cc:1048] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:46.235463  6340 leader_election.cc:290] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:46.237564  6141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:46.237751  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:46.238579  6289 raft_consensus.cc:2393] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bab20726a5e647358591df3262751012 in current term 1: Already voted for candidate 3ed410022a0347998e98296e2ded5313 in this term.
I20260709 12:29:46.239593  6181 leader_election.cc:304] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:46.240372  6215 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:46.242285  6340 raft_consensus.cc:3060] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:46.248615  6336 consensus_queue.cc:1048] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:46.259497  6340 raft_consensus.cc:2749] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:46.262348  6357 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:46.268885  6364 consensus_queue.cc:1048] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:46.281912  6357 consensus_queue.cc:1048] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.291857  6357 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.313941  6357 consensus_queue.cc:1048] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:46.314795  6367 consensus_queue.cc:1048] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:46.333683  6335 consensus_queue.cc:1048] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:46.408597  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:46.450976  6335 consensus_queue.cc:1048] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:46.463124  6335 consensus_queue.cc:1048] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:47.068430  6362 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:47.068933  6362 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:47.070797  6362 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:47.071647  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313" is_pre_election: true
I20260709 12:29:47.071933  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:47.072424  6140 raft_consensus.cc:2468] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:47.072569  6216 raft_consensus.cc:2468] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:47.073742  6253 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:47.074476  6362 raft_consensus.cc:2804] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:47.074926  6362 raft_consensus.cc:493] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:47.075367  6362 raft_consensus.cc:3060] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.081060  6362 raft_consensus.cc:515] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:47.082831  6362 leader_election.cc:290] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:47.083738  6140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:47.083849  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7863463b701e46ee99e63b74fd272495" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:47.084393  6140 raft_consensus.cc:3060] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.084455  6216 raft_consensus.cc:3060] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.090025  6140 raft_consensus.cc:2468] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:47.090140  6216 raft_consensus.cc:2468] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:47.091251  6255 leader_election.cc:304] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:47.092027  6362 raft_consensus.cc:2804] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:47.092585  6362 raft_consensus.cc:697] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:47.093468  6362 consensus_queue.cc:237] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:47.101532  6039 catalog_manager.cc:5697] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 0 to 2, leader changed from <none> to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:47.140437 24862 ts_manager.cc:295] Set tserver state for 3ed410022a0347998e98296e2ded5313 to MAINTENANCE_MODE
I20260709 12:29:47.141304 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
I20260709 12:29:47.164801 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:47.165894 24862 tablet_replica.cc:333] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.166826 24862 raft_consensus.cc:2243] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.167647 24862 raft_consensus.cc:2272] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.170639 24862 tablet_replica.cc:333] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.171283 24862 raft_consensus.cc:2243] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.172133 24862 raft_consensus.cc:2272] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.174600 24862 tablet_replica.cc:333] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.175211 24862 raft_consensus.cc:2243] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.175623 24862 raft_consensus.cc:2272] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.177538 24862 tablet_replica.cc:333] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.178095 24862 raft_consensus.cc:2243] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.178516 24862 raft_consensus.cc:2272] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.180452 24862 tablet_replica.cc:333] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.180989 24862 raft_consensus.cc:2243] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.181407 24862 raft_consensus.cc:2272] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.184012 24862 tablet_replica.cc:333] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.184837 24862 raft_consensus.cc:2243] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.185274 24862 raft_consensus.cc:2272] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.187740 24862 tablet_replica.cc:333] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
W20260709 12:29:47.187736  6181 consensus_peers.cc:597] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:47.188638 24862 raft_consensus.cc:2243] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.189560 24862 raft_consensus.cc:2272] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.192032 24862 tablet_replica.cc:333] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.192674 24862 raft_consensus.cc:2243] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.193519 24862 raft_consensus.cc:2272] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.195791 24862 tablet_replica.cc:333] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.196388 24862 raft_consensus.cc:2243] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.196866 24862 raft_consensus.cc:2272] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.198674 24862 tablet_replica.cc:333] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: stopping tablet replica
I20260709 12:29:47.199147 24862 raft_consensus.cc:2243] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.199934 24862 raft_consensus.cc:2272] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.230999 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
W20260709 12:29:47.242250  6255 consensus_peers.cc:597] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:47.409797  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:47.413601  6255 consensus_peers.cc:597] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:47.499658  6216 raft_consensus.cc:1275] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:47.501374  6362 consensus_queue.cc:1048] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:47.516426  6255 consensus_peers.cc:597] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:47.523770  6255 consensus_peers.cc:597] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:47.641984  6255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111) [suppressed 105 similar messages]
I20260709 12:29:48.194936  6430 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.195681  6430 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.197829  6430 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.199402  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
W20260709 12:29:48.201882  6181 leader_election.cc:336] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.202365  6181 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:48.203104  6430 raft_consensus.cc:2749] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:48.232698  6430 raft_consensus.cc:493] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.233254  6430 raft_consensus.cc:515] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.234972  6430 leader_election.cc:290] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.237123  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
W20260709 12:29:48.239310  6181 leader_election.cc:336] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.239682  6181 leader_election.cc:304] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:48.240355  6430 raft_consensus.cc:2749] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:48.280588  6430 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.281169  6430 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.283138  6430 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.284989  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
W20260709 12:29:48.287158  6181 leader_election.cc:336] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.287529  6181 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:48.288143  6430 raft_consensus.cc:2749] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:48.315606  6428 raft_consensus.cc:493] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.316223  6428 raft_consensus.cc:515] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.318238  6428 leader_election.cc:290] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.319530  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
W20260709 12:29:48.322470  6255 leader_election.cc:336] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.322875  6255 leader_election.cc:304] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817; no voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012
I20260709 12:29:48.323582  6428 raft_consensus.cc:2749] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:48.399449  6428 raft_consensus.cc:493] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.399919  6428 raft_consensus.cc:515] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.402055  6428 leader_election.cc:290] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.403251  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:48.404011  6216 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:48.405041  6253 leader_election.cc:304] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:48.405805  6428 raft_consensus.cc:2804] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:48.405886  6255 leader_election.cc:336] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.406150  6428 raft_consensus.cc:493] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.406526  6428 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:48.410987  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:48.412175  6428 raft_consensus.cc:515] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.413097  6430 raft_consensus.cc:493] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.413628  6430 raft_consensus.cc:515] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.414266  6428 leader_election.cc:290] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.415488  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:48.416193  6216 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.417791  6430 leader_election.cc:290] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.419118  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:48.419840  6289 raft_consensus.cc:2468] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 1.
W20260709 12:29:48.420922  6181 leader_election.cc:336] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.421936  6181 leader_election.cc:304] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
W20260709 12:29:48.422132  6255 leader_election.cc:336] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.422724  6430 raft_consensus.cc:2804] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:48.423094  6430 raft_consensus.cc:493] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.423413  6430 raft_consensus.cc:3060] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.424221  6216 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:48.425246  6253 leader_election.cc:304] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
I20260709 12:29:48.425968  6428 raft_consensus.cc:2804] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.426442  6428 raft_consensus.cc:697] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:48.427309  6428 consensus_queue.cc:237] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.431001  6430 raft_consensus.cc:515] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.433071  6430 leader_election.cc:290] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.434584  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c5fcd86214e4cefa4a243936b949014" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:48.435286  6289 raft_consensus.cc:3060] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:48.437161  6181 leader_election.cc:336] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.437054  6038 catalog_manager.cc:5697] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 1 to 2, leader changed from 3ed410022a0347998e98296e2ded5313 (127.24.71.129) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.443825  6289 raft_consensus.cc:2468] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:48.445463  6181 leader_election.cc:304] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
I20260709 12:29:48.446491  6430 raft_consensus.cc:2804] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.449268  6430 raft_consensus.cc:697] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:48.450089  6430 consensus_queue.cc:237] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.453186  6428 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.453907  6428 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.456043  6428 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.457612  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
W20260709 12:29:48.460381  6255 leader_election.cc:336] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.460354  6038 catalog_manager.cc:5697] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 1 to 2, leader changed from 3ed410022a0347998e98296e2ded5313 (127.24.71.129) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:48.461021  6255 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817; no voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012
I20260709 12:29:48.461812  6428 raft_consensus.cc:2749] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:48.497062  6430 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.497718  6430 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.499513  6430 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.501235  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817" is_pre_election: true
I20260709 12:29:48.501883  6289 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 1.
I20260709 12:29:48.503084  6181 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:48.503890  6430 raft_consensus.cc:2804] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:48.503955  6181 leader_election.cc:336] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.504340  6430 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.504752  6430 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.510214  6430 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.512058  6430 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:48.513733  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:48.514288  6289 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:48.516016  6181 leader_election.cc:336] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.520303  6289 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:48.521409  6181 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
I20260709 12:29:48.522168  6430 raft_consensus.cc:2804] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.522651  6430 raft_consensus.cc:697] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:48.523360  6430 consensus_queue.cc:237] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.531020  6038 catalog_manager.cc:5697] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 1 to 2, leader changed from 3ed410022a0347998e98296e2ded5313 (127.24.71.129) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:48.622128  6362 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.622649  6362 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.624478  6362 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.625728  6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:48.626413  6214 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:48.627351  6253 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:48.628075  6362 raft_consensus.cc:2804] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:48.628424  6255 leader_election.cc:336] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.628438  6362 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.629067  6362 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.634383  6362 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.636062  6362 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.637305  6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:48.637986  6214 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:48.640295  6255 leader_election.cc:336] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.644683  6214 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:48.645797  6253 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
I20260709 12:29:48.646587  6362 raft_consensus.cc:2804] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.646991  6362 raft_consensus.cc:697] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:48.647714  6362 consensus_queue.cc:237] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.656194  6038 catalog_manager.cc:5697] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 1 to 2, leader changed from 3ed410022a0347998e98296e2ded5313 (127.24.71.129) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.665848  6362 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.666461  6362 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.668162  6362 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.669529  6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012" is_pre_election: true
I20260709 12:29:48.670384  6214 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 1.
I20260709 12:29:48.671509  6253 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
W20260709 12:29:48.672314  6255 leader_election.cc:336] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.672312  6362 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:48.672840  6362 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ed410022a0347998e98296e2ded5313)
I20260709 12:29:48.673138  6362 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.678460  6362 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.680068  6362 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:48.681178  6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:48.681741  6214 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:48.683822  6255 leader_election.cc:336] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111)
I20260709 12:29:48.686839  6214 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:48.687938  6253 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 3ed410022a0347998e98296e2ded5313
I20260709 12:29:48.689018  6362 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.689540  6362 raft_consensus.cc:697] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:48.690237  6362 consensus_queue.cc:237] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:48.697036  6038 catalog_manager.cc:5697] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 1 to 2, leader changed from 3ed410022a0347998e98296e2ded5313 (127.24.71.129) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.858896  6214 raft_consensus.cc:1275] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.860726  6362 consensus_queue.cc:1048] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:48.876590  6255 consensus_peers.cc:597] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:48.949327  6181 consensus_peers.cc:597] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:48.998610  6289 raft_consensus.cc:1275] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.999956  6430 consensus_queue.cc:1048] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:49.113729  6181 consensus_peers.cc:597] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:49.120904  6214 raft_consensus.cc:1275] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:49.122445  6445 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:49.125650  6289 raft_consensus.cc:1275] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:49.126889  6435 consensus_queue.cc:1048] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:49.138002  6255 consensus_peers.cc:597] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:49.200950  6255 consensus_peers.cc:597] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:49.315583  6214 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:49.317591  6362 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:49.391132  6255 consensus_peers.cc:597] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:49.412074  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:49.699961  6181 consensus_peers.cc:597] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:49.706305  6255 consensus_peers.cc:597] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:49.780292  6255 consensus_peers.cc:597] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:49.926578  6255 consensus_peers.cc:597] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:50.413594  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:51.275926  6181 consensus_peers.cc:597] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:51.414655  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:51.439018  6181 consensus_peers.cc:597] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:51.600854  6255 consensus_peers.cc:597] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:51.622455  6255 consensus_peers.cc:597] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:51.831728  6255 consensus_peers.cc:597] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:52.118337  6255 consensus_peers.cc:597] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:52.130307  6255 consensus_peers.cc:597] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:52.224036  6255 consensus_peers.cc:597] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:52.252432 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:52.260785  6457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:52.261348  6458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:52.262773  6460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:52.262791 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:52.265494 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:52.265846 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:52.266045 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600192266028 us; error 0 us; skew 500 ppm
I20260709 12:29:52.266713 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:52.269769 24862 webserver.cc:533] Webserver started at http://127.24.71.129:37343/ using document root <none> and password file <none>
I20260709 12:29:52.270282 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:52.270440 24862 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:29:52.274753 24862 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:52.279549  6465 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:52.280668 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 12:29:52.281028 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "3ed410022a0347998e98296e2ded5313"
format_stamp: "Formatted at 2026-07-09 12:29:44 on dist-test-slave-3zgk"
I20260709 12:29:52.281652 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:29:52.290172  6181 consensus_peers.cc:597] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:52.299045 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:52.300734 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:52.302482 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:52.306689  6472 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:29:52.363866 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:29:52.364185 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.059s	user 0.002s	sys 0.002s
I20260709 12:29:52.364553 24862 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:29:52.372771  6472 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.391160  6472 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.392148  6472 tablet_bootstrap.cc:492] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.392874  6472 ts_tablet_manager.cc:1403] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.009s
I20260709 12:29:52.395961  6472 raft_consensus.cc:359] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.397051  6472 raft_consensus.cc:740] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.397845  6472 consensus_queue.cc:260] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.400898  6472 ts_tablet_manager.cc:1434] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:29:52.401768  6472 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
W20260709 12:29:52.415669  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:52.419087  6472 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.420099  6472 tablet_bootstrap.cc:492] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.420806  6472 ts_tablet_manager.cc:1403] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 12:29:52.423805  6472 raft_consensus.cc:359] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.424532  6472 raft_consensus.cc:740] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.425302  6472 consensus_queue.cc:260] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.427342  6472 ts_tablet_manager.cc:1434] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:52.428159  6472 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.433948 24862 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:29:52.434379 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.070s	user 0.061s	sys 0.007s
I20260709 12:29:52.447947  6472 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.449278  6472 tablet_bootstrap.cc:492] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.450224  6472 ts_tablet_manager.cc:1403] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:29:52.453856  6472 raft_consensus.cc:359] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.454808  6472 raft_consensus.cc:740] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.455770  6472 consensus_queue.cc:260] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.458243  6472 ts_tablet_manager.cc:1434] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:52.459270  6472 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.477939  6472 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.478746  6472 tablet_bootstrap.cc:492] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.479295  6472 ts_tablet_manager.cc:1403] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:29:52.481405  6472 raft_consensus.cc:359] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.481964  6472 raft_consensus.cc:740] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.482589  6472 consensus_queue.cc:260] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.484344  6472 ts_tablet_manager.cc:1434] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:29:52.485193  6472 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.498185 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:46371
I20260709 12:29:52.498317  6534 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:46371 every 8 connection(s)
I20260709 12:29:52.500514  6472 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.501400  6472 tablet_bootstrap.cc:492] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.502063  6472 ts_tablet_manager.cc:1403] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 12:29:52.503647 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:52.505396  6472 raft_consensus.cc:359] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.506466  6472 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.507179  6472 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.513579  6472 ts_tablet_manager.cc:1434] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:29:52.515017  6472 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.541952  6472 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.543077  6472 tablet_bootstrap.cc:492] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.543715  6535 heartbeater.cc:344] Connected to a master server at 127.24.71.190:33429
I20260709 12:29:52.543843  6472 ts_tablet_manager.cc:1403] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.008s
I20260709 12:29:52.544335  6535 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:52.545396  6535 heartbeater.cc:507] Master 127.24.71.190:33429 requested a full tablet report, sending...
I20260709 12:29:52.547106  6472 raft_consensus.cc:359] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.547799  6472 raft_consensus.cc:740] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.549146  6472 consensus_queue.cc:260] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.550987  6290 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:52.551287  6472 ts_tablet_manager.cc:1434] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:29:52.551867  6290 raft_consensus.cc:606] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:52.552467  6472 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.553180  6038 ts_manager.cc:194] Re-registered known tserver with Master: 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371)
I20260709 12:29:52.555282  6290 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:52.556077  6290 raft_consensus.cc:606] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:52.559211  6290 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2403e18c0044be299916191c9728356"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:52.559881  6290 raft_consensus.cc:606] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:52.561223 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:52.562019 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:52.562893 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:52.564126  6038 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35382
I20260709 12:29:52.569198  6535 heartbeater.cc:499] Master 127.24.71.190:33429 was elected leader, sending a full tablet report...
I20260709 12:29:52.569751  6472 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.570791  6472 tablet_bootstrap.cc:492] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.571480  6472 ts_tablet_manager.cc:1403] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 12:29:52.574211  6472 raft_consensus.cc:359] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.574882  6472 raft_consensus.cc:740] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.575613  6472 consensus_queue.cc:260] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.577777  6472 ts_tablet_manager.cc:1434] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:52.578692  6472 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.595844  6472 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.596745  6472 tablet_bootstrap.cc:492] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.597012  6508 raft_consensus.cc:3060] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.597307  6472 ts_tablet_manager.cc:1403] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:29:52.599416  6472 raft_consensus.cc:359] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.599936  6472 raft_consensus.cc:740] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.600669  6472 consensus_queue.cc:260] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.602584  6472 ts_tablet_manager.cc:1434] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:52.602538  6507 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.604027  6472 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.623206  6472 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.624300  6472 tablet_bootstrap.cc:492] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.624918  6472 ts_tablet_manager.cc:1403] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.004s
I20260709 12:29:52.626940  6472 raft_consensus.cc:359] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.627450  6472 raft_consensus.cc:740] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.628042  6472 consensus_queue.cc:260] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.629796  6472 ts_tablet_manager.cc:1434] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:52.630759  6472 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Bootstrap starting.
I20260709 12:29:52.642043  6472 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:52.643052  6472 tablet_bootstrap.cc:492] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Bootstrap complete.
I20260709 12:29:52.643718  6472 ts_tablet_manager.cc:1403] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:29:52.646318  6453 raft_consensus.cc:993] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:52.646207  6472 raft_consensus.cc:359] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:52.646931  6472 raft_consensus.cc:740] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed410022a0347998e98296e2ded5313, State: Initialized, Role: FOLLOWER
I20260709 12:29:52.646978  6456 raft_consensus.cc:1081] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:52.647660  6472 consensus_queue.cc:260] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } }
I20260709 12:29:52.648862  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2403e18c0044be299916191c9728356"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:52.649515  6216 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.649919  6472 ts_tablet_manager.cc:1434] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:52.649919  6216 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.659703  6216 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.664408  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:52.665299  6216 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:52.665975  6507 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.667752  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:52.668458  6290 raft_consensus.cc:3055] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:52.668761  6290 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:52.669737  6290 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.671687  6290 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.672444  6453 raft_consensus.cc:993] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:52.672986  6453 raft_consensus.cc:1081] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:52.674204  6507 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:52.675755  6181 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:52.676060  6550 raft_consensus.cc:993] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:52.676889  6435 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:52.677034  6214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:52.677452  6456 raft_consensus.cc:1081] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:52.677772  6214 raft_consensus.cc:493] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.678314  6214 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.679508  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:52.680142  6216 raft_consensus.cc:493] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.680625  6216 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.681478  6290 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:52.688614  6216 raft_consensus.cc:515] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.689677  6214 raft_consensus.cc:515] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.691762  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:52.692297  6435 raft_consensus.cc:697] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:52.693683  6214 leader_election.cc:290] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:52.694164  6435 consensus_queue.cc:237] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.695380  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:52.696066  6508 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.695837  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e097ef621f4c629caecf2dab3cf4fa" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:52.696666  6289 raft_consensus.cc:3055] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:52.696816  6290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:52.697057  6289 raft_consensus.cc:740] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:52.697579  6290 raft_consensus.cc:3055] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:52.698025  6290 raft_consensus.cc:740] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:52.698137  6289 consensus_queue.cc:260] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.699522  6289 raft_consensus.cc:3060] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.699331  6290 consensus_queue.cc:260] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.701123  6290 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.704547  6508 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:52.705823  6181 leader_election.cc:304] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:52.706728  6552 raft_consensus.cc:2804] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:52.707931  6216 leader_election.cc:290] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:52.707494  6507 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.710541  6290 raft_consensus.cc:2468] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:52.710604  6038 catalog_manager.cc:5697] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 2 to 3, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:52.711792  6181 leader_election.cc:304] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:52.712718  6435 raft_consensus.cc:2804] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:52.713204  6435 raft_consensus.cc:697] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:52.714278  6435 consensus_queue.cc:237] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.720078  6289 raft_consensus.cc:2468] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:52.724062  6552 raft_consensus.cc:697] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 3 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:52.726207  6552 consensus_queue.cc:237] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:52.731483  6507 raft_consensus.cc:2468] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:52.750510  6038 catalog_manager.cc:5697] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 reported cstate change: term changed from 2 to 3, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:52.751721  6038 catalog_manager.cc:5697] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 1 to 2, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:52.838260  6508 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:53.097266  6289 raft_consensus.cc:1275] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:53.098666  6552 consensus_queue.cc:1048] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:53.109551  6508 raft_consensus.cc:1275] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:53.111801  6552 consensus_queue.cc:1048] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:53.221174  6508 raft_consensus.cc:1275] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:53.222740  6552 consensus_queue.cc:1048] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:53.233330  6289 raft_consensus.cc:1275] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:53.236406  6552 consensus_queue.cc:1048] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:29:53.252735  6508 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:53.254132  6552 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:53.264707  6289 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:53.268777  6554 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:29:53.416770  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:53.573992 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:53.591912  6216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89"
dest_uuid: "bab20726a5e647358591df3262751012"
mode: GRACEFUL
new_leader_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:43808
I20260709 12:29:53.592659  6216 raft_consensus.cc:606] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 LEADER]: Received request to transfer leadership to TS 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:53.595162  6216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2403e18c0044be299916191c9728356"
dest_uuid: "bab20726a5e647358591df3262751012"
mode: GRACEFUL
new_leader_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:43808
I20260709 12:29:53.595654  6216 raft_consensus.cc:606] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Received request to transfer leadership to TS 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:53.596915 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:53.597406 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:53.598081 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:53.824801  6568 raft_consensus.cc:993] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: : Instructing follower 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:53.825307  6554 raft_consensus.cc:1081] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Signalling peer 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:53.827019  6289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2403e18c0044be299916191c9728356"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:57604
I20260709 12:29:53.827510  6289 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:53.827795  6289 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:53.832501  6289 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:53.834139  6289 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 4 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:53.835085  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:53.835269  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:53.835791  6507 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:53.835901  6216 raft_consensus.cc:3055] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:53.836378  6216 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:53.837034  6216 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:53.837991  6216 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:53.841208  6507 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 4.
I20260709 12:29:53.842402  6255 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:53.843161  6216 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 4.
I20260709 12:29:53.843272  6550 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:53.843868  6550 raft_consensus.cc:697] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:53.844748  6550 consensus_queue.cc:237] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:53.851465  6039 catalog_manager.cc:5697] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 3 to 4, leader changed from bab20726a5e647358591df3262751012 (127.24.71.130) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 4 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:54.007216  6568 raft_consensus.cc:993] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: : Instructing follower 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:54.007716  6554 raft_consensus.cc:1081] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 LEADER]: Signalling peer 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:54.009433  6289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:57604
I20260709 12:29:54.009941  6289 raft_consensus.cc:493] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:54.010236  6289 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:54.015192  6289 raft_consensus.cc:515] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.016999  6289 leader_election.cc:290] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:54.017807  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:54.018235  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:54.018391  6507 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:54.018775  6216 raft_consensus.cc:3055] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:54.019049  6216 raft_consensus.cc:740] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:54.019845  6216 consensus_queue.cc:260] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.020869  6216 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:54.024075  6507 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:54.025398  6255 leader_election.cc:304] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:54.026265  6550 raft_consensus.cc:2804] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:54.026489  6216 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 2.
I20260709 12:29:54.026829  6550 raft_consensus.cc:697] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:54.027971  6550 consensus_queue.cc:237] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.036531  6039 catalog_manager.cc:5697] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 1 to 2, leader changed from bab20726a5e647358591df3262751012 (127.24.71.130) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 2 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:54.405089  6216 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:54.406695  6551 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:54.416115  6507 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:29:54.417784  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:54.419368  6580 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:54.441699  6216 raft_consensus.cc:1275] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:54.443164  6580 consensus_queue.cc:1048] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:54.457028  6507 raft_consensus.cc:1275] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:54.459977  6551 consensus_queue.cc:1048] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:54.606603 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:54.614331  6289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:54.614871  6289 raft_consensus.cc:606] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:54.617672  6216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22"
dest_uuid: "bab20726a5e647358591df3262751012"
mode: GRACEFUL
new_leader_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:43808
I20260709 12:29:54.618151  6216 raft_consensus.cc:606] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Received request to transfer leadership to TS 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:54.620425  6289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2403e18c0044be299916191c9728356"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:54.620844  6289 raft_consensus.cc:606] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:54.621780 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:54.622368 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:54.622890 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:54.709033  6554 raft_consensus.cc:993] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: : Instructing follower 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:54.709542  6588 raft_consensus.cc:1081] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Signalling peer 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:54.711105  6289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:57604
I20260709 12:29:54.711540  6289 raft_consensus.cc:493] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:54.711817  6289 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:54.716846  6289 raft_consensus.cc:515] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.718449  6289 leader_election.cc:290] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:54.719432  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:54.720124  6508 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:54.720021  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb57f67ef0f4a8f92d48b137fedfc22" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:54.720842  6216 raft_consensus.cc:3055] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:54.721167  6216 raft_consensus.cc:740] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:54.722020  6216 consensus_queue.cc:260] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.723263  6216 raft_consensus.cc:3060] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:54.727602  6508 raft_consensus.cc:2468] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 3.
I20260709 12:29:54.728950  6255 leader_election.cc:304] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:54.729818  6580 raft_consensus.cc:2804] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:54.729734  6216 raft_consensus.cc:2468] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 3.
I20260709 12:29:54.730446  6580 raft_consensus.cc:697] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:54.731362  6580 consensus_queue.cc:237] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.739053  6039 catalog_manager.cc:5697] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 2 to 3, leader changed from bab20726a5e647358591df3262751012 (127.24.71.130) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:54.799053  6551 raft_consensus.cc:993] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:54.799733  6580 raft_consensus.cc:1081] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:54.801191  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2403e18c0044be299916191c9728356"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:54.801678  6216 raft_consensus.cc:493] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:54.801967  6216 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:54.806901  6216 raft_consensus.cc:515] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.808549  6216 leader_election.cc:290] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 5 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:54.809373  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:54.809798  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2403e18c0044be299916191c9728356" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:54.810078  6508 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:54.810410  6289 raft_consensus.cc:3055] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:29:54.810709  6289 raft_consensus.cc:740] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:54.811498  6289 consensus_queue.cc:260] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.812738  6289 raft_consensus.cc:3060] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:54.817610  6508 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 5.
I20260709 12:29:54.818840  6181 leader_election.cc:304] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:54.819600  6588 raft_consensus.cc:2804] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:54.819857  6289 raft_consensus.cc:2468] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 5.
I20260709 12:29:54.820184  6588 raft_consensus.cc:697] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 5 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:54.821254  6588 consensus_queue.cc:237] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:54.828526  6039 catalog_manager.cc:5697] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 4 to 5, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 5 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.059218  6551 raft_consensus.cc:993] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:55.059751  6593 raft_consensus.cc:1081] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:55.061559  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:55.062085  6216 raft_consensus.cc:493] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.062374  6216 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.067473  6216 raft_consensus.cc:515] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:55.068969  6216 leader_election.cc:290] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:55.070096  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:55.070551  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:55.070832  6508 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.071166  6289 raft_consensus.cc:3055] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.071501  6289 raft_consensus.cc:740] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:55.072538  6289 consensus_queue.cc:260] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:55.074082  6289 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.079214  6508 raft_consensus.cc:2468] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:55.080792  6181 leader_election.cc:304] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:55.081259  6289 raft_consensus.cc:2468] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:55.081737  6588 raft_consensus.cc:2804] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.082319  6588 raft_consensus.cc:697] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:55.083791  6588 consensus_queue.cc:237] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:55.092247  6039 catalog_manager.cc:5697] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 1 to 2, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.112766  6216 raft_consensus.cc:1275] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:55.114145  6593 consensus_queue.cc:1048] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:55.124374  6508 raft_consensus.cc:1275] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:55.126052  6580 consensus_queue.cc:1048] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:55.309134  6289 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 5 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:29:55.310480  6588 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:29:55.319299  6508 raft_consensus.cc:1275] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 5 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:29:55.320824  6602 consensus_queue.cc:1048] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:29:55.418911  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:55.525970  6289 raft_consensus.cc:1275] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.527765  6602 consensus_queue.cc:1048] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:55.538275  6508 raft_consensus.cc:1275] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.540311  6588 consensus_queue.cc:1048] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:55.629418 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:55.637296  6216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83"
dest_uuid: "bab20726a5e647358591df3262751012"
mode: GRACEFUL
new_leader_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:43808
I20260709 12:29:55.637858  6216 raft_consensus.cc:606] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Received request to transfer leadership to TS 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:55.640661  6216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076"
dest_uuid: "bab20726a5e647358591df3262751012"
mode: GRACEFUL
new_leader_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:43808
I20260709 12:29:55.641149  6216 raft_consensus.cc:606] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Received request to transfer leadership to TS 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:55.643436  6289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:55.643910  6289 raft_consensus.cc:606] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:55.645071 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:55.645543 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:55.646121 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:55.866262  6588 raft_consensus.cc:993] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: : Instructing follower 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:55.866698  6594 raft_consensus.cc:1081] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Signalling peer 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:55.868324  6289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:57604
I20260709 12:29:55.868752  6289 raft_consensus.cc:493] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.869005  6289 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:55.873798  6289 raft_consensus.cc:515] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:55.875201  6289 leader_election.cc:290] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:55.876153  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:55.876385  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373cff2e13c443f8b363fe366dddd076" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:55.876842  6508 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:55.877076  6216 raft_consensus.cc:3055] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:55.877414  6216 raft_consensus.cc:740] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:55.878135  6216 consensus_queue.cc:260] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:55.879199  6216 raft_consensus.cc:3060] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:55.885412  6508 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 3.
I20260709 12:29:55.886426  6216 raft_consensus.cc:2468] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 3.
I20260709 12:29:55.886574  6255 leader_election.cc:304] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:55.887392  6605 raft_consensus.cc:2804] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:55.887794  6605 raft_consensus.cc:697] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:55.888527  6605 consensus_queue.cc:237] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:55.895794  6039 catalog_manager.cc:5697] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 2 to 3, leader changed from bab20726a5e647358591df3262751012 (127.24.71.130) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:56.046875  6588 raft_consensus.cc:993] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: : Instructing follower 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:56.047369  6594 raft_consensus.cc:1081] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Signalling peer 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:56.048949  6289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:57604
I20260709 12:29:56.049506  6289 raft_consensus.cc:493] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:56.049769  6289 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.055289  6289 raft_consensus.cc:515] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.056836  6289 leader_election.cc:290] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:56.057822  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:56.057827  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0848ca0813aa4fb48fc7089056db5f83" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:56.058473  6216 raft_consensus.cc:3055] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:56.058808  6216 raft_consensus.cc:740] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:56.058866  6508 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.059490  6216 consensus_queue.cc:260] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.060499  6216 raft_consensus.cc:3060] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.064942  6216 raft_consensus.cc:2468] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 3.
I20260709 12:29:56.065409  6508 raft_consensus.cc:2468] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 3.
I20260709 12:29:56.066397  6253 leader_election.cc:304] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5da4a94b64457bad9c235b50ab9817, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:56.067265  6605 raft_consensus.cc:2804] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:56.067768  6605 raft_consensus.cc:697] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:56.068559  6605 consensus_queue.cc:237] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.075179  6039 catalog_manager.cc:5697] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 2 to 3, leader changed from bab20726a5e647358591df3262751012 (127.24.71.130) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:56.127125  6606 raft_consensus.cc:993] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:56.127691  6605 raft_consensus.cc:1081] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:56.129453  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:56.129953  6216 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:56.130385  6216 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.135361  6216 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.136869  6216 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:56.137962  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:56.138018  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:56.138607  6508 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.138957  6289 raft_consensus.cc:3055] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:56.139247  6289 raft_consensus.cc:740] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:56.139971  6289 consensus_queue.cc:260] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.141142  6289 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.144166  6508 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:56.145350  6181 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:56.146128  6594 raft_consensus.cc:2804] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:56.146272  6289 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:56.146682  6594 raft_consensus.cc:697] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:56.147830  6594 consensus_queue.cc:237] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.155077  6039 catalog_manager.cc:5697] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 2 to 3, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:56.417217  6216 raft_consensus.cc:1275] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:56.418514  6605 consensus_queue.cc:1048] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:29:56.420444  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:56.430433  6508 raft_consensus.cc:1275] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:56.432212  6605 consensus_queue.cc:1048] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:56.559809  6216 raft_consensus.cc:1275] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:56.561455  6605 consensus_queue.cc:1048] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:56.570611  6508 raft_consensus.cc:1275] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:56.572162  6620 consensus_queue.cc:1048] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:56.592582  6508 raft_consensus.cc:1275] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:56.593928  6594 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:56.603863  6289 raft_consensus.cc:1275] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:56.605738  6623 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:56.652631 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:56.663829  6289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:56.664438  6289 raft_consensus.cc:606] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:56.667003  6289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77170d07974647869c088b9050034fdf"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
mode: GRACEFUL
new_leader_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:49968
I20260709 12:29:56.667443  6289 raft_consensus.cc:606] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Received request to transfer leadership to TS bab20726a5e647358591df3262751012
I20260709 12:29:56.670034  6216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641"
dest_uuid: "bab20726a5e647358591df3262751012"
mode: GRACEFUL
new_leader_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:43808
I20260709 12:29:56.670511  6216 raft_consensus.cc:606] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Received request to transfer leadership to TS 3f5da4a94b64457bad9c235b50ab9817
I20260709 12:29:56.671627 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:56.672096 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:56.672668 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:56.962535  6614 raft_consensus.cc:993] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:56.963146  6606 raft_consensus.cc:1081] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:56.964568  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:56.965026  6216 raft_consensus.cc:493] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:56.965454  6216 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.970958  6216 raft_consensus.cc:515] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.972631  6216 leader_election.cc:290] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:56.973803  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:56.973588  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa4bb002c2842b78da14a6de8d0da89" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:56.974334  6289 raft_consensus.cc:3055] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:56.974376  6508 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.974746  6289 raft_consensus.cc:740] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:56.975548  6289 consensus_queue.cc:260] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.976794  6289 raft_consensus.cc:3060] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:56.981561  6508 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:56.982831  6181 leader_election.cc:304] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:56.983268  6289 raft_consensus.cc:2468] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 3.
I20260709 12:29:56.983583  6623 raft_consensus.cc:2804] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:56.984792  6623 raft_consensus.cc:697] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:56.985530  6623 consensus_queue.cc:237] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:56.993265  6038 catalog_manager.cc:5697] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 reported cstate change: term changed from 2 to 3, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.022756  6614 raft_consensus.cc:993] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: : Instructing follower bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:57.023310  6614 raft_consensus.cc:1081] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Signalling peer bab20726a5e647358591df3262751012 to start an election
I20260709 12:29:57.025205  6216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77170d07974647869c088b9050034fdf"
dest_uuid: "bab20726a5e647358591df3262751012"
 from {username='slave'} at 127.0.0.1:57986
I20260709 12:29:57.025695  6216 raft_consensus.cc:493] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.025982  6216 raft_consensus.cc:3060] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.030958  6216 raft_consensus.cc:515] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:57.032507  6216 leader_election.cc:290] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131:35819)
I20260709 12:29:57.033417  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77170d07974647869c088b9050034fdf" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:57.033651  6289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77170d07974647869c088b9050034fdf" candidate_uuid: "bab20726a5e647358591df3262751012" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
I20260709 12:29:57.034049  6508 raft_consensus.cc:3060] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.034171  6289 raft_consensus.cc:3055] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:57.034550  6289 raft_consensus.cc:740] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:57.035425  6289 consensus_queue.cc:260] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:57.036831  6289 raft_consensus.cc:3060] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.042567  6508 raft_consensus.cc:2468] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:57.044135  6181 leader_election.cc:304] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, bab20726a5e647358591df3262751012; no voters: 
I20260709 12:29:57.044874  6289 raft_consensus.cc:2468] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab20726a5e647358591df3262751012 in term 2.
I20260709 12:29:57.045019  6623 raft_consensus.cc:2804] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:57.045744  6623 raft_consensus.cc:697] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 2 LEADER]: Becoming Leader. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:57.046685  6623 consensus_queue.cc:237] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:57.054580  6038 catalog_manager.cc:5697] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 reported cstate change: term changed from 1 to 2, leader changed from 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131) to bab20726a5e647358591df3262751012 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "bab20726a5e647358591df3262751012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.225523  6588 raft_consensus.cc:993] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: : Instructing follower 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:57.226083  6623 raft_consensus.cc:1081] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Signalling peer 3f5da4a94b64457bad9c235b50ab9817 to start an election
I20260709 12:29:57.227751  6289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641"
dest_uuid: "3f5da4a94b64457bad9c235b50ab9817"
 from {username='slave'} at 127.0.0.1:57604
I20260709 12:29:57.228376  6289 raft_consensus.cc:493] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.228739  6289 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:57.234283  6289 raft_consensus.cc:515] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:57.235903  6289 leader_election.cc:290] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 4 election: Requested vote from peers 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371), bab20726a5e647358591df3262751012 (127.24.71.130:38441)
I20260709 12:29:57.236970  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ed410022a0347998e98296e2ded5313"
I20260709 12:29:57.237103  6216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea1f12226fbc4fe1b4386433458ca641" candidate_uuid: "3f5da4a94b64457bad9c235b50ab9817" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bab20726a5e647358591df3262751012"
I20260709 12:29:57.237792  6508 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:57.237793  6216 raft_consensus.cc:3055] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:57.238288  6216 raft_consensus.cc:740] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bab20726a5e647358591df3262751012, State: Running, Role: LEADER
I20260709 12:29:57.239094  6216 consensus_queue.cc:260] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:57.240307  6216 raft_consensus.cc:3060] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:57.245069  6508 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 4.
I20260709 12:29:57.246462  6255 leader_election.cc:304] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed410022a0347998e98296e2ded5313, 3f5da4a94b64457bad9c235b50ab9817; no voters: 
I20260709 12:29:57.246966  6216 raft_consensus.cc:2468] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f5da4a94b64457bad9c235b50ab9817 in term 4.
I20260709 12:29:57.247153  6614 raft_consensus.cc:2804] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:57.247843  6614 raft_consensus.cc:697] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Becoming Leader. State: Replica: 3f5da4a94b64457bad9c235b50ab9817, State: Running, Role: LEADER
I20260709 12:29:57.248667  6614 consensus_queue.cc:237] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } }
I20260709 12:29:57.254837  6038 catalog_manager.cc:5697] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 reported cstate change: term changed from 3 to 4, leader changed from bab20726a5e647358591df3262751012 (127.24.71.130) to 3f5da4a94b64457bad9c235b50ab9817 (127.24.71.131). New cstate: current_term: 4 leader_uuid: "3f5da4a94b64457bad9c235b50ab9817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:57.421522  6087 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:57.442337  6508 raft_consensus.cc:1275] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:57.443719  6623 consensus_queue.cc:1048] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:57.453601  6289 raft_consensus.cc:1275] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:57.455096  6623 consensus_queue.cc:1048] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:57.585384  6508 raft_consensus.cc:1275] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:57.586889  6623 consensus_queue.cc:1048] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:57.593438  6289 raft_consensus.cc:1275] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Refusing update from remote peer bab20726a5e647358591df3262751012: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:57.595866  6588 consensus_queue.cc:1048] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5da4a94b64457bad9c235b50ab9817" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 35819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:57.638541  6216 raft_consensus.cc:1275] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:57.639971  6606 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab20726a5e647358591df3262751012" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:57.650766  6508 raft_consensus.cc:1275] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Refusing update from remote peer 3f5da4a94b64457bad9c235b50ab9817: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:57.652452  6639 consensus_queue.cc:1048] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed410022a0347998e98296e2ded5313" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:57.678953 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:57.710733 24862 tablet_server.cc:179] TabletServer@127.24.71.129:46371 shutting down...
I20260709 12:29:57.731316 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:57.732390 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.733203 24862 raft_consensus.cc:2243] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.733709 24862 raft_consensus.cc:2272] T 7863463b701e46ee99e63b74fd272495 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.735751 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.736491 24862 raft_consensus.cc:2243] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.736940 24862 raft_consensus.cc:2272] T 1c5fcd86214e4cefa4a243936b949014 P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.738869 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.739423 24862 raft_consensus.cc:2243] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.739853 24862 raft_consensus.cc:2272] T 0fa4bb002c2842b78da14a6de8d0da89 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.742769 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.743383 24862 raft_consensus.cc:2243] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.743844 24862 raft_consensus.cc:2272] T ea1f12226fbc4fe1b4386433458ca641 P 3ed410022a0347998e98296e2ded5313 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.746171 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.746754 24862 raft_consensus.cc:2243] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.747189 24862 raft_consensus.cc:2272] T 0848ca0813aa4fb48fc7089056db5f83 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.749053 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.749578 24862 raft_consensus.cc:2243] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.750013 24862 raft_consensus.cc:2272] T abb57f67ef0f4a8f92d48b137fedfc22 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.751778 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.752380 24862 raft_consensus.cc:2243] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.752789 24862 raft_consensus.cc:2272] T 77170d07974647869c088b9050034fdf P 3ed410022a0347998e98296e2ded5313 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.755002 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.755551 24862 raft_consensus.cc:2243] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.755985 24862 raft_consensus.cc:2272] T 16e097ef621f4c629caecf2dab3cf4fa P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.757773 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.758270 24862 raft_consensus.cc:2243] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.758679 24862 raft_consensus.cc:2272] T c2403e18c0044be299916191c9728356 P 3ed410022a0347998e98296e2ded5313 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.760563 24862 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:57.761247 24862 raft_consensus.cc:2243] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.761691 24862 raft_consensus.cc:2272] T 373cff2e13c443f8b363fe366dddd076 P 3ed410022a0347998e98296e2ded5313 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:57.763095  6255 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 101 similar messages]
W20260709 12:29:57.766381  6255 consensus_peers.cc:597] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:57.790539 24862 tablet_server.cc:196] TabletServer@127.24.71.129:46371 shutdown complete.
W20260709 12:29:57.803743  6181 consensus_peers.cc:597] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:57.813437 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
I20260709 12:29:57.840947 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:57.841652 24862 tablet_replica.cc:333] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.842535 24862 raft_consensus.cc:2243] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.843058 24862 raft_consensus.cc:2272] T 373cff2e13c443f8b363fe366dddd076 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.845520 24862 tablet_replica.cc:333] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.846170 24862 raft_consensus.cc:2243] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.846987 24862 raft_consensus.cc:2272] T 77170d07974647869c088b9050034fdf P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.849828 24862 tablet_replica.cc:333] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.850498 24862 raft_consensus.cc:2243] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.850975 24862 raft_consensus.cc:2272] T ea1f12226fbc4fe1b4386433458ca641 P bab20726a5e647358591df3262751012 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.854180 24862 tablet_replica.cc:333] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.854840 24862 raft_consensus.cc:2243] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.855337 24862 raft_consensus.cc:2272] T 0848ca0813aa4fb48fc7089056db5f83 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.857828 24862 tablet_replica.cc:333] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.858500 24862 raft_consensus.cc:2243] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.858942 24862 raft_consensus.cc:2272] T abb57f67ef0f4a8f92d48b137fedfc22 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.861616 24862 tablet_replica.cc:333] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.862210 24862 raft_consensus.cc:2243] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.863000 24862 raft_consensus.cc:2272] T c2403e18c0044be299916191c9728356 P bab20726a5e647358591df3262751012 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:57.863175  6255 consensus_peers.cc:597] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:57.865494 24862 tablet_replica.cc:333] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.866254 24862 raft_consensus.cc:2243] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.867044 24862 raft_consensus.cc:2272] T 16e097ef621f4c629caecf2dab3cf4fa P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.869407 24862 tablet_replica.cc:333] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.870294 24862 raft_consensus.cc:2243] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.871284 24862 raft_consensus.cc:2272] T 0fa4bb002c2842b78da14a6de8d0da89 P bab20726a5e647358591df3262751012 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.873975 24862 tablet_replica.cc:333] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.874783 24862 raft_consensus.cc:2243] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.875778 24862 raft_consensus.cc:2272] T 1c5fcd86214e4cefa4a243936b949014 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.878459 24862 tablet_replica.cc:333] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012: stopping tablet replica
I20260709 12:29:57.879155 24862 raft_consensus.cc:2243] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.879696 24862 raft_consensus.cc:2272] T 7863463b701e46ee99e63b74fd272495 P bab20726a5e647358591df3262751012 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:57.880872  6255 consensus_peers.cc:597] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer 3ed410022a0347998e98296e2ded5313 (127.24.71.129:46371): Couldn't send request to peer 3ed410022a0347998e98296e2ded5313. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:46371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:57.915061 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
W20260709 12:29:57.924710  6253 consensus_peers.cc:597] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 -> Peer bab20726a5e647358591df3262751012 (127.24.71.130:38441): Couldn't send request to peer bab20726a5e647358591df3262751012. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:38441: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:57.943890 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
I20260709 12:29:57.975510 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:57.976325 24862 tablet_replica.cc:333] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.976979 24862 raft_consensus.cc:2243] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.977762 24862 raft_consensus.cc:2272] T 373cff2e13c443f8b363fe366dddd076 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.979837 24862 tablet_replica.cc:333] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.980544 24862 raft_consensus.cc:2243] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.980939 24862 raft_consensus.cc:2272] T 0fa4bb002c2842b78da14a6de8d0da89 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.984009 24862 tablet_replica.cc:333] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.984656 24862 raft_consensus.cc:2243] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.985450 24862 raft_consensus.cc:2272] T ea1f12226fbc4fe1b4386433458ca641 P 3f5da4a94b64457bad9c235b50ab9817 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.987969 24862 tablet_replica.cc:333] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.988600 24862 raft_consensus.cc:2243] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.989012 24862 raft_consensus.cc:2272] T c2403e18c0044be299916191c9728356 P 3f5da4a94b64457bad9c235b50ab9817 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.990962 24862 tablet_replica.cc:333] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.991442 24862 raft_consensus.cc:2243] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.991811 24862 raft_consensus.cc:2272] T 16e097ef621f4c629caecf2dab3cf4fa P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.993908 24862 tablet_replica.cc:333] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.994364 24862 raft_consensus.cc:2243] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:57.994778 24862 raft_consensus.cc:2272] T 1c5fcd86214e4cefa4a243936b949014 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:57.996948 24862 tablet_replica.cc:333] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:57.997421 24862 raft_consensus.cc:2243] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:57.998173 24862 raft_consensus.cc:2272] T 7863463b701e46ee99e63b74fd272495 P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:58.001012 24862 tablet_replica.cc:333] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:58.001585 24862 raft_consensus.cc:2243] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:58.002012 24862 raft_consensus.cc:2272] T 77170d07974647869c088b9050034fdf P 3f5da4a94b64457bad9c235b50ab9817 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:58.005030 24862 tablet_replica.cc:333] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:58.005689 24862 raft_consensus.cc:2243] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:58.006646 24862 raft_consensus.cc:2272] T 0848ca0813aa4fb48fc7089056db5f83 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:58.008910 24862 tablet_replica.cc:333] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817: stopping tablet replica
I20260709 12:29:58.009400 24862 raft_consensus.cc:2243] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:58.010182 24862 raft_consensus.cc:2272] T abb57f67ef0f4a8f92d48b137fedfc22 P 3f5da4a94b64457bad9c235b50ab9817 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:58.044458 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:29:58.071712 24862 master.cc:562] Master@127.24.71.190:33429 shutting down...
I20260709 12:29:58.102501 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:58.103137 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:58.103478 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8c8d2ad8e3a04631a3260fcb14a176f3: stopping tablet replica
I20260709 12:29:58.126230 24862 master.cc:584] Master@127.24.71.190:33429 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13945 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:29:58.174603 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:46387
I20260709 12:29:58.175707 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:58.182149  6647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:58.182196  6648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:58.183720  6650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:58.185209 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:58.186270 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:58.186461 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:58.186650 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600198186635 us; error 0 us; skew 500 ppm
I20260709 12:29:58.187258 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:58.190382 24862 webserver.cc:533] Webserver started at http://127.24.71.190:37021/ using document root <none> and password file <none>
I20260709 12:29:58.191023 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:58.191244 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:58.191599 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:58.192953 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "1e64307aad2e4d33947f218dbae22d8a"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.198297 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:29:58.202469  6655 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.203352 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:29:58.203691 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "1e64307aad2e4d33947f218dbae22d8a"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.204015 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:58.223109 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:58.224417 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:58.267123 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:46387
I20260709 12:29:58.267431  6706 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:46387 every 8 connection(s)
I20260709 12:29:58.271066  6707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.282889  6707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a: Bootstrap starting.
I20260709 12:29:58.287885  6707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.292774  6707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a: No bootstrap required, opened a new log
I20260709 12:29:58.295090  6707 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64307aad2e4d33947f218dbae22d8a" member_type: VOTER }
I20260709 12:29:58.295492  6707 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.295682  6707 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e64307aad2e4d33947f218dbae22d8a, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.296348  6707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64307aad2e4d33947f218dbae22d8a" member_type: VOTER }
I20260709 12:29:58.296804  6707 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:58.297001  6707 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:58.297222  6707 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.302649  6707 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64307aad2e4d33947f218dbae22d8a" member_type: VOTER }
I20260709 12:29:58.303187  6707 leader_election.cc:304] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e64307aad2e4d33947f218dbae22d8a; no voters: 
I20260709 12:29:58.304461  6707 leader_election.cc:290] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:58.304857  6710 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:58.306301  6710 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 1 LEADER]: Becoming Leader. State: Replica: 1e64307aad2e4d33947f218dbae22d8a, State: Running, Role: LEADER
I20260709 12:29:58.306994  6710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64307aad2e4d33947f218dbae22d8a" member_type: VOTER }
I20260709 12:29:58.307593  6707 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:58.312089  6712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e64307aad2e4d33947f218dbae22d8a. Latest consensus state: current_term: 1 leader_uuid: "1e64307aad2e4d33947f218dbae22d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64307aad2e4d33947f218dbae22d8a" member_type: VOTER } }
I20260709 12:29:58.312943  6712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:58.313696  6711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e64307aad2e4d33947f218dbae22d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64307aad2e4d33947f218dbae22d8a" member_type: VOTER } }
I20260709 12:29:58.314282  6711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:58.315002  6717 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:58.320128  6717 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:58.325428 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:58.329530  6717 catalog_manager.cc:1383] Generated new cluster ID: 8f53f48ce3e147558381343a1be4f3e6
I20260709 12:29:58.329824  6717 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:58.371502  6717 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:58.373054  6717 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:58.391249  6717 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a: Generated new TSK 0
I20260709 12:29:58.391930  6717 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:58.457526 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:58.464001  6728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:58.465643  6729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:58.467257  6731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:58.467461 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:58.468628 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:58.468863 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:58.469031 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600198469009 us; error 0 us; skew 500 ppm
I20260709 12:29:58.469570 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:58.472337 24862 webserver.cc:533] Webserver started at http://127.24.71.129:43229/ using document root <none> and password file <none>
I20260709 12:29:58.472839 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:58.473062 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:58.473342 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:58.474488 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "a2645100642f4b1e8559b708fe054d6d"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.479451 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:29:58.483397  6737 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.484508 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 12:29:58.484864 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "a2645100642f4b1e8559b708fe054d6d"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.485188 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:58.502640 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:58.503881 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:58.505435 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:58.507972 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:58.508212 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.508472 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:58.508642 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.552588 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:38001
I20260709 12:29:58.552752  6799 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:38001 every 8 connection(s)
I20260709 12:29:58.557754 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:58.566356  6804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:58.566829  6805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:58.570046 24862 server_base.cc:1061] running on GCE node
W20260709 12:29:58.570745  6807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:58.572022 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:58.572400 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:58.572633 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600198572615 us; error 0 us; skew 500 ppm
I20260709 12:29:58.573352 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:58.573984  6800 heartbeater.cc:344] Connected to a master server at 127.24.71.190:46387
I20260709 12:29:58.574354  6800 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:58.575109  6800 heartbeater.cc:507] Master 127.24.71.190:46387 requested a full tablet report, sending...
I20260709 12:29:58.576268 24862 webserver.cc:533] Webserver started at http://127.24.71.130:42609/ using document root <none> and password file <none>
I20260709 12:29:58.576956 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:58.577188 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:58.577463 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:58.577801  6672 ts_manager.cc:194] Registered new tserver with Master: a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001)
I20260709 12:29:58.578685 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "a35f8f9fd6084cd080fe389b2d580230"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.579738  6672 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35366
I20260709 12:29:58.583809 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:29:58.587999  6812 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.589193 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:58.589632 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "a35f8f9fd6084cd080fe389b2d580230"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.589949 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:58.600541 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:58.601773 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:58.603262 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:58.605532 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:58.605772 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.606040 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:58.606222 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.651387 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:34033
I20260709 12:29:58.651484  6874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:34033 every 8 connection(s)
I20260709 12:29:58.666126 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:58.674808  6878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:58.678290  6879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:58.678460  6875 heartbeater.cc:344] Connected to a master server at 127.24.71.190:46387
I20260709 12:29:58.679133  6875 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:58.679584  6881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:58.680361  6875 heartbeater.cc:507] Master 127.24.71.190:46387 requested a full tablet report, sending...
I20260709 12:29:58.681260 24862 server_base.cc:1061] running on GCE node
I20260709 12:29:58.682333 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:58.682570 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:58.682688  6672 ts_manager.cc:194] Registered new tserver with Master: a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:58.682765 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600198682747 us; error 0 us; skew 500 ppm
I20260709 12:29:58.683681 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:58.684628  6672 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35374
I20260709 12:29:58.686829 24862 webserver.cc:533] Webserver started at http://127.24.71.131:45321/ using document root <none> and password file <none>
I20260709 12:29:58.687439 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:58.687659 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:58.687924 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:58.689142 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "052c3ab125684cc0bef2003abeb9e952"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.693681 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:58.697149  6886 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.698009 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:58.698333 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "052c3ab125684cc0bef2003abeb9e952"
format_stamp: "Formatted at 2026-07-09 12:29:58 on dist-test-slave-3zgk"
I20260709 12:29:58.698632 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:58.719983 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:58.721240 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:58.722734 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:58.724954 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:58.725186 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.725430 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:58.725603 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:58.769378 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:40349
I20260709 12:29:58.769492  6948 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:40349 every 8 connection(s)
I20260709 12:29:58.784157  6949 heartbeater.cc:344] Connected to a master server at 127.24.71.190:46387
I20260709 12:29:58.784700  6949 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:58.785473  6949 heartbeater.cc:507] Master 127.24.71.190:46387 requested a full tablet report, sending...
I20260709 12:29:58.787700  6672 ts_manager.cc:194] Registered new tserver with Master: 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:29:58.788753 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016108265s
I20260709 12:29:58.789301  6672 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35376
I20260709 12:29:58.818270  6672 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35382:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:58.820791  6672 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:58.884908  6914 tablet_service.cc:1511] Processing CreateTablet for tablet 3a20fb5560e945a9b04da2be383bad5f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=94e09e3ec6884e11b94fe88dd91a8494]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:58.886422  6914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a20fb5560e945a9b04da2be383bad5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.887921  6913 tablet_service.cc:1511] Processing CreateTablet for tablet 2f3c162a99e74b3f9c37c993f5b6e2e5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=94e09e3ec6884e11b94fe88dd91a8494]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:58.889542  6913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f3c162a99e74b3f9c37c993f5b6e2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.890110  6765 tablet_service.cc:1511] Processing CreateTablet for tablet 3a20fb5560e945a9b04da2be383bad5f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=94e09e3ec6884e11b94fe88dd91a8494]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:58.888024  6764 tablet_service.cc:1511] Processing CreateTablet for tablet 2f3c162a99e74b3f9c37c993f5b6e2e5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=94e09e3ec6884e11b94fe88dd91a8494]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:58.891520  6765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a20fb5560e945a9b04da2be383bad5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.892395  6764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f3c162a99e74b3f9c37c993f5b6e2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.904500  6840 tablet_service.cc:1511] Processing CreateTablet for tablet 3a20fb5560e945a9b04da2be383bad5f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=94e09e3ec6884e11b94fe88dd91a8494]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:58.906033  6840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a20fb5560e945a9b04da2be383bad5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.908469  6839 tablet_service.cc:1511] Processing CreateTablet for tablet 2f3c162a99e74b3f9c37c993f5b6e2e5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=94e09e3ec6884e11b94fe88dd91a8494]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:58.910005  6839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f3c162a99e74b3f9c37c993f5b6e2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.929179  6964 tablet_bootstrap.cc:492] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: Bootstrap starting.
I20260709 12:29:58.938637  6964 tablet_bootstrap.cc:654] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.940124  6966 tablet_bootstrap.cc:492] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: Bootstrap starting.
I20260709 12:29:58.941229  6965 tablet_bootstrap.cc:492] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230: Bootstrap starting.
I20260709 12:29:58.944868  6964 tablet_bootstrap.cc:492] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: No bootstrap required, opened a new log
I20260709 12:29:58.945631  6964 ts_tablet_manager.cc:1403] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.005s
I20260709 12:29:58.946923  6966 tablet_bootstrap.cc:654] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.948961  6965 tablet_bootstrap.cc:654] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.949137  6964 raft_consensus.cc:359] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:58.950016  6964 raft_consensus.cc:385] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.950381  6964 raft_consensus.cc:740] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.951251  6964 consensus_queue.cc:260] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:58.952199  6966 tablet_bootstrap.cc:492] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: No bootstrap required, opened a new log
I20260709 12:29:58.953982  6800 heartbeater.cc:499] Master 127.24.71.190:46387 was elected leader, sending a full tablet report...
I20260709 12:29:58.954739  6964 ts_tablet_manager.cc:1434] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.006s
I20260709 12:29:58.953982  6966 ts_tablet_manager.cc:1403] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:29:58.955343  6965 tablet_bootstrap.cc:492] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230: No bootstrap required, opened a new log
I20260709 12:29:58.956416  6965 ts_tablet_manager.cc:1403] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 12:29:58.956586  6964 tablet_bootstrap.cc:492] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d: Bootstrap starting.
I20260709 12:29:58.959369  6966 raft_consensus.cc:359] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:58.959565  6965 raft_consensus.cc:359] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:58.960357  6966 raft_consensus.cc:385] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.960440  6965 raft_consensus.cc:385] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.960815  6966 raft_consensus.cc:740] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.960933  6965 raft_consensus.cc:740] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.961958  6966 consensus_queue.cc:260] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:58.961959  6965 consensus_queue.cc:260] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:58.964416  6964 tablet_bootstrap.cc:654] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.971300  6875 heartbeater.cc:499] Master 127.24.71.190:46387 was elected leader, sending a full tablet report...
I20260709 12:29:58.972432  6965 ts_tablet_manager.cc:1434] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 12:29:58.973562  6965 tablet_bootstrap.cc:492] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230: Bootstrap starting.
I20260709 12:29:58.975616  6949 heartbeater.cc:499] Master 127.24.71.190:46387 was elected leader, sending a full tablet report...
I20260709 12:29:58.976662  6966 ts_tablet_manager.cc:1434] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: Time spent starting tablet: real 0.021s	user 0.014s	sys 0.001s
I20260709 12:29:58.977810  6966 tablet_bootstrap.cc:492] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952: Bootstrap starting.
I20260709 12:29:58.986176  6966 tablet_bootstrap.cc:654] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.986176  6965 tablet_bootstrap.cc:654] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.998816  6966 tablet_bootstrap.cc:492] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952: No bootstrap required, opened a new log
I20260709 12:29:58.999491  6966 ts_tablet_manager.cc:1403] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.010s
I20260709 12:29:59.004000  6966 raft_consensus.cc:359] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.005039  6966 raft_consensus.cc:385] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.005342  6966 raft_consensus.cc:740] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.006765  6964 tablet_bootstrap.cc:492] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d: No bootstrap required, opened a new log
I20260709 12:29:59.006340  6966 consensus_queue.cc:260] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.007195  6965 tablet_bootstrap.cc:492] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230: No bootstrap required, opened a new log
I20260709 12:29:59.007391  6964 ts_tablet_manager.cc:1403] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d: Time spent bootstrapping tablet: real 0.051s	user 0.023s	sys 0.025s
I20260709 12:29:59.007814  6965 ts_tablet_manager.cc:1403] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.005s
I20260709 12:29:59.010878  6964 raft_consensus.cc:359] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.011407  6966 ts_tablet_manager.cc:1434] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:29:59.011274  6965 raft_consensus.cc:359] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.011822  6964 raft_consensus.cc:385] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.012281  6965 raft_consensus.cc:385] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.012362  6964 raft_consensus.cc:740] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.012650  6965 raft_consensus.cc:740] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.013325  6964 consensus_queue.cc:260] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.013552  6965 consensus_queue.cc:260] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.015632  6964 ts_tablet_manager.cc:1434] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 12:29:59.017735  6965 ts_tablet_manager.cc:1434] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:29:59.032768  6972 raft_consensus.cc:493] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.033461  6972 raft_consensus.cc:515] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.035674  6972 leader_election.cc:290] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.047000  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3c162a99e74b3f9c37c993f5b6e2e5" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:29:59.047752  6775 raft_consensus.cc:2468] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 0.
I20260709 12:29:59.049098  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3c162a99e74b3f9c37c993f5b6e2e5" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230" is_pre_election: true
I20260709 12:29:59.049176  6890 leader_election.cc:304] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d; no voters: 
I20260709 12:29:59.050187  6850 raft_consensus.cc:2468] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 0.
I20260709 12:29:59.050647  6972 raft_consensus.cc:2804] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.051083  6972 raft_consensus.cc:493] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.051618  6972 raft_consensus.cc:3060] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.057557  6972 raft_consensus.cc:515] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.059231  6972 leader_election.cc:290] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.060055  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3c162a99e74b3f9c37c993f5b6e2e5" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:29:59.060273  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3c162a99e74b3f9c37c993f5b6e2e5" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:29:59.060685  6775 raft_consensus.cc:3060] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.060938  6850 raft_consensus.cc:3060] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.066488  6775 raft_consensus.cc:2468] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 1.
I20260709 12:29:59.066486  6850 raft_consensus.cc:2468] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 1.
I20260709 12:29:59.067724  6890 leader_election.cc:304] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d; no voters: 
I20260709 12:29:59.068537  6972 raft_consensus.cc:2804] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.069621  6972 raft_consensus.cc:697] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Becoming Leader. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Running, Role: LEADER
I20260709 12:29:59.070299  6972 consensus_queue.cc:237] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.079784  6671 catalog_manager.cc:5697] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 reported cstate change: term changed from 0 to 1, leader changed from <none> to 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "052c3ab125684cc0bef2003abeb9e952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:59.254566  6972 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.255210  6972 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:59.257540  6972 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.258168  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:29:59.258693  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230" is_pre_election: true
I20260709 12:29:59.259016  6775 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 0.
I20260709 12:29:59.259408  6850 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 0.
I20260709 12:29:59.260308  6890 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d; no voters: 
I20260709 12:29:59.261046  6972 raft_consensus.cc:2804] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.261435  6972 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.261704  6972 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.267211  6972 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:59.269326  6972 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.270265  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:29:59.270493  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:29:59.270884  6775 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.271188  6850 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.276654  6850 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 1.
I20260709 12:29:59.277247  6775 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 1.
I20260709 12:29:59.277807  6887 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:29:59.278854  6972 raft_consensus.cc:2804] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.279402  6972 raft_consensus.cc:697] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Becoming Leader. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Running, Role: LEADER
I20260709 12:29:59.280081  6972 consensus_queue.cc:237] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:59.287321  6671 catalog_manager.cc:5697] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 reported cstate change: term changed from 0 to 1, leader changed from <none> to 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "052c3ab125684cc0bef2003abeb9e952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.376698  6671 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35394:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:59.378800  6671 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:59.405221  6765 tablet_service.cc:1511] Processing CreateTablet for tablet a5684765fde24d788681d55f4a194ed4 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=62989894740e4f74b399030da4640402]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:59.405939  6839 tablet_service.cc:1511] Processing CreateTablet for tablet a5684765fde24d788681d55f4a194ed4 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=62989894740e4f74b399030da4640402]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:59.406598  6765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5684765fde24d788681d55f4a194ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:59.407236  6839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5684765fde24d788681d55f4a194ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:59.408030  6914 tablet_service.cc:1511] Processing CreateTablet for tablet a5684765fde24d788681d55f4a194ed4 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=62989894740e4f74b399030da4640402]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:59.409394  6914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5684765fde24d788681d55f4a194ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:59.414009  6764 tablet_service.cc:1511] Processing CreateTablet for tablet 4a36daab7bbb4b428d3e32f0a633947d (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=62989894740e4f74b399030da4640402]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:59.415405  6764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a36daab7bbb4b428d3e32f0a633947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:59.416554  6913 tablet_service.cc:1511] Processing CreateTablet for tablet 4a36daab7bbb4b428d3e32f0a633947d (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=62989894740e4f74b399030da4640402]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:59.417873  6913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a36daab7bbb4b428d3e32f0a633947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:59.420151  6840 tablet_service.cc:1511] Processing CreateTablet for tablet 4a36daab7bbb4b428d3e32f0a633947d (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=62989894740e4f74b399030da4640402]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:59.421559  6840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a36daab7bbb4b428d3e32f0a633947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:59.425763  6965 tablet_bootstrap.cc:492] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: Bootstrap starting.
I20260709 12:29:59.432319  6965 tablet_bootstrap.cc:654] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.433818  6964 tablet_bootstrap.cc:492] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: Bootstrap starting.
I20260709 12:29:59.441650  6964 tablet_bootstrap.cc:654] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.453166  6966 tablet_bootstrap.cc:492] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952: Bootstrap starting.
I20260709 12:29:59.453689  6964 tablet_bootstrap.cc:492] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: No bootstrap required, opened a new log
I20260709 12:29:59.454264  6964 ts_tablet_manager.cc:1403] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 12:29:59.457686  6964 raft_consensus.cc:359] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.458541  6964 raft_consensus.cc:385] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.458956  6964 raft_consensus.cc:740] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.458148  6965 tablet_bootstrap.cc:492] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: No bootstrap required, opened a new log
I20260709 12:29:59.460083  6965 ts_tablet_manager.cc:1403] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.005s
I20260709 12:29:59.460573  6964 consensus_queue.cc:260] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.461176  6966 tablet_bootstrap.cc:654] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.462997  6964 ts_tablet_manager.cc:1434] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:59.464145  6964 tablet_bootstrap.cc:492] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: Bootstrap starting.
I20260709 12:29:59.463110  6965 raft_consensus.cc:359] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.467164  6965 raft_consensus.cc:385] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.467509  6965 raft_consensus.cc:740] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.468439  6965 consensus_queue.cc:260] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.470724  6965 ts_tablet_manager.cc:1434] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.003s
I20260709 12:29:59.471789  6965 tablet_bootstrap.cc:492] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230: Bootstrap starting.
I20260709 12:29:59.474071  6964 tablet_bootstrap.cc:654] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.478767  6965 tablet_bootstrap.cc:654] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.480079  6966 tablet_bootstrap.cc:492] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952: No bootstrap required, opened a new log
I20260709 12:29:59.480770  6966 ts_tablet_manager.cc:1403] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.003s
I20260709 12:29:59.486255  6966 raft_consensus.cc:359] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.487243  6966 raft_consensus.cc:385] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.487622  6966 raft_consensus.cc:740] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.488489  6966 consensus_queue.cc:260] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.488915  6965 tablet_bootstrap.cc:492] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230: No bootstrap required, opened a new log
I20260709 12:29:59.488922  6964 tablet_bootstrap.cc:492] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: No bootstrap required, opened a new log
I20260709 12:29:59.489566  6965 ts_tablet_manager.cc:1403] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 12:29:59.489665  6964 ts_tablet_manager.cc:1403] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.007s
I20260709 12:29:59.490554  6966 ts_tablet_manager.cc:1434] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:29:59.491788  6966 tablet_bootstrap.cc:492] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: Bootstrap starting.
I20260709 12:29:59.492868  6965 raft_consensus.cc:359] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.493153  6964 raft_consensus.cc:359] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.493705  6965 raft_consensus.cc:385] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.493978  6964 raft_consensus.cc:385] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.494108  6965 raft_consensus.cc:740] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.494441  6964 raft_consensus.cc:740] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.495157  6965 consensus_queue.cc:260] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.495381  6964 consensus_queue.cc:260] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.497891  6965 ts_tablet_manager.cc:1434] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:59.497928  6964 ts_tablet_manager.cc:1434] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:59.499737  6966 tablet_bootstrap.cc:654] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.511344  6966 tablet_bootstrap.cc:492] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: No bootstrap required, opened a new log
I20260709 12:29:59.511911  6966 ts_tablet_manager.cc:1403] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 12:29:59.514389  6966 raft_consensus.cc:359] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.515002  6966 raft_consensus.cc:385] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.515259  6966 raft_consensus.cc:740] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.515967  6966 consensus_queue.cc:260] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.518095  6966 ts_tablet_manager.cc:1434] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:59.567814  6971 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.568517  6971 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.570830  6971 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:29:59.582762  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:29:59.583479  6924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952" is_pre_election: true
I20260709 12:29:59.583630  6775 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 0.
I20260709 12:29:59.584460  6924 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 0.
I20260709 12:29:59.585065  6816 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:29:59.586202  6971 raft_consensus.cc:2804] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.586587  6971 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.586953  6971 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.593051  6971 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.594772  6971 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 1 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:29:59.595654  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:29:59.596035  6924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:29:59.596284  6775 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.596764  6924 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.601783  6775 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 1.
I20260709 12:29:59.602954  6924 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 1.
I20260709 12:29:59.602972  6816 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:29:59.604166  6971 raft_consensus.cc:2804] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.605657  6971 raft_consensus.cc:697] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 LEADER]: Becoming Leader. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Running, Role: LEADER
I20260709 12:29:59.606415  6971 consensus_queue.cc:237] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.614270  6669 catalog_manager.cc:5697] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 reported cstate change: term changed from 0 to 1, leader changed from <none> to a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "a35f8f9fd6084cd080fe389b2d580230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.643784  6972 consensus_queue.cc:1048] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.657558  6977 consensus_queue.cc:1048] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:59.674124  6972 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.674679  6972 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.676621  6972 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.677551  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:29:59.678164  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230" is_pre_election: true
I20260709 12:29:59.678201  6775 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 0.
I20260709 12:29:59.678819  6850 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 0.
I20260709 12:29:59.679414  6890 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d; no voters: 
I20260709 12:29:59.680461  6972 raft_consensus.cc:2804] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.680768  6972 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.681043  6972 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.686245  6972 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.687947  6972 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.688999  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:29:59.688935  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:29:59.689802  6850 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.689858  6775 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.697366  6775 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 1.
I20260709 12:29:59.697366  6850 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 1.
I20260709 12:29:59.698901  6887 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:29:59.699726  6972 raft_consensus.cc:2804] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.700392  6972 raft_consensus.cc:697] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Becoming Leader. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Running, Role: LEADER
I20260709 12:29:59.701113  6972 consensus_queue.cc:237] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:29:59.709070  6669 catalog_manager.cc:5697] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 reported cstate change: term changed from 0 to 1, leader changed from <none> to 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "052c3ab125684cc0bef2003abeb9e952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:59.748685  6924 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f"
dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
mode: GRACEFUL
new_leader_uuid: "a2645100642f4b1e8559b708fe054d6d"
 from {username='slave'} at 127.0.0.1:42564
I20260709 12:29:59.749471  6924 raft_consensus.cc:606] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Received request to transfer leadership to TS a2645100642f4b1e8559b708fe054d6d
I20260709 12:29:59.768631  6850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4"
dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
mode: GRACEFUL
new_leader_uuid: "a2645100642f4b1e8559b708fe054d6d"
 from {username='slave'} at 127.0.0.1:53206
I20260709 12:29:59.769383  6850 raft_consensus.cc:606] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 LEADER]: Received request to transfer leadership to TS a2645100642f4b1e8559b708fe054d6d
I20260709 12:29:59.775591  6924 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d"
dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
mode: GRACEFUL
new_leader_uuid: "a2645100642f4b1e8559b708fe054d6d"
 from {username='slave'} at 127.0.0.1:42564
I20260709 12:29:59.776193  6924 raft_consensus.cc:606] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Received request to transfer leadership to TS a2645100642f4b1e8559b708fe054d6d
I20260709 12:29:59.880700  6972 consensus_queue.cc:1048] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.892515  6977 consensus_queue.cc:1048] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.901595  7001 raft_consensus.cc:993] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: : Instructing follower a2645100642f4b1e8559b708fe054d6d to start an election
I20260709 12:29:59.902153  6977 raft_consensus.cc:1081] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Signalling peer a2645100642f4b1e8559b708fe054d6d to start an election
I20260709 12:29:59.904294  6775 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f"
dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
 from {username='slave'} at 127.0.0.1:48946
I20260709 12:29:59.904794  6775 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:59.905118  6775 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.911249  6775 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:59.913888  6775 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 election: Requested vote from peers 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:29:59.925515  6924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:29:59.926102  6924 raft_consensus.cc:3055] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:59.926482  6924 raft_consensus.cc:740] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Running, Role: LEADER
I20260709 12:29:59.927340  6924 consensus_queue.cc:260] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:59.928609  6924 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.929131  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:29:59.929774  6850 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.936311  6924 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2645100642f4b1e8559b708fe054d6d in term 2.
I20260709 12:29:59.937225  6850 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2645100642f4b1e8559b708fe054d6d in term 2.
I20260709 12:29:59.937706  6741 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d; no voters: 
I20260709 12:29:59.939179  6970 raft_consensus.cc:2804] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:59.940570  6970 raft_consensus.cc:697] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Becoming Leader. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Running, Role: LEADER
I20260709 12:29:59.941457  6970 consensus_queue.cc:237] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:29:59.988998  6672 catalog_manager.cc:5697] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d reported cstate change: term changed from 1 to 2, leader changed from 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131) to a2645100642f4b1e8559b708fe054d6d (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2645100642f4b1e8559b708fe054d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:00.044194  6971 consensus_queue.cc:1048] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:00.062186  6971 consensus_queue.cc:1048] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:00.073721  6996 raft_consensus.cc:993] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: : Instructing follower a2645100642f4b1e8559b708fe054d6d to start an election
I20260709 12:30:00.074254  6971 raft_consensus.cc:1081] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 LEADER]: Signalling peer a2645100642f4b1e8559b708fe054d6d to start an election
I20260709 12:30:00.076107  6775 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4"
dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
 from {username='slave'} at 127.0.0.1:48958
I20260709 12:30:00.076823  6775 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:00.077317  6775 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.086649  6775 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:00.088893  6775 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 election: Requested vote from peers a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:00.090512  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:30:00.090683  6924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:30:00.091287  6850 raft_consensus.cc:3055] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:00.091399  6924 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.091707  6850 raft_consensus.cc:740] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Running, Role: LEADER
I20260709 12:30:00.092942  6850 consensus_queue.cc:260] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:00.094077  6850 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.099864  6924 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2645100642f4b1e8559b708fe054d6d in term 2.
I20260709 12:30:00.101420  6741 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d; no voters: 
I20260709 12:30:00.102525  6850 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2645100642f4b1e8559b708fe054d6d in term 2.
I20260709 12:30:00.102638  6970 raft_consensus.cc:2804] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:00.103320  6970 raft_consensus.cc:697] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Becoming Leader. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Running, Role: LEADER
I20260709 12:30:00.104507  6970 consensus_queue.cc:237] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:00.111094  6672 catalog_manager.cc:5697] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d reported cstate change: term changed from 1 to 2, leader changed from a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130) to a2645100642f4b1e8559b708fe054d6d (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2645100642f4b1e8559b708fe054d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:00.177114  6977 consensus_queue.cc:1048] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:00.185716  7001 raft_consensus.cc:993] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: : Instructing follower a2645100642f4b1e8559b708fe054d6d to start an election
I20260709 12:30:00.186535  6972 raft_consensus.cc:1081] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Signalling peer a2645100642f4b1e8559b708fe054d6d to start an election
I20260709 12:30:00.189545  6774 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d"
dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
 from {username='slave'} at 127.0.0.1:48946
I20260709 12:30:00.190361  6774 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:00.190905  6774 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.196398  7001 consensus_queue.cc:1048] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:00.201817  6774 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:00.208185  6774 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 election: Requested vote from peers a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:00.208488  6924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:30:00.208755  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:30:00.209309  6924 raft_consensus.cc:3055] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:00.209470  6850 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.209750  6924 raft_consensus.cc:740] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 052c3ab125684cc0bef2003abeb9e952, State: Running, Role: LEADER
I20260709 12:30:00.211033  6924 consensus_queue.cc:260] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:00.212709  6924 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.217594  6850 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2645100642f4b1e8559b708fe054d6d in term 2.
I20260709 12:30:00.218977  6738 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:00.219918  6970 raft_consensus.cc:2804] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:00.220259  6924 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2645100642f4b1e8559b708fe054d6d in term 2.
I20260709 12:30:00.220557  6970 raft_consensus.cc:697] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Becoming Leader. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Running, Role: LEADER
I20260709 12:30:00.221915  6970 consensus_queue.cc:237] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:00.228956  6671 catalog_manager.cc:5697] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d reported cstate change: term changed from 1 to 2, leader changed from 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131) to a2645100642f4b1e8559b708fe054d6d (127.24.71.129). New cstate: current_term: 2 leader_uuid: "a2645100642f4b1e8559b708fe054d6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:02.181623  7018 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:02.183606  7018 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.186436  7018 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.199237  7018 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:02.200362  7018 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:30:02.203553  7020 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:02.204358  7020 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.202998  7018 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.206305  7021 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:02.207423  7020 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.208043  7021 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:30:02.210924  7021 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:30:02.219022  7021 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:02.219815  7021 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.223307  7021 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:30:02.229682  6850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230" is_pre_election: true
I20260709 12:30:02.230487  6924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952" is_pre_election: true
I20260709 12:30:02.230494  6850 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 2.
I20260709 12:30:02.230794  6923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952" is_pre_election: true
I20260709 12:30:02.231374  6924 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 2.
I20260709 12:30:02.231563  6849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230" is_pre_election: true
I20260709 12:30:02.231752  6923 raft_consensus.cc:2468] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 2.
I20260709 12:30:02.232447  6849 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052c3ab125684cc0bef2003abeb9e952 in term 2.
I20260709 12:30:02.232849  6887 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:02.233757  7021 raft_consensus.cc:2804] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:02.234237  6816 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:02.234401  7021 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:02.234387  6887 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:02.235136  7021 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.235299  6816 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:02.236459  7018 raft_consensus.cc:2804] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:02.236842  7018 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:02.237330  7018 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.236723  6922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952" is_pre_election: true
I20260709 12:30:02.243643  6922 raft_consensus.cc:2468] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 2.
I20260709 12:30:02.246619  7021 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:30:02.247560  7018 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:30:02.248688  7020 raft_consensus.cc:2804] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:02.250378  7021 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:30:02.252480 24862 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:30:02.253834  6816 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:02.254307  6922 raft_consensus.cc:1240] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Rejecting Update request from peer a2645100642f4b1e8559b708fe054d6d for earlier term 2. Current term is 3. Ops: []
I20260709 12:30:02.255465  7020 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:02.256953  7020 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.257800  7019 consensus_queue.cc:1059] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:02.259008  7019 raft_consensus.cc:3055] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:02.259236 24862 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:30:02.259353  7019 raft_consensus.cc:740] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Running, Role: LEADER
I20260709 12:30:02.260416 24862 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:30:02.260927  7019 consensus_queue.cc:260] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:30:02.262311  7019 raft_consensus.cc:3060] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.256954  7018 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.254660  7024 raft_consensus.cc:2804] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:02.266196  7024 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:02.266634  7024 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.267705  6922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:30:02.268733  6922 raft_consensus.cc:2393] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 in current term 3: Already voted for candidate 052c3ab125684cc0bef2003abeb9e952 in this term.
I20260709 12:30:02.270393  6849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:30:02.271433  6849 raft_consensus.cc:2393] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 052c3ab125684cc0bef2003abeb9e952 in current term 3: Already voted for candidate a35f8f9fd6084cd080fe389b2d580230 in this term.
I20260709 12:30:02.276538  7018 raft_consensus.cc:2804] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:02.277182  7024 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.280210  7024 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
I20260709 12:30:02.277812  7018 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:02.282431  6922 raft_consensus.cc:1275] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Refusing update from remote peer a2645100642f4b1e8559b708fe054d6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:02.283560  7018 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.282161  6849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230"
I20260709 12:30:02.283975  7020 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.280731  7022 raft_consensus.cc:493] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 052c3ab125684cc0bef2003abeb9e952)
I20260709 12:30:02.285168  7022 raft_consensus.cc:515] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.286868  6922 raft_consensus.cc:1240] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Rejecting Update request from peer a2645100642f4b1e8559b708fe054d6d for earlier term 2. Current term is 3. Ops: []
I20260709 12:30:02.289306  6922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3c162a99e74b3f9c37c993f5b6e2e5" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952" is_pre_election: true
I20260709 12:30:02.291558  7022 leader_election.cc:290] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.292893  7022 consensus_queue.cc:1059] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:02.291545  7019 consensus_queue.cc:1048] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:02.295885  7018 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.297811  7019 raft_consensus.cc:3055] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:02.297857  6849 raft_consensus.cc:2393] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 052c3ab125684cc0bef2003abeb9e952 in current term 3: Already voted for candidate a35f8f9fd6084cd080fe389b2d580230 in this term.
I20260709 12:30:02.298331  7019 raft_consensus.cc:740] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Running, Role: LEADER
I20260709 12:30:02.302953  7020 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.303385  6922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:30:02.299500  7019 consensus_queue.cc:260] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.304448  6922 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.308913  6774 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4"
dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
mode: GRACEFUL
new_leader_uuid: "a35f8f9fd6084cd080fe389b2d580230"
 from {username='slave'} at 127.0.0.1:53314
I20260709 12:30:02.309916  6774 raft_consensus.cc:606] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Received request to transfer leadership to TS a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:02.311976 24862 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:30:02.314932  6923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "052c3ab125684cc0bef2003abeb9e952"
I20260709 12:30:02.316419  6923 raft_consensus.cc:2393] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 in current term 3: Already voted for candidate 052c3ab125684cc0bef2003abeb9e952 in this term.
I20260709 12:30:02.316118  6922 raft_consensus.cc:2468] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a35f8f9fd6084cd080fe389b2d580230 in term 3.
I20260709 12:30:02.321998  7018 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 election: Requested vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), 052c3ab125684cc0bef2003abeb9e952 (127.24.71.131:40349)
I20260709 12:30:02.324307  6816 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230; no voters: 
I20260709 12:30:02.326077  7018 raft_consensus.cc:2804] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:02.326382  7019 raft_consensus.cc:3060] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.327078  7018 raft_consensus.cc:697] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 LEADER]: Becoming Leader. State: Replica: a35f8f9fd6084cd080fe389b2d580230, State: Running, Role: LEADER
I20260709 12:30:02.328521  7018 consensus_queue.cc:237] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.331696  6923 raft_consensus.cc:1240] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Rejecting Update request from peer a2645100642f4b1e8559b708fe054d6d for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:30:02.336858  7022 consensus_queue.cc:1059] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:02.337814  7022 raft_consensus.cc:3055] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:02.338141  7022 raft_consensus.cc:740] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2645100642f4b1e8559b708fe054d6d, State: Running, Role: LEADER
I20260709 12:30:02.339401  7022 consensus_queue.cc:260] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:02.340855  7022 raft_consensus.cc:3060] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:02.353004  6849 raft_consensus.cc:1240] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Rejecting Update request from peer a2645100642f4b1e8559b708fe054d6d for earlier term 2. Current term is 3. Ops: []
I20260709 12:30:02.353521  6774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:30:02.354123  6848 raft_consensus.cc:1240] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 LEADER]: Rejecting Update request from peer a2645100642f4b1e8559b708fe054d6d for earlier term 2. Current term is 3. Ops: []
I20260709 12:30:02.354624  6774 raft_consensus.cc:2410] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.354722  6847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3c162a99e74b3f9c37c993f5b6e2e5" candidate_uuid: "a2645100642f4b1e8559b708fe054d6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a35f8f9fd6084cd080fe389b2d580230" is_pre_election: true
I20260709 12:30:02.356366  6850 raft_consensus.cc:1240] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Rejecting Update request from peer a2645100642f4b1e8559b708fe054d6d for earlier term 2. Current term is 3. Ops: []
I20260709 12:30:02.356766  6774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:30:02.357848  6774 raft_consensus.cc:2410] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.359745  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:30:02.360843  6775 raft_consensus.cc:2410] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.361984  6772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:30:02.361856  6672 catalog_manager.cc:5697] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 reported cstate change: term changed from 2 to 3, leader changed from a2645100642f4b1e8559b708fe054d6d (127.24.71.129) to a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130). New cstate: current_term: 3 leader_uuid: "a35f8f9fd6084cd080fe389b2d580230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:02.363341  6772 raft_consensus.cc:2410] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.363574  6771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:30:02.362092  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5684765fde24d788681d55f4a194ed4" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:30:02.364672  6771 raft_consensus.cc:2410] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 052c3ab125684cc0bef2003abeb9e952 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.365010  6773 raft_consensus.cc:2410] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.366317  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:30:02.366703  6771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a20fb5560e945a9b04da2be383bad5f" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:30:02.367466  6773 raft_consensus.cc:2410] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 052c3ab125684cc0bef2003abeb9e952 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.367738  6771 raft_consensus.cc:2410] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 052c3ab125684cc0bef2003abeb9e952 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.368371  6774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "a35f8f9fd6084cd080fe389b2d580230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d" is_pre_election: true
I20260709 12:30:02.369514  6774 raft_consensus.cc:2410] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a35f8f9fd6084cd080fe389b2d580230 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.369436  6771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a36daab7bbb4b428d3e32f0a633947d" candidate_uuid: "052c3ab125684cc0bef2003abeb9e952" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2645100642f4b1e8559b708fe054d6d"
I20260709 12:30:02.370596  6771 raft_consensus.cc:2410] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 052c3ab125684cc0bef2003abeb9e952 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:30:02.373155  6816 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a35f8f9fd6084cd080fe389b2d580230; no voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d
I20260709 12:30:02.374635  6816 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a35f8f9fd6084cd080fe389b2d580230; no voters: 052c3ab125684cc0bef2003abeb9e952, a2645100642f4b1e8559b708fe054d6d
I20260709 12:30:02.376217  6890 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952; no voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:02.376415  7020 raft_consensus.cc:2749] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:30:02.377619  6738 leader_election.cc:304] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2645100642f4b1e8559b708fe054d6d; no voters: 052c3ab125684cc0bef2003abeb9e952, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:02.377811  6890 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952; no voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:02.375519  7018 raft_consensus.cc:2749] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:30:02.378537  7022 raft_consensus.cc:2749] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:02.378963  7024 raft_consensus.cc:2749] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:30:02.378679  7021 raft_consensus.cc:2749] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:30:02.394105 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
I20260709 12:30:02.419984 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:02.420986 24862 tablet_replica.cc:333] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: stopping tablet replica
I20260709 12:30:02.421785 24862 raft_consensus.cc:2243] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.422252 24862 pending_rounds.cc:70] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: Trying to abort 1 pending ops.
I20260709 12:30:02.422415 24862 pending_rounds.cc:77] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626419635945472 op_type: NO_OP noop_request { }
I20260709 12:30:02.422673 24862 raft_consensus.cc:2889] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:02.422911 24862 raft_consensus.cc:2272] T a5684765fde24d788681d55f4a194ed4 P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.425375 24862 tablet_replica.cc:333] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: stopping tablet replica
I20260709 12:30:02.426098 24862 raft_consensus.cc:2243] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.426468 24862 pending_rounds.cc:70] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: Trying to abort 1 pending ops.
I20260709 12:30:02.426627 24862 pending_rounds.cc:77] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626420118433792 op_type: NO_OP noop_request { }
I20260709 12:30:02.426857 24862 raft_consensus.cc:2889] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:02.427075 24862 raft_consensus.cc:2272] T 4a36daab7bbb4b428d3e32f0a633947d P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.429430 24862 tablet_replica.cc:333] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d: stopping tablet replica
I20260709 12:30:02.429949 24862 raft_consensus.cc:2243] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.430343 24862 raft_consensus.cc:2272] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a2645100642f4b1e8559b708fe054d6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.432744 24862 tablet_replica.cc:333] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: stopping tablet replica
I20260709 12:30:02.433226 24862 raft_consensus.cc:2243] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.433596 24862 pending_rounds.cc:70] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: Trying to abort 1 pending ops.
I20260709 12:30:02.433739 24862 pending_rounds.cc:77] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626418972618752 op_type: NO_OP noop_request { }
I20260709 12:30:02.433984 24862 raft_consensus.cc:2889] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:02.434206 24862 raft_consensus.cc:2272] T 3a20fb5560e945a9b04da2be383bad5f P a2645100642f4b1e8559b708fe054d6d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.463261 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
I20260709 12:30:02.482972 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
I20260709 12:30:02.507521 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:02.508363 24862 tablet_replica.cc:333] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: stopping tablet replica
I20260709 12:30:02.509070 24862 raft_consensus.cc:2243] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:02.509728 24862 pending_rounds.cc:70] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: Trying to abort 1 pending ops.
I20260709 12:30:02.509948 24862 pending_rounds.cc:77] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230: Aborting op as it isn't in flight: id { term: 3 index: 2 } timestamp: 7305626428752773120 op_type: NO_OP noop_request { }
I20260709 12:30:02.510231 24862 raft_consensus.cc:2889] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:02.510510 24862 raft_consensus.cc:2272] T a5684765fde24d788681d55f4a194ed4 P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.513679 24862 tablet_replica.cc:333] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230: stopping tablet replica
I20260709 12:30:02.514325 24862 raft_consensus.cc:2243] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.514868 24862 raft_consensus.cc:2272] T 4a36daab7bbb4b428d3e32f0a633947d P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.517623 24862 tablet_replica.cc:333] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230: stopping tablet replica
I20260709 12:30:02.518219 24862 raft_consensus.cc:2243] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.518750 24862 raft_consensus.cc:2272] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P a35f8f9fd6084cd080fe389b2d580230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:02.521481 24862 tablet_replica.cc:333] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230: stopping tablet replica
I20260709 12:30:02.522056 24862 raft_consensus.cc:2243] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:02.522578 24862 raft_consensus.cc:2272] T 3a20fb5560e945a9b04da2be383bad5f P a35f8f9fd6084cd080fe389b2d580230 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:02.669556  6890 consensus_peers.cc:597] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 -> Peer a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001): Couldn't send request to peer a2645100642f4b1e8559b708fe054d6d. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:38001: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:02.694854  6887 consensus_peers.cc:597] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 -> Peer a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033): Couldn't send request to peer a35f8f9fd6084cd080fe389b2d580230. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:02.805416  6871 debug-util.cc:398] Leaking SignalData structure 0x7b08001a82a0 after lost signal to thread 24865
W20260709 12:30:02.806874  6871 debug-util.cc:398] Leaking SignalData structure 0x7b080029c800 after lost signal to thread 6706
W20260709 12:30:02.809660  6871 debug-util.cc:398] Leaking SignalData structure 0x7b08003cda60 after lost signal to thread 6948
W20260709 12:30:03.254621  6887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260709 12:30:03.539098 24862 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 6871)
I20260709 12:30:03.966441  7039 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:03.967017  7039 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:03.968999  7039 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
W20260709 12:30:03.974443  6890 leader_election.cc:336] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001): Network error: Client connection negotiation failed: client connection to 127.24.71.129:38001: connect: Connection refused (error 111)
W20260709 12:30:03.976897  6887 leader_election.cc:336] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033): Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111)
I20260709 12:30:03.977283  6887 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952; no voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:03.977933  7039 raft_consensus.cc:2749] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:30:04.259477  7039 raft_consensus.cc:493] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:04.260000  7039 raft_consensus.cc:515] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } }
I20260709 12:30:04.262364  7039 leader_election.cc:290] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
W20260709 12:30:04.267688  6887 leader_election.cc:336] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033): Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111)
W20260709 12:30:04.268260  6890 leader_election.cc:336] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001): Network error: Client connection negotiation failed: client connection to 127.24.71.129:38001: connect: Connection refused (error 111)
I20260709 12:30:04.268779  6890 leader_election.cc:304] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952; no voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:04.269695  7039 raft_consensus.cc:2749] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:30:04.539755 24862 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 6871)
I20260709 12:30:04.595953  7039 raft_consensus.cc:493] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:04.596619  7039 raft_consensus.cc:515] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:04.598361  7039 leader_election.cc:290] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
W20260709 12:30:04.602483  6890 leader_election.cc:336] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001): Network error: Client connection negotiation failed: client connection to 127.24.71.129:38001: connect: Connection refused (error 111)
W20260709 12:30:04.603814  6887 leader_election.cc:336] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033): Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111)
I20260709 12:30:04.604264  6887 leader_election.cc:304] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952; no voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:04.604969  7039 raft_consensus.cc:2749] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:30:04.931634  6890 consensus_peers.cc:597] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 -> Peer a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001): Couldn't send request to peer a2645100642f4b1e8559b708fe054d6d. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:38001: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:05.242136  6887 consensus_peers.cc:597] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 -> Peer a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033): Couldn't send request to peer a35f8f9fd6084cd080fe389b2d580230. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:05.540549 24862 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 6871)
I20260709 12:30:05.593317  7039 raft_consensus.cc:493] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:05.593907  7039 raft_consensus.cc:515] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2645100642f4b1e8559b708fe054d6d" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38001 } } peers { permanent_uuid: "a35f8f9fd6084cd080fe389b2d580230" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 34033 } } peers { permanent_uuid: "052c3ab125684cc0bef2003abeb9e952" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 40349 } }
I20260709 12:30:05.595914  7039 leader_election.cc:290] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001), a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033)
W20260709 12:30:05.600585  6890 leader_election.cc:336] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2645100642f4b1e8559b708fe054d6d (127.24.71.129:38001): Network error: Client connection negotiation failed: client connection to 127.24.71.129:38001: connect: Connection refused (error 111)
W20260709 12:30:05.601475  6887 leader_election.cc:336] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a35f8f9fd6084cd080fe389b2d580230 (127.24.71.130:34033): Network error: Client connection negotiation failed: client connection to 127.24.71.130:34033: connect: Connection refused (error 111)
I20260709 12:30:05.601871  6887 leader_election.cc:304] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 052c3ab125684cc0bef2003abeb9e952; no voters: a2645100642f4b1e8559b708fe054d6d, a35f8f9fd6084cd080fe389b2d580230
I20260709 12:30:05.602540  7039 raft_consensus.cc:2749] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:30:05.891353 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
I20260709 12:30:05.910583 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
I20260709 12:30:05.934801 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:05.935648 24862 tablet_replica.cc:333] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952: stopping tablet replica
I20260709 12:30:05.936414 24862 raft_consensus.cc:2243] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.936951 24862 raft_consensus.cc:2272] T a5684765fde24d788681d55f4a194ed4 P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.939292 24862 tablet_replica.cc:333] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952: stopping tablet replica
I20260709 12:30:05.939826 24862 raft_consensus.cc:2243] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.940326 24862 raft_consensus.cc:2272] T 4a36daab7bbb4b428d3e32f0a633947d P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.942530 24862 tablet_replica.cc:333] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952: stopping tablet replica
I20260709 12:30:05.943027 24862 raft_consensus.cc:2243] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:05.943873 24862 raft_consensus.cc:2272] T 2f3c162a99e74b3f9c37c993f5b6e2e5 P 052c3ab125684cc0bef2003abeb9e952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.947110 24862 tablet_replica.cc:333] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952: stopping tablet replica
I20260709 12:30:05.947731 24862 raft_consensus.cc:2243] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:05.948199 24862 raft_consensus.cc:2272] T 3a20fb5560e945a9b04da2be383bad5f P 052c3ab125684cc0bef2003abeb9e952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:05.973860 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:30:05.992314 24862 master.cc:562] Master@127.24.71.190:46387 shutting down...
I20260709 12:30:06.013255 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:06.013875 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:06.014226 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1e64307aad2e4d33947f218dbae22d8a: stopping tablet replica
I20260709 12:30:06.034334 24862 master.cc:584] Master@127.24.71.190:46387 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (7903 ms)
[----------] 6 tests from LeaderRebalancerTest (473542 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:30:06.077980 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:42049
I20260709 12:30:06.079154 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:06.084769  7045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.084806  7046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.087942  7048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:06.087997 24862 server_base.cc:1061] running on GCE node
I20260709 12:30:06.089460 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:06.089640 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:06.089848 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600206089812 us; error 0 us; skew 500 ppm
I20260709 12:30:06.090401 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:06.092998 24862 webserver.cc:533] Webserver started at http://127.24.71.190:35309/ using document root <none> and password file <none>
I20260709 12:30:06.093540 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:06.093743 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:06.094044 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:06.095592 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "04e6ae9d4b584aa6937959e5840cc509"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.100435 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:30:06.104115  7053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.105057 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:06.105464 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "04e6ae9d4b584aa6937959e5840cc509"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.105808 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:06.129487 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:06.130785 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:06.173702 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:42049
I20260709 12:30:06.173827  7104 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:42049 every 8 connection(s)
I20260709 12:30:06.177729  7105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.190081  7105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509: Bootstrap starting.
I20260709 12:30:06.195437  7105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:06.200641  7105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509: No bootstrap required, opened a new log
I20260709 12:30:06.202983  7105 raft_consensus.cc:359] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e6ae9d4b584aa6937959e5840cc509" member_type: VOTER }
I20260709 12:30:06.203434  7105 raft_consensus.cc:385] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:06.203660  7105 raft_consensus.cc:740] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e6ae9d4b584aa6937959e5840cc509, State: Initialized, Role: FOLLOWER
I20260709 12:30:06.204377  7105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e6ae9d4b584aa6937959e5840cc509" member_type: VOTER }
I20260709 12:30:06.204843  7105 raft_consensus.cc:399] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:06.205051  7105 raft_consensus.cc:493] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:06.205355  7105 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:06.210631  7105 raft_consensus.cc:515] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e6ae9d4b584aa6937959e5840cc509" member_type: VOTER }
I20260709 12:30:06.211182  7105 leader_election.cc:304] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e6ae9d4b584aa6937959e5840cc509; no voters: 
I20260709 12:30:06.212457  7105 leader_election.cc:290] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:06.212837  7108 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:06.214462  7108 raft_consensus.cc:697] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 1 LEADER]: Becoming Leader. State: Replica: 04e6ae9d4b584aa6937959e5840cc509, State: Running, Role: LEADER
I20260709 12:30:06.215159  7108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e6ae9d4b584aa6937959e5840cc509" member_type: VOTER }
I20260709 12:30:06.215451  7105 sys_catalog.cc:565] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:06.221251  7110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e6ae9d4b584aa6937959e5840cc509. Latest consensus state: current_term: 1 leader_uuid: "04e6ae9d4b584aa6937959e5840cc509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e6ae9d4b584aa6937959e5840cc509" member_type: VOTER } }
I20260709 12:30:06.222280  7109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e6ae9d4b584aa6937959e5840cc509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e6ae9d4b584aa6937959e5840cc509" member_type: VOTER } }
I20260709 12:30:06.223166  7109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:06.224531  7110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:06.229244  7118 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:06.234807  7118 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:06.236153 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:06.245349  7118 catalog_manager.cc:1383] Generated new cluster ID: 83f8c1d215a74a258b8d3f13d921ae67
I20260709 12:30:06.245729  7118 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:06.257839  7118 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:06.259200  7118 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:06.274420  7118 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509: Generated new TSK 0
I20260709 12:30:06.275146  7118 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:06.304719 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:06.312206  7126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.313333  7127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.316099  7129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:06.317430 24862 server_base.cc:1061] running on GCE node
I20260709 12:30:06.318470 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:06.318714 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:06.318848 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600206318836 us; error 0 us; skew 500 ppm
I20260709 12:30:06.319350 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:06.321952 24862 webserver.cc:533] Webserver started at http://127.24.71.129:43725/ using document root <none> and password file <none>
I20260709 12:30:06.322422 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:06.322580 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:06.322788 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:06.323904 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "255aee4d52c8404fbd9da5f2aa03e346"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.328895 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:30:06.332871  7134 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.333906 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:30:06.334267 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "255aee4d52c8404fbd9da5f2aa03e346"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.334589 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:06.349911 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:06.351226 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:06.352787 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:06.355250 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:06.355487 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.355742 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:06.355940 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.402622 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:33067
I20260709 12:30:06.402729  7196 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:33067 every 8 connection(s)
I20260709 12:30:06.408030 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:06.416741  7201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.419436  7202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.420188  7204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:06.421677 24862 server_base.cc:1061] running on GCE node
I20260709 12:30:06.422819  7197 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42049
I20260709 12:30:06.423260  7197 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:06.423573 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:06.423858 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:06.424043 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600206424027 us; error 0 us; skew 500 ppm
I20260709 12:30:06.424136  7197 heartbeater.cc:507] Master 127.24.71.190:42049 requested a full tablet report, sending...
I20260709 12:30:06.424724 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:06.426849  7070 ts_manager.cc:194] Registered new tserver with Master: 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067)
I20260709 12:30:06.427753 24862 webserver.cc:533] Webserver started at http://127.24.71.130:42301/ using document root <none> and password file <none>
I20260709 12:30:06.428473 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:06.428694 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:06.428978 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:06.429008  7070 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37698
I20260709 12:30:06.430370 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "4124877098414253a130e4e140806788"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.435484 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:30:06.439393  7209 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.440616 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:06.440937 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "4124877098414253a130e4e140806788"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.441227 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:06.463958 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:06.465250 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:06.466810 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:06.469164 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:06.469393 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.469640 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:06.469823 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.514458 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:36863
I20260709 12:30:06.514585  7271 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:36863 every 8 connection(s)
I20260709 12:30:06.519572 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:06.527776  7275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:06.528432  7276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:06.533113 24862 server_base.cc:1061] running on GCE node
W20260709 12:30:06.533726  7278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:06.534009  7272 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42049
I20260709 12:30:06.534581  7272 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:06.535068 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:06.535398 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:06.535531  7272 heartbeater.cc:507] Master 127.24.71.190:42049 requested a full tablet report, sending...
I20260709 12:30:06.535622 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600206535600 us; error 0 us; skew 500 ppm
I20260709 12:30:06.536748 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:06.538048  7070 ts_manager.cc:194] Registered new tserver with Master: 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:06.539554  7070 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37704
I20260709 12:30:06.542310 24862 webserver.cc:533] Webserver started at http://127.24.71.131:36171/ using document root <none> and password file <none>
I20260709 12:30:06.543011 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:06.543295 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:06.543612 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:06.545003 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "273e786494a845ff87c99139442c371d"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.549728 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:30:06.553730  7283 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.554759 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:30:06.555040 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "273e786494a845ff87c99139442c371d"
format_stamp: "Formatted at 2026-07-09 12:30:06 on dist-test-slave-3zgk"
I20260709 12:30:06.555291 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:06.573514 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:06.574766 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:06.576185 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:06.578513 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:06.578706 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.579048 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:06.579191 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:06.624089 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:41695
I20260709 12:30:06.624193  7345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:41695 every 8 connection(s)
I20260709 12:30:06.639968  7346 heartbeater.cc:344] Connected to a master server at 127.24.71.190:42049
I20260709 12:30:06.640517  7346 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:06.641304  7346 heartbeater.cc:507] Master 127.24.71.190:42049 requested a full tablet report, sending...
I20260709 12:30:06.643474  7070 ts_manager.cc:194] Registered new tserver with Master: 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:06.643859 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016657056s
I20260709 12:30:06.645453  7070 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37710
I20260709 12:30:06.675348  7070 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37726:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:06.678329  7070 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:06.746232  7311 tablet_service.cc:1511] Processing CreateTablet for tablet 70bd6da3b3924e2896be369b7e6a253c (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:06.747761  7311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70bd6da3b3924e2896be369b7e6a253c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.753021  7310 tablet_service.cc:1511] Processing CreateTablet for tablet 64e2d5ba97c54e5a9130875f239c8833 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:06.754710  7310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e2d5ba97c54e5a9130875f239c8833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.757831  7309 tablet_service.cc:1511] Processing CreateTablet for tablet 9f9b66df1b924fa0831f6be36ed78a60 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:06.764710  7309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9b66df1b924fa0831f6be36ed78a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.769515  7235 tablet_service.cc:1511] Processing CreateTablet for tablet 9f9b66df1b924fa0831f6be36ed78a60 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:06.769840  7233 tablet_service.cc:1511] Processing CreateTablet for tablet 5e3992fbed914c4787d3089d7932f264 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:06.770267  7308 tablet_service.cc:1511] Processing CreateTablet for tablet 239fc23a25f8480b97f6b032d88d9105 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:06.771752  7234 tablet_service.cc:1511] Processing CreateTablet for tablet 239fc23a25f8480b97f6b032d88d9105 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:06.773623  7236 tablet_service.cc:1511] Processing CreateTablet for tablet 64e2d5ba97c54e5a9130875f239c8833 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:06.774719  7237 tablet_service.cc:1511] Processing CreateTablet for tablet 70bd6da3b3924e2896be369b7e6a253c (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:06.774818  7307 tablet_service.cc:1511] Processing CreateTablet for tablet 5e3992fbed914c4787d3089d7932f264 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:06.778381  7232 tablet_service.cc:1511] Processing CreateTablet for tablet ca42f5a8e6e14af896744c41a9c2d65f (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:06.779876  7306 tablet_service.cc:1511] Processing CreateTablet for tablet ca42f5a8e6e14af896744c41a9c2d65f (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:06.771817  7308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239fc23a25f8480b97f6b032d88d9105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.782251  7235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9b66df1b924fa0831f6be36ed78a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.798588  7233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e3992fbed914c4787d3089d7932f264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.787087  7160 tablet_service.cc:1511] Processing CreateTablet for tablet 9f9b66df1b924fa0831f6be36ed78a60 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:06.799276  7232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca42f5a8e6e14af896744c41a9c2d65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.799849  7160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9b66df1b924fa0831f6be36ed78a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.799938  7234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239fc23a25f8480b97f6b032d88d9105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.800736  7236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e2d5ba97c54e5a9130875f239c8833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.789225  7231 tablet_service.cc:1511] Processing CreateTablet for tablet e384731a794542d3aa4feec313f2a98f (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:06.803833  7231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e384731a794542d3aa4feec313f2a98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.790828  7157 tablet_service.cc:1511] Processing CreateTablet for tablet ca42f5a8e6e14af896744c41a9c2d65f (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:06.805467  7157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca42f5a8e6e14af896744c41a9c2d65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.792227  7162 tablet_service.cc:1511] Processing CreateTablet for tablet 70bd6da3b3924e2896be369b7e6a253c (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:06.811060  7162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70bd6da3b3924e2896be369b7e6a253c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.793499  7161 tablet_service.cc:1511] Processing CreateTablet for tablet 64e2d5ba97c54e5a9130875f239c8833 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:06.813549  7161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e2d5ba97c54e5a9130875f239c8833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.794807  7159 tablet_service.cc:1511] Processing CreateTablet for tablet 239fc23a25f8480b97f6b032d88d9105 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:06.817572  7159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239fc23a25f8480b97f6b032d88d9105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.795961  7156 tablet_service.cc:1511] Processing CreateTablet for tablet e384731a794542d3aa4feec313f2a98f (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:06.820029  7156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e384731a794542d3aa4feec313f2a98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.785120  7158 tablet_service.cc:1511] Processing CreateTablet for tablet 5e3992fbed914c4787d3089d7932f264 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:06.797763  7306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca42f5a8e6e14af896744c41a9c2d65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.822340  7158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e3992fbed914c4787d3089d7932f264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.822765  7307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e3992fbed914c4787d3089d7932f264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.824396  7237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70bd6da3b3924e2896be369b7e6a253c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.863739  7361 tablet_bootstrap.cc:492] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:06.863473  7305 tablet_service.cc:1511] Processing CreateTablet for tablet e384731a794542d3aa4feec313f2a98f (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:06.865092  7305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e384731a794542d3aa4feec313f2a98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.908821  7155 tablet_service.cc:1511] Processing CreateTablet for tablet aa90fd87ec3e44068275986fa849c02e (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:06.910189  7361 tablet_bootstrap.cc:654] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:06.910342  7155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa90fd87ec3e44068275986fa849c02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.917332  7230 tablet_service.cc:1511] Processing CreateTablet for tablet aa90fd87ec3e44068275986fa849c02e (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:06.918838  7230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa90fd87ec3e44068275986fa849c02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.927821  7309 tablet_service.cc:1511] Processing CreateTablet for tablet aa90fd87ec3e44068275986fa849c02e (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:06.929332  7309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa90fd87ec3e44068275986fa849c02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.940882  7363 tablet_bootstrap.cc:492] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:06.960425  7363 tablet_bootstrap.cc:654] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:06.974351  7156 tablet_service.cc:1511] Processing CreateTablet for tablet 701e4ba57f074d4b97828ec0d9c6a705 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:06.975816  7156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701e4ba57f074d4b97828ec0d9c6a705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.977068  7229 tablet_service.cc:1511] Processing CreateTablet for tablet 701e4ba57f074d4b97828ec0d9c6a705 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:06.978466  7229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701e4ba57f074d4b97828ec0d9c6a705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:06.998428  7363 tablet_bootstrap.cc:492] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:06.999155  7363 ts_tablet_manager.cc:1403] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.003s
I20260709 12:30:07.002696  7363 raft_consensus.cc:359] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.003885  7363 raft_consensus.cc:385] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.004308  7363 raft_consensus.cc:740] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.005357  7365 tablet_bootstrap.cc:492] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.005187  7363 consensus_queue.cc:260] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.006340  7308 tablet_service.cc:1511] Processing CreateTablet for tablet 701e4ba57f074d4b97828ec0d9c6a705 (DEFAULT_TABLE table=test-workload [id=c9f845a238b34249bfa7d39c170312f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:07.008064  7308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701e4ba57f074d4b97828ec0d9c6a705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:07.006839  7361 tablet_bootstrap.cc:492] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.024063  7361 ts_tablet_manager.cc:1403] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.161s	user 0.021s	sys 0.019s
I20260709 12:30:07.037376  7197 heartbeater.cc:499] Master 127.24.71.190:42049 was elected leader, sending a full tablet report...
I20260709 12:30:07.041184  7365 tablet_bootstrap.cc:654] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.044649  7361 raft_consensus.cc:359] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.042697  7363 ts_tablet_manager.cc:1434] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.043s	user 0.013s	sys 0.021s
I20260709 12:30:07.048935  7363 tablet_bootstrap.cc:492] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.049379  7365 tablet_bootstrap.cc:492] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.050079  7365 ts_tablet_manager.cc:1403] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.005s
I20260709 12:30:07.052404  7361 raft_consensus.cc:385] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.053416  7361 raft_consensus.cc:740] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.053083  7365 raft_consensus.cc:359] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.053982  7365 raft_consensus.cc:385] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.054356  7365 raft_consensus.cc:740] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.054535  7361 consensus_queue.cc:260] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.055238  7365 consensus_queue.cc:260] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.055907  7363 tablet_bootstrap.cc:654] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.063458  7346 heartbeater.cc:499] Master 127.24.71.190:42049 was elected leader, sending a full tablet report...
I20260709 12:30:07.065140  7361 ts_tablet_manager.cc:1434] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.007s
I20260709 12:30:07.066658  7361 tablet_bootstrap.cc:492] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.069463  7272 heartbeater.cc:499] Master 127.24.71.190:42049 was elected leader, sending a full tablet report...
I20260709 12:30:07.070099  7365 ts_tablet_manager.cc:1434] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.011s
I20260709 12:30:07.071220  7365 tablet_bootstrap.cc:492] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.073982  7363 tablet_bootstrap.cc:492] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.074649  7363 ts_tablet_manager.cc:1403] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.003s
I20260709 12:30:07.077888  7363 raft_consensus.cc:359] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.078424  7365 tablet_bootstrap.cc:654] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.078877  7363 raft_consensus.cc:385] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.079334  7363 raft_consensus.cc:740] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.080647  7363 consensus_queue.cc:260] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.083215  7363 ts_tablet_manager.cc:1434] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:07.084452  7363 tablet_bootstrap.cc:492] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.084810  7361 tablet_bootstrap.cc:654] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.092491  7363 tablet_bootstrap.cc:654] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.092878  7365 tablet_bootstrap.cc:492] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.093076  7361 tablet_bootstrap.cc:492] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.093565  7365 ts_tablet_manager.cc:1403] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 12:30:07.093565  7361 ts_tablet_manager.cc:1403] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.004s
I20260709 12:30:07.098904  7365 raft_consensus.cc:359] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.099769  7365 raft_consensus.cc:385] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.100142  7365 raft_consensus.cc:740] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.101053  7361 raft_consensus.cc:359] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.101094  7365 consensus_queue.cc:260] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.102128  7361 raft_consensus.cc:385] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.102826  7361 raft_consensus.cc:740] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.103814  7365 ts_tablet_manager.cc:1434] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:30:07.103746  7361 consensus_queue.cc:260] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.105067  7365 tablet_bootstrap.cc:492] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.106395  7361 ts_tablet_manager.cc:1434] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 12:30:07.107601  7361 tablet_bootstrap.cc:492] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.107748  7363 tablet_bootstrap.cc:492] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.108517  7363 ts_tablet_manager.cc:1403] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.011s
I20260709 12:30:07.112061  7363 raft_consensus.cc:359] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.113162  7363 raft_consensus.cc:385] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.113349  7365 tablet_bootstrap.cc:654] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.113600  7363 raft_consensus.cc:740] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.114820  7363 consensus_queue.cc:260] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.115461  7361 tablet_bootstrap.cc:654] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.121193  7363 ts_tablet_manager.cc:1434] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:30:07.122531  7363 tablet_bootstrap.cc:492] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.125710  7365 tablet_bootstrap.cc:492] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.126374  7365 ts_tablet_manager.cc:1403] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:30:07.126622  7361 tablet_bootstrap.cc:492] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.127243  7361 ts_tablet_manager.cc:1403] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.020s	user 0.001s	sys 0.013s
I20260709 12:30:07.129841  7365 raft_consensus.cc:359] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.130308  7361 raft_consensus.cc:359] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.130766  7365 raft_consensus.cc:385] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.131088  7361 raft_consensus.cc:385] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.131209  7365 raft_consensus.cc:740] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.131580  7363 tablet_bootstrap.cc:654] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.132264  7365 consensus_queue.cc:260] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.131531  7361 raft_consensus.cc:740] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.135241  7365 ts_tablet_manager.cc:1434] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:30:07.136132  7361 consensus_queue.cc:260] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.136533  7365 tablet_bootstrap.cc:492] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.138720  7363 tablet_bootstrap.cc:492] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.139361  7363 ts_tablet_manager.cc:1403] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:30:07.139963  7361 ts_tablet_manager.cc:1434] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:30:07.141552  7361 tablet_bootstrap.cc:492] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.142707  7363 raft_consensus.cc:359] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.143597  7363 raft_consensus.cc:385] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.143937  7363 raft_consensus.cc:740] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.144728  7365 tablet_bootstrap.cc:654] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.144939  7363 consensus_queue.cc:260] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.150348  7363 ts_tablet_manager.cc:1434] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.005s
I20260709 12:30:07.150636  7361 tablet_bootstrap.cc:654] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.152030  7363 tablet_bootstrap.cc:492] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.156718  7365 tablet_bootstrap.cc:492] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.159613  7365 ts_tablet_manager.cc:1403] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 12:30:07.160275  7363 tablet_bootstrap.cc:654] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.163828  7361 tablet_bootstrap.cc:492] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.163228  7365 raft_consensus.cc:359] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.164574  7365 raft_consensus.cc:385] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.164585  7361 ts_tablet_manager.cc:1403] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.006s
I20260709 12:30:07.165236  7365 raft_consensus.cc:740] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.167086  7365 consensus_queue.cc:260] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.169495  7361 raft_consensus.cc:359] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.170441  7361 raft_consensus.cc:385] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.170797  7361 raft_consensus.cc:740] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.171783  7361 consensus_queue.cc:260] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.173739  7365 ts_tablet_manager.cc:1434] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:30:07.174741  7361 ts_tablet_manager.cc:1434] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:07.175426  7365 tablet_bootstrap.cc:492] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.176108  7361 tablet_bootstrap.cc:492] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.184532  7361 tablet_bootstrap.cc:654] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.184794  7363 tablet_bootstrap.cc:492] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.185298  7365 tablet_bootstrap.cc:654] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.185488  7363 ts_tablet_manager.cc:1403] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.012s
I20260709 12:30:07.189334  7363 raft_consensus.cc:359] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.190459  7363 raft_consensus.cc:385] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.190898  7363 raft_consensus.cc:740] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.191937  7363 consensus_queue.cc:260] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.194849  7363 ts_tablet_manager.cc:1434] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:07.196031  7363 tablet_bootstrap.cc:492] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.196977  7361 tablet_bootstrap.cc:492] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.197690  7361 ts_tablet_manager.cc:1403] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 12:30:07.200114  7365 tablet_bootstrap.cc:492] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.200850  7365 ts_tablet_manager.cc:1403] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.002s
I20260709 12:30:07.201036  7361 raft_consensus.cc:359] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.201994  7361 raft_consensus.cc:385] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.202363  7361 raft_consensus.cc:740] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.203496  7363 tablet_bootstrap.cc:654] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.203404  7361 consensus_queue.cc:260] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.204200  7365 raft_consensus.cc:359] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.205240  7365 raft_consensus.cc:385] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.205607  7365 raft_consensus.cc:740] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.206633  7365 consensus_queue.cc:260] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.208371  7361 ts_tablet_manager.cc:1434] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:30:07.209714  7361 tablet_bootstrap.cc:492] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.209864  7369 raft_consensus.cc:493] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.210577  7369 raft_consensus.cc:515] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.215745  7369 leader_election.cc:290] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.217666  7361 tablet_bootstrap.cc:654] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.221432  7363 tablet_bootstrap.cc:492] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.222623  7368 raft_consensus.cc:493] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.224426  7368 raft_consensus.cc:515] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.225931  7363 ts_tablet_manager.cc:1403] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.011s
W20260709 12:30:07.227150  7117 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:07.227465  7368 leader_election.cc:290] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.224426  7365 ts_tablet_manager.cc:1434] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 12:30:07.229859  7365 tablet_bootstrap.cc:492] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.253931  7361 tablet_bootstrap.cc:492] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.254586  7361 ts_tablet_manager.cc:1403] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.008s
I20260709 12:30:07.258111  7363 raft_consensus.cc:359] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.258581  7366 raft_consensus.cc:493] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.258952  7363 raft_consensus.cc:385] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.259325  7363 raft_consensus.cc:740] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.259150  7366 raft_consensus.cc:515] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.259713  7361 raft_consensus.cc:359] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.261202  7369 raft_consensus.cc:493] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.260120  7363 consensus_queue.cc:260] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.262259  7369 raft_consensus.cc:515] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.265115  7368 raft_consensus.cc:493] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.265844  7368 raft_consensus.cc:515] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.266330  7369 leader_election.cc:290] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.261560  7361 raft_consensus.cc:385] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.271675  7366 leader_election.cc:290] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.271878  7361 raft_consensus.cc:740] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.278369  7368 leader_election.cc:290] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067)
I20260709 12:30:07.293257  7363 ts_tablet_manager.cc:1434] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.066s	user 0.013s	sys 0.012s
I20260709 12:30:07.294602  7363 tablet_bootstrap.cc:492] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.297443  7361 consensus_queue.cc:260] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.301891  7363 tablet_bootstrap.cc:654] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.303398  7365 tablet_bootstrap.cc:654] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.304850  7368 raft_consensus.cc:493] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.306353  7368 raft_consensus.cc:515] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.309727  7368 leader_election.cc:290] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.310585  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.311336  7247 raft_consensus.cc:2468] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:07.312959  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.323344  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.325700  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.326849  7171 raft_consensus.cc:2468] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:07.328485  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.329564  7170 raft_consensus.cc:2468] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:07.330914  7365 tablet_bootstrap.cc:492] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.334501  7365 ts_tablet_manager.cc:1403] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.105s	user 0.017s	sys 0.012s
W20260709 12:30:07.338486  7284 leader_election.cc:343] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.339733  7284 leader_election.cc:304] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.341310  7284 leader_election.cc:304] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.342583  7368 raft_consensus.cc:2804] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.342998  7368 raft_consensus.cc:493] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.343400  7368 raft_consensus.cc:3060] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.353623  7369 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.339661  7365 raft_consensus.cc:359] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.354310  7369 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.356086  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.356882  7246 raft_consensus.cc:2468] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:07.358271  7369 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.359283  7365 raft_consensus.cc:385] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.359794  7365 raft_consensus.cc:740] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.361115  7365 consensus_queue.cc:260] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.362197  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.363265  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.364084  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.366233  7171 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:07.364857  7389 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.367036  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.367132  7389 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
W20260709 12:30:07.368919  7138 leader_election.cc:343] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.369143  7172 raft_consensus.cc:2468] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:07.363878  7368 raft_consensus.cc:515] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.374850  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.375926  7321 raft_consensus.cc:2468] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:07.376272  7365 ts_tablet_manager.cc:1434] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.002s
I20260709 12:30:07.378029  7365 tablet_bootstrap.cc:492] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.379287  7368 leader_election.cc:290] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067)
I20260709 12:30:07.384907  7368 raft_consensus.cc:2804] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.385529  7368 raft_consensus.cc:493] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.386070  7368 raft_consensus.cc:3060] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.390079  7369 raft_consensus.cc:493] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:07.390473  7210 leader_election.cc:343] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.390981  7369 raft_consensus.cc:515] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.393756  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.394711  7171 raft_consensus.cc:3060] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.391662  7210 leader_election.cc:304] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.398432  7136 leader_election.cc:304] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 4124877098414253a130e4e140806788
I20260709 12:30:07.400458  7366 raft_consensus.cc:2804] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.400974  7366 raft_consensus.cc:493] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.401420  7366 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.418705  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.419636  7246 raft_consensus.cc:2468] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:07.420758  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.421461  7138 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.418951  7389 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.423612  7389 raft_consensus.cc:2804] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.423704  7395 raft_consensus.cc:493] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.424023  7389 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:30:07.423806  7136 leader_election.cc:343] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.425668  7389 raft_consensus.cc:3060] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.431815  7398 raft_consensus.cc:2804] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.434310  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa90fd87ec3e44068275986fa849c02e" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.436475  7369 leader_election.cc:290] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.437948  7369 raft_consensus.cc:493] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.436079  7361 ts_tablet_manager.cc:1434] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.181s	user 0.048s	sys 0.048s
I20260709 12:30:07.444707  7369 raft_consensus.cc:515] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.436825  7172 raft_consensus.cc:2468] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:07.446332  7368 raft_consensus.cc:515] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.446975  7369 leader_election.cc:290] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 273e786494a845ff87c99139442c371d (127.24.71.131:41695), 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067)
I20260709 12:30:07.424309  7395 raft_consensus.cc:515] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.458050  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.458520  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa90fd87ec3e44068275986fa849c02e" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.458874  7287 leader_election.cc:304] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:07.439806  7398 raft_consensus.cc:493] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.460070  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.460119  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.460558  7398 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.460613  7397 raft_consensus.cc:2804] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.443077  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.443869  7363 tablet_bootstrap.cc:492] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.444887  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.445701  7361 tablet_bootstrap.cc:492] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
W20260709 12:30:07.460273  7287 leader_election.cc:343] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.462090  7320 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:07.452131  7366 raft_consensus.cc:515] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.435216  7210 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.463738  7400 raft_consensus.cc:2804] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.464146  7400 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.464430  7366 leader_election.cc:290] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.464514  7400 raft_consensus.cc:3060] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.447963  7389 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.464668  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.465757  7246 raft_consensus.cc:3060] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.466809  7389 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.439138  7171 raft_consensus.cc:2468] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:07.441920  7365 tablet_bootstrap.cc:654] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.468593  7284 leader_election.cc:304] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.459973  7395 leader_election.cc:290] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.461055  7317 raft_consensus.cc:2393] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:07.461054  7170 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:07.462049  7363 ts_tablet_manager.cc:1403] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.168s	user 0.076s	sys 0.051s
I20260709 12:30:07.472754  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.473511  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.473807  7363 raft_consensus.cc:359] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.461201  7397 raft_consensus.cc:493] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.452610  7368 leader_election.cc:290] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.449808  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.477190  7210 leader_election.cc:304] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.474642  7363 raft_consensus.cc:385] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:07.476393  7138 leader_election.cc:343] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.478158  7363 raft_consensus.cc:740] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.478601  7369 raft_consensus.cc:2804] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.477746  7172 raft_consensus.cc:3060] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.479003  7369 raft_consensus.cc:493] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.477514  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.479480  7369 raft_consensus.cc:3060] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.480008  7170 raft_consensus.cc:2393] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:07.479003  7363 consensus_queue.cc:260] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.469682  7361 tablet_bootstrap.cc:654] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.484655  7363 ts_tablet_manager.cc:1434] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:30:07.487958  7246 raft_consensus.cc:2468] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:07.489948  7389 raft_consensus.cc:493] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.490792  7363 tablet_bootstrap.cc:492] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.490612  7389 raft_consensus.cc:515] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.482813  7397 raft_consensus.cc:3060] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.495577  7389 leader_election.cc:290] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.496572  7369 raft_consensus.cc:515] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.497207  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.497879  7247 raft_consensus.cc:3060] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.499390  7369 leader_election.cc:290] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.500216  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa90fd87ec3e44068275986fa849c02e" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.500942  7170 raft_consensus.cc:3060] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.482592  7395 raft_consensus.cc:2804] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:30:07.506218  7211 leader_election.cc:343] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:07.509370  7211 leader_election.cc:343] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.510066  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.508208  7395 raft_consensus.cc:697] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:07.512840  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.522707  7284 leader_election.cc:304] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.515219  7400 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.517905  7244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.522740  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.524011  7401 raft_consensus.cc:2804] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.524505  7401 raft_consensus.cc:493] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.521405  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.511678  7246 raft_consensus.cc:2468] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:07.521344  7363 tablet_bootstrap.cc:654] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.525655  7401 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.526289  7138 leader_election.cc:304] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.526432  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.526845  7365 tablet_bootstrap.cc:492] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.527273  7389 raft_consensus.cc:2804] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.527400  7365 ts_tablet_manager.cc:1403] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.150s	user 0.018s	sys 0.012s
I20260709 12:30:07.527819  7389 raft_consensus.cc:493] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.528294  7389 raft_consensus.cc:3060] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.525741  7400 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.531514  7365 raft_consensus.cc:359] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.523236  7317 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.532337  7365 raft_consensus.cc:385] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.532681  7365 raft_consensus.cc:740] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
W20260709 12:30:07.532827  7136 leader_election.cc:343] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.524056  7247 raft_consensus.cc:2468] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:07.524574  7245 raft_consensus.cc:2393] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 255aee4d52c8404fbd9da5f2aa03e346 in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:07.538391  7138 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346; no voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788
I20260709 12:30:07.527324  7171 raft_consensus.cc:2393] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 255aee4d52c8404fbd9da5f2aa03e346 in this term.
I20260709 12:30:07.540772  7366 raft_consensus.cc:2749] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.541122  7401 raft_consensus.cc:515] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.542019  7211 leader_election.cc:304] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:07.540194  7365 consensus_queue.cc:260] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.542924  7400 raft_consensus.cc:2749] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.522881  7170 raft_consensus.cc:2468] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:07.521696  7397 raft_consensus.cc:515] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.549480  7210 leader_election.cc:304] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.550071  7398 raft_consensus.cc:515] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.551477  7244 raft_consensus.cc:2393] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:07.551677  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.552402  7171 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.552816  7401 leader_election.cc:290] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.553800  7398 leader_election.cc:290] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.554262  7172 raft_consensus.cc:2468] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:07.554788  7321 raft_consensus.cc:2393] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:07.550485  7369 raft_consensus.cc:2804] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:07.556609  7369 raft_consensus.cc:697] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:07.556347  7397 leader_election.cc:290] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.556746  7389 raft_consensus.cc:515] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.557570  7369 consensus_queue.cc:237] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.559031  7365 ts_tablet_manager.cc:1434] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.001s
I20260709 12:30:07.559185  7389 leader_election.cc:290] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.561268  7284 leader_election.cc:304] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.561604  7365 tablet_bootstrap.cc:492] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.562777  7397 raft_consensus.cc:2804] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:07.562845  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.552460  7317 raft_consensus.cc:2468] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:07.563041  7363 tablet_bootstrap.cc:492] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.563843  7321 raft_consensus.cc:2393] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 255aee4d52c8404fbd9da5f2aa03e346 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:07.563923  7363 ts_tablet_manager.cc:1403] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.073s	user 0.009s	sys 0.012s
I20260709 12:30:07.564424  7136 leader_election.cc:304] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 4124877098414253a130e4e140806788
I20260709 12:30:07.564980  7244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.565610  7244 raft_consensus.cc:3060] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.566619  7389 raft_consensus.cc:2804] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:07.567145  7363 raft_consensus.cc:359] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.568364  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.568053  7363 raft_consensus.cc:385] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.568531  7397 raft_consensus.cc:697] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:07.568989  7363 raft_consensus.cc:740] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.570631  7363 consensus_queue.cc:260] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.571381  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.571676  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa90fd87ec3e44068275986fa849c02e" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.569834  7397 consensus_queue.cc:237] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.573127  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:07.573967  7170 raft_consensus.cc:2393] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 255aee4d52c8404fbd9da5f2aa03e346 in this term.
I20260709 12:30:07.543684  7395 consensus_queue.cc:237] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:07.567351  7389 raft_consensus.cc:697] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:07.577342  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.578202  7318 raft_consensus.cc:2393] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:07.578037  7389 consensus_queue.cc:237] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.580621  7366 raft_consensus.cc:493] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.581203  7366 raft_consensus.cc:515] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.582448  7171 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:07.585482  7363 ts_tablet_manager.cc:1434] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:30:07.579092  7365 tablet_bootstrap.cc:654] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.588457  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239fc23a25f8480b97f6b032d88d9105" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:07.589193  7361 tablet_bootstrap.cc:492] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.589272  7318 raft_consensus.cc:2468] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:07.589843  7361 ts_tablet_manager.cc:1403] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.144s	user 0.010s	sys 0.004s
I20260709 12:30:07.588743  7363 tablet_bootstrap.cc:492] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:07.585017  7366 leader_election.cc:290] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.594128  7211 leader_election.cc:304] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:07.591423  7284 leader_election.cc:304] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.599565  7363 tablet_bootstrap.cc:654] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.600175  7401 raft_consensus.cc:2804] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:07.601276  7401 raft_consensus.cc:697] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:07.602162  7401 consensus_queue.cc:237] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.600625  7136 leader_election.cc:304] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:07.605787  7366 raft_consensus.cc:2804] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:07.606273  7366 raft_consensus.cc:493] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:07.606691  7366 raft_consensus.cc:3060] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.596064  7068 catalog_manager.cc:5697] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:07.597747  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.586012  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239fc23a25f8480b97f6b032d88d9105" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.585008  7244 raft_consensus.cc:2468] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:07.613139  7245 raft_consensus.cc:2393] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:07.616173  7369 raft_consensus.cc:2749] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.624684  7138 leader_election.cc:304] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 273e786494a845ff87c99139442c371d
I20260709 12:30:07.617077  7246 raft_consensus.cc:2468] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:07.624554  7361 raft_consensus.cc:359] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.631389  7361 raft_consensus.cc:385] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.631793  7361 raft_consensus.cc:740] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.632835  7361 consensus_queue.cc:260] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.619367  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
W20260709 12:30:07.621372  7211 leader_election.cc:343] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.641230  7389 raft_consensus.cc:2804] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:07.642796  7389 raft_consensus.cc:697] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:07.643983  7363 tablet_bootstrap.cc:492] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:07.644726  7363 ts_tablet_manager.cc:1403] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.013s
I20260709 12:30:07.644001  7389 consensus_queue.cc:237] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.648869  7363 raft_consensus.cc:359] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.649824  7363 raft_consensus.cc:385] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.650149  7363 raft_consensus.cc:740] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.651131  7366 raft_consensus.cc:515] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.650966  7363 consensus_queue.cc:260] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.653841  7363 ts_tablet_manager.cc:1434] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:30:07.654835  7066 catalog_manager.cc:5697] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:07.658787  7247 raft_consensus.cc:2393] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 255aee4d52c8404fbd9da5f2aa03e346 in this term.
I20260709 12:30:07.659741  7211 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:07.660251  7287 leader_election.cc:304] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 273e786494a845ff87c99139442c371d; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788
W20260709 12:30:07.661242  7211 leader_election.cc:343] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:07.661473  7397 raft_consensus.cc:2749] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.662335  7211 leader_election.cc:304] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:07.667294  7361 ts_tablet_manager.cc:1434] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.001s
I20260709 12:30:07.668402  7361 tablet_bootstrap.cc:492] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.672478  7398 raft_consensus.cc:2749] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.673324  7398 raft_consensus.cc:2749] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.676651  7361 tablet_bootstrap.cc:654] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.681213  7366 leader_election.cc:290] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:07.681615  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239fc23a25f8480b97f6b032d88d9105" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:07.681788  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239fc23a25f8480b97f6b032d88d9105" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:07.682756  7318 raft_consensus.cc:3060] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.682756  7247 raft_consensus.cc:3060] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.688730  7069 catalog_manager.cc:5697] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 0 to 1, leader changed from <none> to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 1 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:07.693449  7318 raft_consensus.cc:2468] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:07.696559  7247 raft_consensus.cc:2468] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:07.697978  7365 tablet_bootstrap.cc:492] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.698468  7365 ts_tablet_manager.cc:1403] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.137s	user 0.022s	sys 0.013s
I20260709 12:30:07.700294  7138 leader_election.cc:304] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:07.702152  7389 raft_consensus.cc:2804] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:07.702816  7389 raft_consensus.cc:697] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:07.703959  7389 consensus_queue.cc:237] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.712299  7365 raft_consensus.cc:359] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.714660  7365 raft_consensus.cc:385] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.715013  7365 raft_consensus.cc:740] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.715744  7361 tablet_bootstrap.cc:492] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.715809  7365 consensus_queue.cc:260] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.716500  7361 ts_tablet_manager.cc:1403] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.022s
I20260709 12:30:07.718557  7365 ts_tablet_manager.cc:1434] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:30:07.719499  7365 tablet_bootstrap.cc:492] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:07.719405  7361 raft_consensus.cc:359] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.720358  7361 raft_consensus.cc:385] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.720702  7361 raft_consensus.cc:740] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.721586  7361 consensus_queue.cc:260] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.726435  7365 tablet_bootstrap.cc:654] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.727001  7401 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.727617  7401 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.731678  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.732896  7247 raft_consensus.cc:2393] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:07.735504  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.727563  7361 ts_tablet_manager.cc:1434] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:30:07.737248  7171 raft_consensus.cc:2393] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 255aee4d52c8404fbd9da5f2aa03e346 in this term.
I20260709 12:30:07.736137  7401 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.738970  7284 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 273e786494a845ff87c99139442c371d; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788
I20260709 12:30:07.741592  7401 raft_consensus.cc:3060] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.748828  7361 tablet_bootstrap.cc:492] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:07.754081  7068 catalog_manager.cc:5697] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:07.755838  7068 catalog_manager.cc:5697] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:07.757241  7066 catalog_manager.cc:5697] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 0 to 1, leader changed from <none> to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 1 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:07.759245  7066 catalog_manager.cc:5697] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 0 to 1, leader changed from <none> to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 1 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:07.761925  7401 raft_consensus.cc:2749] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.764827  7361 tablet_bootstrap.cc:654] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:07.769160  7365 tablet_bootstrap.cc:492] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:07.769737  7365 ts_tablet_manager.cc:1403] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.027s
I20260709 12:30:07.772099  7365 raft_consensus.cc:359] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.772861  7365 raft_consensus.cc:385] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.773190  7365 raft_consensus.cc:740] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.773921  7365 consensus_queue.cc:260] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.775949  7365 ts_tablet_manager.cc:1434] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:30:07.776648  7361 tablet_bootstrap.cc:492] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:07.777271  7361 ts_tablet_manager.cc:1403] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.000s
I20260709 12:30:07.779568  7361 raft_consensus.cc:359] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.780094  7361 raft_consensus.cc:385] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:07.780373  7361 raft_consensus.cc:740] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:07.780925  7361 consensus_queue.cc:260] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.783041  7361 ts_tablet_manager.cc:1434] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:30:07.822989  7401 raft_consensus.cc:493] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:07.823607  7401 raft_consensus.cc:515] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:07.825657  7401 leader_election.cc:290] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:07.826699  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa90fd87ec3e44068275986fa849c02e" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:07.826803  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa90fd87ec3e44068275986fa849c02e" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:07.827513  7171 raft_consensus.cc:2393] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:07.828714  7284 leader_election.cc:304] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 273e786494a845ff87c99139442c371d; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788
I20260709 12:30:07.829504  7401 raft_consensus.cc:3060] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:07.835120  7401 raft_consensus.cc:2749] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:07.989967  7389 consensus_queue.cc:1048] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.000990  7247 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:08.010609  7389 consensus_queue.cc:1048] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.068060  7389 consensus_queue.cc:1048] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.073729  7401 consensus_queue.cc:1048] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.081132  7366 consensus_queue.cc:1048] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.087641  7368 consensus_queue.cc:1048] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.091732  7397 consensus_queue.cc:1048] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.104964  7397 consensus_queue.cc:1048] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.110221  7398 consensus_queue.cc:1048] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.126386  7369 consensus_queue.cc:1048] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:08.136794  7428 raft_consensus.cc:493] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:08.136978  7366 consensus_queue.cc:1048] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.137332  7428 raft_consensus.cc:515] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:08.139711  7428 leader_election.cc:290] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:08.141425  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:08.141564  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:08.143326  7318 raft_consensus.cc:2468] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:08.143514  7247 raft_consensus.cc:2468] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:08.144863  7136 leader_election.cc:304] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:08.145617  7428 raft_consensus.cc:2804] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:08.146057  7428 raft_consensus.cc:493] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:08.146471  7428 raft_consensus.cc:3060] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:08.157336  7428 raft_consensus.cc:515] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:08.160457  7428 leader_election.cc:290] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:08.160406  7366 consensus_queue.cc:1048] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.160820  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:08.161507  7245 raft_consensus.cc:3060] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:08.162349  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:08.162950  7318 raft_consensus.cc:3060] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:08.169088  7245 raft_consensus.cc:2468] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:08.170686  7138 leader_election.cc:304] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:08.174983  7366 raft_consensus.cc:2804] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:08.175640  7366 raft_consensus.cc:697] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:08.176797  7397 consensus_queue.cc:1048] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.179543  7318 raft_consensus.cc:2468] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:08.181984  7366 consensus_queue.cc:237] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:08.190860  7368 consensus_queue.cc:1048] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.195438  7069 catalog_manager.cc:5697] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:08.231986  7117 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:08.583879  7366 consensus_queue.cc:1048] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:08.594976  7428 consensus_queue.cc:1048] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:09.236673  7117 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:09.725499  7441 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:09.726086  7441 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:09.728034  7441 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:09.729027  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:09.729123  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:09.729794  7170 raft_consensus.cc:2468] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:09.729823  7318 raft_consensus.cc:2468] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:09.731302  7210 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:09.732039  7441 raft_consensus.cc:2804] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:09.732476  7441 raft_consensus.cc:493] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:09.732906  7441 raft_consensus.cc:3060] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:09.738572  7441 raft_consensus.cc:515] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:09.740401  7441 leader_election.cc:290] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:09.741407  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:09.741719  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e3992fbed914c4787d3089d7932f264" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:09.742327  7170 raft_consensus.cc:3060] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:09.742442  7318 raft_consensus.cc:3060] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:09.748528  7318 raft_consensus.cc:2468] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:09.749179  7170 raft_consensus.cc:2468] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:09.749670  7211 leader_election.cc:304] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:09.750553  7444 raft_consensus.cc:2804] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:09.751075  7444 raft_consensus.cc:697] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 2 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:09.751806  7444 consensus_queue.cc:237] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:09.759457  7066 catalog_manager.cc:5697] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 reported cstate change: term changed from 0 to 2, leader changed from <none> to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:10.116415  7318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c"
dest_uuid: "273e786494a845ff87c99139442c371d"
mode: GRACEFUL
new_leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:55710
I20260709 12:30:10.117340  7318 raft_consensus.cc:606] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Received request to transfer leadership to TS 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:10.123553  7318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833"
dest_uuid: "273e786494a845ff87c99139442c371d"
mode: GRACEFUL
new_leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:55710
I20260709 12:30:10.124089  7318 raft_consensus.cc:606] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Received request to transfer leadership to TS 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:10.130311  7318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60"
dest_uuid: "273e786494a845ff87c99139442c371d"
mode: GRACEFUL
new_leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:55710
I20260709 12:30:10.130848  7318 raft_consensus.cc:606] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Received request to transfer leadership to TS 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:10.156337  7170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
mode: GRACEFUL
new_leader_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:60400
I20260709 12:30:10.157106  7170 raft_consensus.cc:606] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Received request to transfer leadership to TS 4124877098414253a130e4e140806788
I20260709 12:30:10.163674  7170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
mode: GRACEFUL
new_leader_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:60400
I20260709 12:30:10.164390  7170 raft_consensus.cc:606] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Received request to transfer leadership to TS 4124877098414253a130e4e140806788
I20260709 12:30:10.174031  7170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
mode: GRACEFUL
new_leader_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:60400
I20260709 12:30:10.174785  7170 raft_consensus.cc:606] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Received request to transfer leadership to TS 273e786494a845ff87c99139442c371d
I20260709 12:30:10.210212  7170 raft_consensus.cc:1275] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:10.212867  7444 consensus_queue.cc:1048] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:10.228528  7318 raft_consensus.cc:1275] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:10.230742  7441 consensus_queue.cc:1048] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:10.365288  7389 raft_consensus.cc:993] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: : Instructing follower 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:10.365957  7460 raft_consensus.cc:1081] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Signalling peer 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:10.367880  7460 raft_consensus.cc:993] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: : Instructing follower 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:10.368402  7245 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f"
dest_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:51022
I20260709 12:30:10.368427  7389 raft_consensus.cc:1081] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Signalling peer 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:10.369134  7245 raft_consensus.cc:493] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:10.369688  7245 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.372135  7247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f"
dest_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:51022
I20260709 12:30:10.372833  7247 raft_consensus.cc:493] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:10.373238  7247 raft_consensus.cc:3060] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.380317  7245 raft_consensus.cc:515] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.381757  7247 raft_consensus.cc:515] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.383162  7245 leader_election.cc:290] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:10.383651  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:10.384086  7247 leader_election.cc:290] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:10.384526  7170 raft_consensus.cc:3055] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:10.384895  7170 raft_consensus.cc:740] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:10.385248  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:10.385697  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca42f5a8e6e14af896744c41a9c2d65f" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:10.386097  7171 raft_consensus.cc:3055] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:10.386500  7317 raft_consensus.cc:3060] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.386533  7171 raft_consensus.cc:740] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:10.385650  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:10.387665  7318 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.386144  7170 consensus_queue.cc:260] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.387909  7171 consensus_queue.cc:260] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.389670  7170 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.389760  7171 raft_consensus.cc:3060] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.398756  7317 raft_consensus.cc:2468] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:10.399263  7318 raft_consensus.cc:2468] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:10.400149  7170 raft_consensus.cc:2468] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:10.401129  7171 raft_consensus.cc:2468] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:10.402377  7211 leader_election.cc:304] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:10.403499  7441 raft_consensus.cc:2804] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:10.404021  7441 raft_consensus.cc:697] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:10.404549  7210 leader_election.cc:304] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:10.405367  7441 consensus_queue.cc:237] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.405906  7444 raft_consensus.cc:2804] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:10.410244  7444 raft_consensus.cc:697] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:10.411625  7444 consensus_queue.cc:237] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.417274  7068 catalog_manager.cc:5697] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 reported cstate change: term changed from 1 to 2, leader changed from 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129) to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:10.436793  7068 catalog_manager.cc:5697] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 reported cstate change: term changed from 1 to 2, leader changed from 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129) to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:10.465456  7451 raft_consensus.cc:993] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: : Instructing follower 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:10.466008  7451 raft_consensus.cc:1081] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Signalling peer 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:10.468273  7171 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:60386
I20260709 12:30:10.468823  7171 raft_consensus.cc:493] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:10.469190  7171 raft_consensus.cc:3060] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.474793  7171 raft_consensus.cc:515] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:10.477309  7171 leader_election.cc:290] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Requested vote from peers 273e786494a845ff87c99139442c371d (127.24.71.131:41695), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:10.477825  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:10.477898  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70bd6da3b3924e2896be369b7e6a253c" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:10.478829  7245 raft_consensus.cc:3060] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.478865  7317 raft_consensus.cc:3055] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:10.479365  7317 raft_consensus.cc:740] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:10.480777  7317 consensus_queue.cc:260] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:10.482452  7317 raft_consensus.cc:3060] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.488129  7245 raft_consensus.cc:2468] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:10.489938  7138 leader_election.cc:304] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:10.490142  7317 raft_consensus.cc:2468] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:10.490834  7389 raft_consensus.cc:2804] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:10.491369  7389 raft_consensus.cc:697] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:10.492360  7389 consensus_queue.cc:237] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } }
I20260709 12:30:10.500561  7068 catalog_manager.cc:5697] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 1 to 2, leader changed from 273e786494a845ff87c99139442c371d (127.24.71.131) to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:10.538506  7460 raft_consensus.cc:993] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: : Instructing follower 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:10.539036  7389 raft_consensus.cc:1081] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Signalling peer 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:10.541002  7317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705"
dest_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:55700
I20260709 12:30:10.541679  7317 raft_consensus.cc:493] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:10.542011  7317 raft_consensus.cc:3060] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.550024  7317 raft_consensus.cc:515] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.551980  7317 leader_election.cc:290] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:10.553206  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:10.553233  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701e4ba57f074d4b97828ec0d9c6a705" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:10.554069  7245 raft_consensus.cc:3060] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.554195  7171 raft_consensus.cc:3055] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:10.554697  7171 raft_consensus.cc:740] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:10.555879  7171 consensus_queue.cc:260] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.557335  7171 raft_consensus.cc:3060] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.560828  7245 raft_consensus.cc:2468] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 2.
I20260709 12:30:10.562368  7287 leader_election.cc:304] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:10.563346  7451 raft_consensus.cc:2804] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:10.563963  7451 raft_consensus.cc:697] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 2 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:10.564659  7171 raft_consensus.cc:2468] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 2.
I20260709 12:30:10.565181  7451 consensus_queue.cc:237] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.569121  7469 raft_consensus.cc:993] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: : Instructing follower 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:10.570057  7401 raft_consensus.cc:1081] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Signalling peer 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:10.572395  7171 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:60386
I20260709 12:30:10.573192  7171 raft_consensus.cc:493] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:10.573619  7171 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.575920  7068 catalog_manager.cc:5697] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 1 to 2, leader changed from 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129) to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 2 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:10.581404  7171 raft_consensus.cc:515] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.584510  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:10.584649  7171 leader_election.cc:290] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:10.585017  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:10.586225  7317 raft_consensus.cc:3055] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:10.586596  7317 raft_consensus.cc:740] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:10.587481  7317 consensus_queue.cc:260] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.585439  7245 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.588671  7317 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.594257  7245 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:10.594406  7317 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:10.595592  7138 leader_election.cc:304] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:10.596473  7389 raft_consensus.cc:2804] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:10.596982  7389 raft_consensus.cc:697] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:10.597874  7389 consensus_queue.cc:237] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.605326  7068 catalog_manager.cc:5697] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 1 to 2, leader changed from 273e786494a845ff87c99139442c371d (127.24.71.131) to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:10.622851  7451 raft_consensus.cc:993] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: : Instructing follower 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:10.623534  7401 raft_consensus.cc:1081] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Signalling peer 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:10.625548  7171 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:60386
I20260709 12:30:10.626171  7171 raft_consensus.cc:493] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:10.626515  7171 raft_consensus.cc:3060] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.632575  7171 raft_consensus.cc:515] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.634305  7171 leader_election.cc:290] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:10.635330  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:10.635371  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e2d5ba97c54e5a9130875f239c8833" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:10.636080  7317 raft_consensus.cc:3055] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:10.636133  7245 raft_consensus.cc:3060] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.636489  7317 raft_consensus.cc:740] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:10.637601  7317 consensus_queue.cc:260] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.638715  7317 raft_consensus.cc:3060] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:10.642231  7245 raft_consensus.cc:2468] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:10.643548  7138 leader_election.cc:304] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:10.644465  7389 raft_consensus.cc:2804] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:10.644632  7317 raft_consensus.cc:2468] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:10.645015  7389 raft_consensus.cc:697] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:10.646251  7389 consensus_queue.cc:237] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:10.654505  7068 catalog_manager.cc:5697] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 1 to 2, leader changed from 273e786494a845ff87c99139442c371d (127.24.71.131) to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:10.840205  7317 raft_consensus.cc:1275] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:10.841665  7444 consensus_queue.cc:1048] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:10.857064  7171 raft_consensus.cc:1275] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:10.859650  7462 consensus_queue.cc:1048] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:10.888680  7171 raft_consensus.cc:1275] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:10.890336  7462 consensus_queue.cc:1048] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:10.890519  7317 raft_consensus.cc:1275] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:10.891947  7389 consensus_queue.cc:1048] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:10.904843  7321 raft_consensus.cc:1275] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:10.906394  7245 raft_consensus.cc:1275] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:10.907295  7441 consensus_queue.cc:1048] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:10.908376  7480 consensus_queue.cc:1048] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:11.011677  7171 raft_consensus.cc:1275] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:11.013322  7401 consensus_queue.cc:1048] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:11.023489  7245 raft_consensus.cc:1275] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:11.025174  7401 consensus_queue.cc:1048] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:11.058452  7245 raft_consensus.cc:1275] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:11.060075  7480 consensus_queue.cc:1048] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:11.070111  7317 raft_consensus.cc:1275] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:11.072142  7480 consensus_queue.cc:1048] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:11.115518  7245 raft_consensus.cc:1275] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:11.116948  7480 consensus_queue.cc:1048] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:11.127079  7317 raft_consensus.cc:1275] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:11.129199  7480 consensus_queue.cc:1048] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:13.211229  7066 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50668:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:13.214643  7066 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:13.269956  7156 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd7b1fae3ec4166ae6c100580a91a78 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:13.271502  7229 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd7b1fae3ec4166ae6c100580a91a78 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:13.271896  7156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd7b1fae3ec4166ae6c100580a91a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.273124  7229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd7b1fae3ec4166ae6c100580a91a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.272977  7308 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd7b1fae3ec4166ae6c100580a91a78 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:13.274734  7308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd7b1fae3ec4166ae6c100580a91a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.276089  7159 tablet_service.cc:1511] Processing CreateTablet for tablet 2ba3b888b8cf49398138f9a55bcdf7be (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:13.277601  7159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ba3b888b8cf49398138f9a55bcdf7be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.280561  7307 tablet_service.cc:1511] Processing CreateTablet for tablet 2ba3b888b8cf49398138f9a55bcdf7be (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:13.282168  7307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ba3b888b8cf49398138f9a55bcdf7be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:13.282259  7117 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:13.282481  7157 tablet_service.cc:1511] Processing CreateTablet for tablet 99753f55a73849b5946167ef4bb928ff (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:13.283941  7157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99753f55a73849b5946167ef4bb928ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.286579  7231 tablet_service.cc:1511] Processing CreateTablet for tablet 2ba3b888b8cf49398138f9a55bcdf7be (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:13.287811  7309 tablet_service.cc:1511] Processing CreateTablet for tablet 99753f55a73849b5946167ef4bb928ff (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:13.289578  7234 tablet_service.cc:1511] Processing CreateTablet for tablet 99753f55a73849b5946167ef4bb928ff (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:13.290771  7160 tablet_service.cc:1511] Processing CreateTablet for tablet 8d727a6836cf41c0a6d6a557bf425bb6 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:13.292160  7160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d727a6836cf41c0a6d6a557bf425bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.292577  7231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ba3b888b8cf49398138f9a55bcdf7be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.294512  7234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99753f55a73849b5946167ef4bb928ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.298666  7230 tablet_service.cc:1511] Processing CreateTablet for tablet 8d727a6836cf41c0a6d6a557bf425bb6 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:13.301538  7310 tablet_service.cc:1511] Processing CreateTablet for tablet 8d727a6836cf41c0a6d6a557bf425bb6 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:13.304184  7161 tablet_service.cc:1511] Processing CreateTablet for tablet bf09a69e231c4890b4a80587cb30a4d0 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:13.305781  7161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf09a69e231c4890b4a80587cb30a4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.306162  7237 tablet_service.cc:1511] Processing CreateTablet for tablet bf09a69e231c4890b4a80587cb30a4d0 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:13.308759  7305 tablet_service.cc:1511] Processing CreateTablet for tablet bf09a69e231c4890b4a80587cb30a4d0 (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:13.311445  7155 tablet_service.cc:1511] Processing CreateTablet for tablet 6972b81958a24101b9e7b7182d43abcd (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:13.312995  7155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6972b81958a24101b9e7b7182d43abcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.315032  7311 tablet_service.cc:1511] Processing CreateTablet for tablet 6972b81958a24101b9e7b7182d43abcd (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:13.317611  7233 tablet_service.cc:1511] Processing CreateTablet for tablet 6972b81958a24101b9e7b7182d43abcd (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:13.317946  7162 tablet_service.cc:1511] Processing CreateTablet for tablet b4b4fec5921e4dafa914dc0a63a80bcb (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:13.319213  7162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b4fec5921e4dafa914dc0a63a80bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.320277  7232 tablet_service.cc:1511] Processing CreateTablet for tablet b4b4fec5921e4dafa914dc0a63a80bcb (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:13.323256  7306 tablet_service.cc:1511] Processing CreateTablet for tablet b4b4fec5921e4dafa914dc0a63a80bcb (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:13.327157  7158 tablet_service.cc:1511] Processing CreateTablet for tablet 491a5616a3df46adba0b22b9731ad2df (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:13.328575  7236 tablet_service.cc:1511] Processing CreateTablet for tablet 491a5616a3df46adba0b22b9731ad2df (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:13.329118  7158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491a5616a3df46adba0b22b9731ad2df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.332011  7304 tablet_service.cc:1511] Processing CreateTablet for tablet 491a5616a3df46adba0b22b9731ad2df (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:13.333377  7230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d727a6836cf41c0a6d6a557bf425bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.342029  7154 tablet_service.cc:1511] Processing CreateTablet for tablet 70b8bd49811b4b6f8108cdaad13fbada (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:13.343842  7154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8bd49811b4b6f8108cdaad13fbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.347971  7235 tablet_service.cc:1511] Processing CreateTablet for tablet 70b8bd49811b4b6f8108cdaad13fbada (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:13.289609  7309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99753f55a73849b5946167ef4bb928ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.350595  7304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491a5616a3df46adba0b22b9731ad2df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.351284  7310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d727a6836cf41c0a6d6a557bf425bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.353284  7311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6972b81958a24101b9e7b7182d43abcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.364802  7305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf09a69e231c4890b4a80587cb30a4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.365780  7306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b4fec5921e4dafa914dc0a63a80bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.348667  7236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491a5616a3df46adba0b22b9731ad2df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.381796  7303 tablet_service.cc:1511] Processing CreateTablet for tablet 70b8bd49811b4b6f8108cdaad13fbada (DEFAULT_TABLE table=soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:13.383821  7303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8bd49811b4b6f8108cdaad13fbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.384904  7237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf09a69e231c4890b4a80587cb30a4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.387229  7233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6972b81958a24101b9e7b7182d43abcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.392637  7232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b4fec5921e4dafa914dc0a63a80bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.428360  7503 tablet_bootstrap.cc:492] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.436493  7235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8bd49811b4b6f8108cdaad13fbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:13.460744  7503 tablet_bootstrap.cc:654] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.477165  7504 tablet_bootstrap.cc:492] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.516999  7505 tablet_bootstrap.cc:492] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:13.519237  7503 tablet_bootstrap.cc:492] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.519963  7503 ts_tablet_manager.cc:1403] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.092s	user 0.002s	sys 0.012s
I20260709 12:30:13.522514  7504 tablet_bootstrap.cc:654] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.524039  7503 raft_consensus.cc:359] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.538592  7503 raft_consensus.cc:385] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.539016  7503 raft_consensus.cc:740] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.542392  7503 consensus_queue.cc:260] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.545682  7503 ts_tablet_manager.cc:1434] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.004s
I20260709 12:30:13.546756  7503 tablet_bootstrap.cc:492] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.553879  7505 tablet_bootstrap.cc:654] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.558255  7503 tablet_bootstrap.cc:654] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.562068  7504 tablet_bootstrap.cc:492] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.562770  7504 ts_tablet_manager.cc:1403] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.002s
I20260709 12:30:13.565387  7503 tablet_bootstrap.cc:492] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.566123  7503 ts_tablet_manager.cc:1403] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.002s
I20260709 12:30:13.566327  7504 raft_consensus.cc:359] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.567247  7504 raft_consensus.cc:385] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.567616  7504 raft_consensus.cc:740] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.568521  7504 consensus_queue.cc:260] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.570742  7505 tablet_bootstrap.cc:492] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.570616  7503 raft_consensus.cc:359] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.571405  7505 ts_tablet_manager.cc:1403] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.000s
I20260709 12:30:13.571493  7504 ts_tablet_manager.cc:1434] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:30:13.571645  7503 raft_consensus.cc:385] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.572075  7503 raft_consensus.cc:740] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.573160  7504 tablet_bootstrap.cc:492] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.573117  7503 consensus_queue.cc:260] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.576880  7503 ts_tablet_manager.cc:1434] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:30:13.578209  7503 tablet_bootstrap.cc:492] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.582386  7504 tablet_bootstrap.cc:654] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.582371  7505 raft_consensus.cc:359] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.583523  7505 raft_consensus.cc:385] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.583940  7505 raft_consensus.cc:740] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.585078  7505 consensus_queue.cc:260] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.587393  7503 tablet_bootstrap.cc:654] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.587994  7505 ts_tablet_manager.cc:1434] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.016s	user 0.001s	sys 0.006s
I20260709 12:30:13.589269  7505 tablet_bootstrap.cc:492] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:13.596652  7505 tablet_bootstrap.cc:654] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.611459  7505 tablet_bootstrap.cc:492] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.612001  7505 ts_tablet_manager.cc:1403] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:30:13.612495  7503 tablet_bootstrap.cc:492] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.613282  7503 ts_tablet_manager.cc:1403] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.011s
I20260709 12:30:13.615217  7505 raft_consensus.cc:359] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.615902  7505 raft_consensus.cc:385] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.616189  7505 raft_consensus.cc:740] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.617087  7505 consensus_queue.cc:260] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.617314  7503 raft_consensus.cc:359] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.618054  7503 raft_consensus.cc:385] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.618209  7504 tablet_bootstrap.cc:492] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.618412  7503 raft_consensus.cc:740] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.619019  7504 ts_tablet_manager.cc:1403] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.012s
I20260709 12:30:13.619719  7505 ts_tablet_manager.cc:1434] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:30:13.619323  7503 consensus_queue.cc:260] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.620693  7444 raft_consensus.cc:493] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.620867  7505 tablet_bootstrap.cc:492] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:13.621337  7444 raft_consensus.cc:515] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.622779  7504 raft_consensus.cc:359] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.623679  7504 raft_consensus.cc:385] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.624045  7504 raft_consensus.cc:740] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.624949  7504 consensus_queue.cc:260] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.625382  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:13.627656  7504 ts_tablet_manager.cc:1434] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:30:13.628962  7505 tablet_bootstrap.cc:654] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.629825  7504 tablet_bootstrap.cc:492] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.630077  7503 ts_tablet_manager.cc:1434] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.016s	user 0.012s	sys 0.003s
W20260709 12:30:13.632555  7210 leader_election.cc:343] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.636719  7444 leader_election.cc:290] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:13.638033  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:13.638638  7503 tablet_bootstrap.cc:492] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: Bootstrap starting.
W20260709 12:30:13.639835  7211 leader_election.cc:343] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.640391  7211 leader_election.cc:304] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:13.641762  7505 tablet_bootstrap.cc:492] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.642383  7505 ts_tablet_manager.cc:1403] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 12:30:13.643664  7444 raft_consensus.cc:2749] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.644863  7504 tablet_bootstrap.cc:654] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.646627  7503 tablet_bootstrap.cc:654] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.645777  7505 raft_consensus.cc:359] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.653225  7505 raft_consensus.cc:385] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.653573  7505 raft_consensus.cc:740] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.654409  7505 consensus_queue.cc:260] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.657397  7505 ts_tablet_manager.cc:1434] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 12:30:13.658617  7505 tablet_bootstrap.cc:492] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:13.661008  7504 tablet_bootstrap.cc:492] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.661713  7504 ts_tablet_manager.cc:1403] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.008s
I20260709 12:30:13.661976  7490 raft_consensus.cc:493] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.662722  7490 raft_consensus.cc:515] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.665607  7504 raft_consensus.cc:359] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.668933  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
W20260709 12:30:13.671346  7284 leader_election.cc:343] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.667627  7504 raft_consensus.cc:385] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.673997  7490 leader_election.cc:290] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.676064  7504 raft_consensus.cc:740] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.677127  7505 tablet_bootstrap.cc:654] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.676951  7504 consensus_queue.cc:260] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.677917  7503 tablet_bootstrap.cc:492] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.678488  7503 ts_tablet_manager.cc:1403] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.040s	user 0.025s	sys 0.012s
I20260709 12:30:13.679822  7504 ts_tablet_manager.cc:1434] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:30:13.681042  7504 tablet_bootstrap.cc:492] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.681950  7503 raft_consensus.cc:359] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.682792  7503 raft_consensus.cc:385] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.683101  7503 raft_consensus.cc:740] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.675275  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:13.683979  7503 consensus_queue.cc:260] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.684602  7245 raft_consensus.cc:2468] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:13.685451  7490 raft_consensus.cc:493] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.686188  7490 raft_consensus.cc:515] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.686803  7503 ts_tablet_manager.cc:1434] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:13.687958  7503 tablet_bootstrap.cc:492] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.688848  7287 leader_election.cc:304] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:13.692140  7490 leader_election.cc:290] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.693295  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:13.694970  7504 tablet_bootstrap.cc:654] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.696806  7170 raft_consensus.cc:2468] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:13.697836  7490 raft_consensus.cc:2804] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.698848  7490 raft_consensus.cc:493] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.699312  7490 raft_consensus.cc:3060] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.699045  7284 leader_election.cc:304] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:13.720439  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:13.721527  7505 tablet_bootstrap.cc:492] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.723750  7518 raft_consensus.cc:2804] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.725914  7518 raft_consensus.cc:493] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.724782  7505 ts_tablet_manager.cc:1403] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.025s
I20260709 12:30:13.726584  7518 raft_consensus.cc:3060] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:13.725100  7287 leader_election.cc:343] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.730429  7505 raft_consensus.cc:359] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.731030  7505 raft_consensus.cc:385] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.731307  7505 raft_consensus.cc:740] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.732018  7505 consensus_queue.cc:260] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.733603  7504 tablet_bootstrap.cc:492] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.734187  7504 ts_tablet_manager.cc:1403] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.012s
I20260709 12:30:13.735755  7518 raft_consensus.cc:515] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.737187  7503 tablet_bootstrap.cc:654] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.738808  7518 leader_election.cc:290] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.738977  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:13.739627  7170 raft_consensus.cc:3060] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.740142  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
W20260709 12:30:13.743582  7287 leader_election.cc:343] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.747123  7170 raft_consensus.cc:2468] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:13.748399  7284 leader_election.cc:304] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 4124877098414253a130e4e140806788
I20260709 12:30:13.749574  7518 raft_consensus.cc:2804] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.750097  7518 raft_consensus.cc:697] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:13.737730  7504 raft_consensus.cc:359] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.750794  7504 raft_consensus.cc:385] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.751147  7504 raft_consensus.cc:740] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.750967  7518 consensus_queue.cc:237] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.752085  7504 consensus_queue.cc:260] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.749706  7490 raft_consensus.cc:515] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.762997  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
W20260709 12:30:13.764367  7284 leader_election.cc:343] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.765604  7490 leader_election.cc:290] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.771733  7490 raft_consensus.cc:493] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.772208  7490 raft_consensus.cc:515] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.775315  7504 ts_tablet_manager.cc:1434] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:30:13.777941  7520 raft_consensus.cc:493] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.778493  7520 raft_consensus.cc:515] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.781426  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:13.785514  7504 tablet_bootstrap.cc:492] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.786010  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:13.786667  7245 raft_consensus.cc:3060] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:13.790634  7284 leader_election.cc:343] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.791558  7490 leader_election.cc:290] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.795401  7504 tablet_bootstrap.cc:654] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.795701  7505 ts_tablet_manager.cc:1434] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.068s	user 0.010s	sys 0.040s
I20260709 12:30:13.795818  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:13.797077  7505 tablet_bootstrap.cc:492] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: Bootstrap starting.
W20260709 12:30:13.797576  7287 leader_election.cc:343] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.798110  7287 leader_election.cc:304] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 273e786494a845ff87c99139442c371d; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788
I20260709 12:30:13.799468  7524 raft_consensus.cc:2749] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.801833  7245 raft_consensus.cc:2468] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:13.803797  7287 leader_election.cc:304] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:13.806079  7513 raft_consensus.cc:493] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.806813  7513 raft_consensus.cc:515] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.808161  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:13.807754  7068 catalog_manager.cc:5697] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 0 to 1, leader changed from <none> to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 1 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.808885  7170 raft_consensus.cc:2468] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 0.
I20260709 12:30:13.809784  7513 leader_election.cc:290] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:13.805362  7523 raft_consensus.cc:2804] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.810669  7284 leader_election.cc:304] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:13.810849  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:13.810997  7523 raft_consensus.cc:697] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:13.812199  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
W20260709 12:30:13.813319  7211 leader_election.cc:343] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.812553  7523 consensus_queue.cc:237] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.813825  7524 raft_consensus.cc:2804] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.814273  7524 raft_consensus.cc:493] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.814666  7524 raft_consensus.cc:3060] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.816437  7503 tablet_bootstrap.cc:492] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.817035  7503 ts_tablet_manager.cc:1403] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.029s
I20260709 12:30:13.821048  7503 raft_consensus.cc:359] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.821803  7503 raft_consensus.cc:385] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.822089  7503 raft_consensus.cc:740] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.822918  7503 consensus_queue.cc:260] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.824573  7170 raft_consensus.cc:2468] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:13.824647  7505 tablet_bootstrap.cc:654] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.826122  7210 leader_election.cc:304] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 273e786494a845ff87c99139442c371d
I20260709 12:30:13.826984  7513 raft_consensus.cc:2804] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.827216  7520 leader_election.cc:290] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.827390  7513 raft_consensus.cc:493] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.827605  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:13.827862  7513 raft_consensus.cc:3060] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:13.829108  7287 leader_election.cc:343] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.829583  7503 ts_tablet_manager.cc:1434] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:30:13.830734  7503 tablet_bootstrap.cc:492] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.831679  7504 tablet_bootstrap.cc:492] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.832432  7504 ts_tablet_manager.cc:1403] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.049s	user 0.025s	sys 0.017s
I20260709 12:30:13.833112  7524 raft_consensus.cc:515] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.835646  7504 raft_consensus.cc:359] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.836472  7504 raft_consensus.cc:385] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.836751  7504 raft_consensus.cc:740] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.836869  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:13.837718  7503 tablet_bootstrap.cc:654] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.837656  7170 raft_consensus.cc:3060] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.837482  7504 consensus_queue.cc:260] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.840013  7504 ts_tablet_manager.cc:1434] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:30:13.847175  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
W20260709 12:30:13.849166  7287 leader_election.cc:343] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.849970  7524 leader_election.cc:290] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:13.851117  7170 raft_consensus.cc:2468] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 1.
I20260709 12:30:13.852487  7284 leader_election.cc:304] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 4124877098414253a130e4e140806788
I20260709 12:30:13.853405  7524 raft_consensus.cc:2804] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.854084  7524 raft_consensus.cc:697] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:13.855448  7524 consensus_queue.cc:237] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.860905  7504 tablet_bootstrap.cc:492] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.855701  7513 raft_consensus.cc:515] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.865718  7513 leader_election.cc:290] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:13.866618  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:13.867201  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:13.871973  7171 raft_consensus.cc:3060] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:13.873898  7211 leader_election.cc:343] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.882010  7505 tablet_bootstrap.cc:492] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.882740  7505 ts_tablet_manager.cc:1403] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.001s
I20260709 12:30:13.884347  7171 raft_consensus.cc:2468] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:13.870676  7504 tablet_bootstrap.cc:654] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.886359  7505 raft_consensus.cc:359] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.888403  7505 raft_consensus.cc:385] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.889173  7505 raft_consensus.cc:740] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.893527  7210 leader_election.cc:304] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 273e786494a845ff87c99139442c371d
I20260709 12:30:13.891547  7505 consensus_queue.cc:260] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.895490  7444 raft_consensus.cc:2804] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.897778  7503 tablet_bootstrap.cc:492] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.898434  7503 ts_tablet_manager.cc:1403] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.068s	user 0.028s	sys 0.025s
I20260709 12:30:13.901018  7505 ts_tablet_manager.cc:1434] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:30:13.901136  7068 catalog_manager.cc:5697] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 0 to 1, leader changed from <none> to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 1 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.905217  7505 tablet_bootstrap.cc:492] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:13.905157  7068 catalog_manager.cc:5697] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 0 to 1, leader changed from <none> to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 1 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.906070  7504 tablet_bootstrap.cc:492] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.906585  7504 ts_tablet_manager.cc:1403] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 12:30:13.909900  7504 raft_consensus.cc:359] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.910619  7504 raft_consensus.cc:385] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.910950  7504 raft_consensus.cc:740] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.902873  7444 raft_consensus.cc:697] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:13.903016  7503 raft_consensus.cc:359] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.911580  7503 raft_consensus.cc:385] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.911883  7503 raft_consensus.cc:740] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.912073  7444 consensus_queue.cc:237] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.912704  7503 consensus_queue.cc:260] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.914449  7503 ts_tablet_manager.cc:1434] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:30:13.915468  7503 tablet_bootstrap.cc:492] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.917403  7505 tablet_bootstrap.cc:654] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.920619  7504 consensus_queue.cc:260] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.923277  7504 ts_tablet_manager.cc:1434] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.005s
I20260709 12:30:13.924363  7504 tablet_bootstrap.cc:492] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.928287  7503 tablet_bootstrap.cc:654] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.936703  7504 tablet_bootstrap.cc:654] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.941784  7505 tablet_bootstrap.cc:492] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.942587  7505 ts_tablet_manager.cc:1403] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.011s
I20260709 12:30:13.946878  7505 raft_consensus.cc:359] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.947785  7505 raft_consensus.cc:385] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.948077  7505 raft_consensus.cc:740] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.948875  7505 consensus_queue.cc:260] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.950345  7503 tablet_bootstrap.cc:492] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.950976  7503 ts_tablet_manager.cc:1403] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.003s
I20260709 12:30:13.951296  7505 ts_tablet_manager.cc:1434] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:13.952775  7505 tablet_bootstrap.cc:492] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:13.953333  7068 catalog_manager.cc:5697] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:13.954282  7503 raft_consensus.cc:359] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.955144  7503 raft_consensus.cc:385] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.955452  7503 raft_consensus.cc:740] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.956435  7503 consensus_queue.cc:260] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.959122  7503 ts_tablet_manager.cc:1434] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:13.961620  7505 tablet_bootstrap.cc:654] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.960700  7504 tablet_bootstrap.cc:492] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:13.962352  7504 ts_tablet_manager.cc:1403] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.002s
I20260709 12:30:13.963541  7503 tablet_bootstrap.cc:492] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:13.966730  7504 raft_consensus.cc:359] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.972550  7504 raft_consensus.cc:385] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.973122  7504 raft_consensus.cc:740] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.974057  7504 consensus_queue.cc:260] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.978767  7503 tablet_bootstrap.cc:654] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.980727  7504 ts_tablet_manager.cc:1434] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 12:30:13.985242  7504 tablet_bootstrap.cc:492] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
I20260709 12:30:13.987233  7505 tablet_bootstrap.cc:492] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:13.987792  7505 ts_tablet_manager.cc:1403] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.014s
I20260709 12:30:13.991060  7505 raft_consensus.cc:359] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.991820  7505 raft_consensus.cc:385] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.993167  7504 tablet_bootstrap.cc:654] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.994105  7503 tablet_bootstrap.cc:492] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:13.994609  7503 ts_tablet_manager.cc:1403] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.033s	user 0.004s	sys 0.008s
I20260709 12:30:13.997555  7503 raft_consensus.cc:359] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.998529  7503 raft_consensus.cc:385] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.998801  7503 raft_consensus.cc:740] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.999420  7503 consensus_queue.cc:260] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:13.999975  7444 raft_consensus.cc:493] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.992125  7505 raft_consensus.cc:740] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.000490  7504 tablet_bootstrap.cc:492] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:14.000579  7444 raft_consensus.cc:515] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.001284  7504 ts_tablet_manager.cc:1403] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 12:30:14.001912  7503 ts_tablet_manager.cc:1434] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:14.002799  7505 consensus_queue.cc:260] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.004318  7504 raft_consensus.cc:359] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.005039  7504 raft_consensus.cc:385] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.004974  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.005566  7504 raft_consensus.cc:740] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.005848  7171 raft_consensus.cc:2468] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:14.006348  7504 consensus_queue.cc:260] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.007361  7505 ts_tablet_manager.cc:1434] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:30:14.008034  7210 leader_election.cc:304] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.008591  7505 tablet_bootstrap.cc:492] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:14.008705  7504 ts_tablet_manager.cc:1434] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:14.008946  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.009757  7504 tablet_bootstrap.cc:492] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: Bootstrap starting.
W20260709 12:30:14.010516  7211 leader_election.cc:343] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.011152  7444 leader_election.cc:290] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.012063  7534 raft_consensus.cc:2804] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.012540  7534 raft_consensus.cc:493] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.012918  7534 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.014773  7503 tablet_bootstrap.cc:492] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: Bootstrap starting.
I20260709 12:30:14.016487  7504 tablet_bootstrap.cc:654] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.018184  7505 tablet_bootstrap.cc:654] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.022517  7503 tablet_bootstrap.cc:654] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.034912  7504 tablet_bootstrap.cc:492] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: No bootstrap required, opened a new log
I20260709 12:30:14.034993  7444 raft_consensus.cc:493] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.035477  7504 ts_tablet_manager.cc:1403] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: Time spent bootstrapping tablet: real 0.026s	user 0.004s	sys 0.012s
I20260709 12:30:14.035856  7444 raft_consensus.cc:515] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.038556  7504 raft_consensus.cc:359] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.039319  7504 raft_consensus.cc:385] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.039537  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.040359  7171 raft_consensus.cc:2468] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:14.040829  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.041699  7321 raft_consensus.cc:2468] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:14.042927  7513 raft_consensus.cc:493] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.043433  7513 raft_consensus.cc:515] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.043588  7211 leader_election.cc:304] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.039568  7504 raft_consensus.cc:740] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.048501  7513 leader_election.cc:290] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.050447  7513 raft_consensus.cc:2804] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.051185  7513 raft_consensus.cc:493] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.051544  7513 raft_consensus.cc:3060] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.045334  7444 leader_election.cc:290] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.049216  7504 consensus_queue.cc:260] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.053242  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972b81958a24101b9e7b7182d43abcd" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.057231  7171 raft_consensus.cc:2468] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 0.
I20260709 12:30:14.056962  7534 raft_consensus.cc:515] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.058694  7210 leader_election.cc:304] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.062911  7444 raft_consensus.cc:2804] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.063319  7444 raft_consensus.cc:493] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.063341  7504 ts_tablet_manager.cc:1434] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 12:30:14.063687  7444 raft_consensus.cc:3060] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.066074  7505 tablet_bootstrap.cc:492] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:14.063776  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972b81958a24101b9e7b7182d43abcd" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.066483  7503 tablet_bootstrap.cc:492] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: No bootstrap required, opened a new log
I20260709 12:30:14.066635  7505 ts_tablet_manager.cc:1403] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.008s
I20260709 12:30:14.067168  7503 ts_tablet_manager.cc:1403] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.020s
W20260709 12:30:14.071339  7211 leader_election.cc:343] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.073616  7545 raft_consensus.cc:493] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.074082  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:14.074184  7545 raft_consensus.cc:515] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.074851  7171 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.076207  7505 raft_consensus.cc:359] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.077335  7545 leader_election.cc:290] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.078640  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.076938  7503 raft_consensus.cc:359] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.078927  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.079515  7170 raft_consensus.cc:2393] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:14.077859  7492 raft_consensus.cc:493] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.077800  7505 raft_consensus.cc:385] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.080909  7505 raft_consensus.cc:740] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.081164  7210 leader_election.cc:304] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:14.077649  7444 raft_consensus.cc:515] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.080190  7492 raft_consensus.cc:515] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.081712  7505 consensus_queue.cc:260] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.083798  7171 raft_consensus.cc:2468] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:14.084622  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972b81958a24101b9e7b7182d43abcd" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:14.083899  7505 ts_tablet_manager.cc:1434] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:30:14.085876  7170 raft_consensus.cc:3060] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.085778  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972b81958a24101b9e7b7182d43abcd" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.086549  7321 raft_consensus.cc:3060] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.086732  7505 tablet_bootstrap.cc:492] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: Bootstrap starting.
I20260709 12:30:14.083910  7492 leader_election.cc:290] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.088804  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.078028  7513 raft_consensus.cc:515] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.092614  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8bd49811b4b6f8108cdaad13fbada" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:14.092834  7545 raft_consensus.cc:3060] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.093443  7245 raft_consensus.cc:2393] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 255aee4d52c8404fbd9da5f2aa03e346 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:14.094496  7513 leader_election.cc:290] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.094808  7138 leader_election.cc:304] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346; no voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788
I20260709 12:30:14.095706  7492 raft_consensus.cc:3060] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.096422  7505 tablet_bootstrap.cc:654] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.096722  7210 leader_election.cc:304] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.079545  7503 raft_consensus.cc:385] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.097404  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:14.097610  7513 raft_consensus.cc:2804] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.098116  7172 raft_consensus.cc:3060] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.098821  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.101260  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.101854  7317 raft_consensus.cc:3060] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.097647  7503 raft_consensus.cc:740] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.103283  7503 consensus_queue.cc:260] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
W20260709 12:30:14.108404  7211 leader_election.cc:343] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 273e786494a845ff87c99139442c371d (127.24.71.131:41695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.109328  7480 raft_consensus.cc:493] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.113271  7545 raft_consensus.cc:2749] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.113435  7480 raft_consensus.cc:515] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.115267  7492 raft_consensus.cc:2749] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.085026  7534 leader_election.cc:290] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.094738  7170 raft_consensus.cc:2468] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:14.117223  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.117659  7172 raft_consensus.cc:2468] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:14.117878  7318 raft_consensus.cc:2468] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:14.112697  7321 raft_consensus.cc:2468] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:14.118650  7480 leader_election.cc:290] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.119974  7136 leader_election.cc:304] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:14.122011  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:14.122756  7245 raft_consensus.cc:2468] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:14.124964  7210 leader_election.cc:304] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.125449  7480 raft_consensus.cc:2804] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.125909  7480 raft_consensus.cc:493] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.126150  7210 leader_election.cc:304] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.127365  7444 leader_election.cc:290] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.126614  7547 raft_consensus.cc:2804] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.127928  7505 tablet_bootstrap.cc:492] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: No bootstrap required, opened a new log
I20260709 12:30:14.126298  7480 raft_consensus.cc:3060] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.128540  7547 raft_consensus.cc:697] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.128021  7534 raft_consensus.cc:2804] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.128963  7505 ts_tablet_manager.cc:1403] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.009s
I20260709 12:30:14.129292  7534 raft_consensus.cc:697] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.129652  7547 consensus_queue.cc:237] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.130640  7534 consensus_queue.cc:237] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.132404  7492 raft_consensus.cc:493] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.133118  7492 raft_consensus.cc:515] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.135226  7317 raft_consensus.cc:2468] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 1.
I20260709 12:30:14.132421  7505 raft_consensus.cc:359] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.137283  7492 leader_election.cc:290] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.136162  7505 raft_consensus.cc:385] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.139571  7505 raft_consensus.cc:740] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.139708  7503 ts_tablet_manager.cc:1434] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.008s
I20260709 12:30:14.140347  7505 consensus_queue.cc:260] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.143051  7523 consensus_queue.cc:1048] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.144137  7513 raft_consensus.cc:697] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.145155  7513 consensus_queue.cc:237] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.145496  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ba3b888b8cf49398138f9a55bcdf7be" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:14.149132  7505 ts_tablet_manager.cc:1434] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:30:14.150799  7245 raft_consensus.cc:2468] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:14.153148  7138 leader_election.cc:304] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.159634  7492 raft_consensus.cc:2804] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.160362  7492 raft_consensus.cc:493] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.160881  7492 raft_consensus.cc:3060] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.163069  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ba3b888b8cf49398138f9a55bcdf7be" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.165622  7317 raft_consensus.cc:2468] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 0.
I20260709 12:30:14.172703  7492 raft_consensus.cc:515] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.185210  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ba3b888b8cf49398138f9a55bcdf7be" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:14.186122  7245 raft_consensus.cc:3060] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.188004  7524 raft_consensus.cc:493] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.188892  7524 raft_consensus.cc:515] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.193059  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.193921  7172 raft_consensus.cc:2393] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:14.197345  7480 raft_consensus.cc:515] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.202098  7523 consensus_queue.cc:1048] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.205331  7492 leader_election.cc:290] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.208194  7547 raft_consensus.cc:493] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.208877  7547 raft_consensus.cc:515] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.209923  7244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:14.210482  7520 consensus_queue.cc:1048] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.211725  7547 leader_election.cc:290] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.210450  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ba3b888b8cf49398138f9a55bcdf7be" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.212420  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.213202  7317 raft_consensus.cc:3060] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.213344  7170 raft_consensus.cc:2393] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4124877098414253a130e4e140806788 in current term 1: Already voted for candidate 273e786494a845ff87c99139442c371d in this term.
I20260709 12:30:14.226420  7287 leader_election.cc:304] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 273e786494a845ff87c99139442c371d; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788
I20260709 12:30:14.193811  7524 leader_election.cc:290] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:14.231952  7524 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.236892  7245 raft_consensus.cc:2468] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:14.242456  7480 leader_election.cc:290] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.242694  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.245190  7318 raft_consensus.cc:3060] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.246171  7247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:14.246949  7247 raft_consensus.cc:3060] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.249821  7520 consensus_queue.cc:1048] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.250407  7524 raft_consensus.cc:2749] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.251883  7520 raft_consensus.cc:493] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.254422  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273e786494a845ff87c99139442c371d" is_pre_election: true
I20260709 12:30:14.254765  7520 raft_consensus.cc:515] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.253247  7068 catalog_manager.cc:5697] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.256856  7211 leader_election.cc:304] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4124877098414253a130e4e140806788; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d
I20260709 12:30:14.257597  7520 leader_election.cc:290] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:14.259399  7171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346" is_pre_election: true
I20260709 12:30:14.261281  7317 raft_consensus.cc:2468] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
W20260709 12:30:14.260390  7138 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:36863, user_credentials={real_user=slave}} blocked reactor thread for 49674.1us
I20260709 12:30:14.262413  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124877098414253a130e4e140806788" is_pre_election: true
I20260709 12:30:14.262919  7138 leader_election.cc:304] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.263775  7480 raft_consensus.cc:2804] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.264294  7480 raft_consensus.cc:697] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:14.265153  7480 consensus_queue.cc:237] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.260279  7171 raft_consensus.cc:2393] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 273e786494a845ff87c99139442c371d in current term 1: Already voted for candidate 4124877098414253a130e4e140806788 in this term.
I20260709 12:30:14.268764  7547 raft_consensus.cc:3060] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.276453  7284 leader_election.cc:304] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 273e786494a845ff87c99139442c371d; no voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788
I20260709 12:30:14.278054  7520 raft_consensus.cc:3060] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:14.285686  7117 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:14.287148  7547 raft_consensus.cc:2749] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.289244  7520 raft_consensus.cc:2749] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.293947  7318 raft_consensus.cc:2468] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:14.296456  7136 leader_election.cc:304] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:14.298295  7247 raft_consensus.cc:2468] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 1.
I20260709 12:30:14.300108  7556 raft_consensus.cc:2804] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.304667  7556 raft_consensus.cc:697] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:14.305639  7556 consensus_queue.cc:237] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.307279  7068 catalog_manager.cc:5697] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.309764  7068 catalog_manager.cc:5697] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.351449  7068 catalog_manager.cc:5697] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.352782  7068 catalog_manager.cc:5697] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.368623  7490 consensus_queue.cc:1048] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.381625  7490 consensus_queue.cc:1048] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.423976  7513 consensus_queue.cc:1048] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.435781  7545 consensus_queue.cc:1048] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.536675  7545 consensus_queue.cc:1048] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.543478  7545 consensus_queue.cc:1048] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.551901  7247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78"
dest_uuid: "4124877098414253a130e4e140806788"
mode: GRACEFUL
new_leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:57386
I20260709 12:30:14.552901  7247 raft_consensus.cc:606] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Received request to transfer leadership to TS 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:14.567162  7534 consensus_queue.cc:1048] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.569636  7534 consensus_queue.cc:1048] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.572700  7317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff"
dest_uuid: "273e786494a845ff87c99139442c371d"
mode: GRACEFUL
new_leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:55710
I20260709 12:30:14.573405  7317 raft_consensus.cc:606] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Received request to transfer leadership to TS 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:14.585860  7247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6"
dest_uuid: "4124877098414253a130e4e140806788"
mode: GRACEFUL
new_leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:57386
I20260709 12:30:14.586584  7247 raft_consensus.cc:606] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Received request to transfer leadership to TS 255aee4d52c8404fbd9da5f2aa03e346
I20260709 12:30:14.596176  7172 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
mode: GRACEFUL
new_leader_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:60400
I20260709 12:30:14.597064  7172 raft_consensus.cc:606] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Received request to transfer leadership to TS 4124877098414253a130e4e140806788
I20260709 12:30:14.616560  7317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb"
dest_uuid: "273e786494a845ff87c99139442c371d"
mode: GRACEFUL
new_leader_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:55710
I20260709 12:30:14.617341  7317 raft_consensus.cc:606] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Received request to transfer leadership to TS 4124877098414253a130e4e140806788
I20260709 12:30:14.644272  7545 consensus_queue.cc:1048] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.647912  7556 consensus_queue.cc:1048] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.652779  7513 raft_consensus.cc:993] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: : Instructing follower 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:14.653462  7513 raft_consensus.cc:1081] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Signalling peer 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:14.660580  7172 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:60378
I20260709 12:30:14.661639  7513 consensus_queue.cc:1048] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.664439  7523 raft_consensus.cc:993] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: : Instructing follower 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:14.665033  7523 raft_consensus.cc:1081] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Signalling peer 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:14.661643  7172 raft_consensus.cc:493] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:14.666486  7172 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.667618  7247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb"
dest_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:51008
I20260709 12:30:14.668298  7247 raft_consensus.cc:493] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:14.668671  7247 raft_consensus.cc:3060] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.675458  7555 consensus_queue.cc:1048] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.677091  7172 raft_consensus.cc:515] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.678290  7170 raft_consensus.cc:1240] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Rejecting Update request from peer 4124877098414253a130e4e140806788 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:14.679483  7172 leader_election.cc:290] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.681267  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:14.682250  7246 raft_consensus.cc:3055] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:14.682613  7246 raft_consensus.cc:740] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.683651  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.683450  7246 consensus_queue.cc:260] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.684263  7318 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.684417  7444 consensus_queue.cc:1059] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.688151  7246 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.703209  7247 raft_consensus.cc:515] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.706806  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:14.707509  7170 raft_consensus.cc:3060] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.708607  7318 raft_consensus.cc:2468] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:14.709123  7246 raft_consensus.cc:2468] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:14.709563  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b4fec5921e4dafa914dc0a63a80bcb" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.709893  7136 leader_election.cc:304] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:14.710217  7317 raft_consensus.cc:3055] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:14.710513  7317 raft_consensus.cc:740] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:14.710842  7555 raft_consensus.cc:2804] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:14.711364  7555 raft_consensus.cc:697] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:14.712132  7317 consensus_queue.cc:260] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.712198  7555 consensus_queue.cc:237] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.713433  7317 raft_consensus.cc:3060] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.716382  7247 leader_election.cc:290] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.716789  7170 raft_consensus.cc:2468] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:14.718005  7210 leader_election.cc:304] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.718907  7444 raft_consensus.cc:2804] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:14.719336  7444 raft_consensus.cc:697] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 2 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.717850  7555 consensus_queue.cc:1048] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.719978  7444 consensus_queue.cc:237] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.725867  7317 raft_consensus.cc:2468] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:14.725478  7066 catalog_manager.cc:5697] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 1 to 2, leader changed from 4124877098414253a130e4e140806788 (127.24.71.130) to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.731879  7068 catalog_manager.cc:5697] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 reported cstate change: term changed from 1 to 2, leader changed from 273e786494a845ff87c99139442c371d (127.24.71.131) to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.732969  7555 consensus_queue.cc:1048] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.753796  7555 raft_consensus.cc:993] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: : Instructing follower 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:14.754770  7480 raft_consensus.cc:1081] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Signalling peer 4124877098414253a130e4e140806788 to start an election
I20260709 12:30:14.757205  7247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0"
dest_uuid: "4124877098414253a130e4e140806788"
 from {username='slave'} at 127.0.0.1:51022
I20260709 12:30:14.757932  7247 raft_consensus.cc:493] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:14.758337  7247 raft_consensus.cc:3060] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.767908  7247 raft_consensus.cc:515] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.768942  7245 raft_consensus.cc:1240] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Rejecting Update request from peer 255aee4d52c8404fbd9da5f2aa03e346 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:14.770767  7170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:14.771018  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf09a69e231c4890b4a80587cb30a4d0" candidate_uuid: "4124877098414253a130e4e140806788" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.771459  7170 raft_consensus.cc:3055] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:14.771631  7317 raft_consensus.cc:3060] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.771862  7170 raft_consensus.cc:740] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:14.771653  7492 consensus_queue.cc:1059] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.773133  7247 leader_election.cc:290] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.773216  7170 consensus_queue.cc:260] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.774829  7170 raft_consensus.cc:3060] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.778626  7317 raft_consensus.cc:2468] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:14.780155  7211 leader_election.cc:304] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.781005  7444 raft_consensus.cc:2804] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:14.781438  7444 raft_consensus.cc:697] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 2 LEADER]: Becoming Leader. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.782038  7170 raft_consensus.cc:2468] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4124877098414253a130e4e140806788 in term 2.
I20260709 12:30:14.782071  7444 consensus_queue.cc:237] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.790401  7068 catalog_manager.cc:5697] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 reported cstate change: term changed from 1 to 2, leader changed from 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129) to 4124877098414253a130e4e140806788 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "4124877098414253a130e4e140806788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.849557  7520 raft_consensus.cc:993] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: : Instructing follower 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:14.850100  7523 raft_consensus.cc:1081] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Signalling peer 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:14.851852  7170 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:60386
I20260709 12:30:14.852478  7170 raft_consensus.cc:493] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:14.852798  7170 raft_consensus.cc:3060] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.857705  7170 raft_consensus.cc:515] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.859512  7170 leader_election.cc:290] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.860589  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.860679  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99753f55a73849b5946167ef4bb928ff" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:14.861392  7317 raft_consensus.cc:3055] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:14.861483  7245 raft_consensus.cc:3060] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.861722  7317 raft_consensus.cc:740] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:14.863188  7317 consensus_queue.cc:260] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.864161  7317 raft_consensus.cc:3060] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.867657  7245 raft_consensus.cc:2468] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:14.868952  7138 leader_election.cc:304] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.869309  7317 raft_consensus.cc:2468] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:14.869845  7492 raft_consensus.cc:2804] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:14.870412  7492 raft_consensus.cc:697] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:14.871219  7492 consensus_queue.cc:237] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.878436  7068 catalog_manager.cc:5697] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 1 to 2, leader changed from 273e786494a845ff87c99139442c371d (127.24.71.131) to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.933880  7513 raft_consensus.cc:993] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: : Instructing follower 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:14.934501  7444 raft_consensus.cc:1081] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Signalling peer 255aee4d52c8404fbd9da5f2aa03e346 to start an election
I20260709 12:30:14.936473  7170 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
 from {username='slave'} at 127.0.0.1:60378
I20260709 12:30:14.937191  7170 raft_consensus.cc:493] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:14.937566  7170 raft_consensus.cc:3060] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.943655  7170 raft_consensus.cc:515] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.945159  7170 leader_election.cc:290] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Requested vote from peers 4124877098414253a130e4e140806788 (127.24.71.130:36863), 273e786494a845ff87c99139442c371d (127.24.71.131:41695)
I20260709 12:30:14.945981  7245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:14.946300  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd7b1fae3ec4166ae6c100580a91a78" candidate_uuid: "255aee4d52c8404fbd9da5f2aa03e346" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "273e786494a845ff87c99139442c371d"
I20260709 12:30:14.946570  7245 raft_consensus.cc:3055] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:14.946906  7317 raft_consensus.cc:3060] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.946925  7245 raft_consensus.cc:740] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:14.947923  7245 consensus_queue.cc:260] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.948904  7245 raft_consensus.cc:3060] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:14.953644  7245 raft_consensus.cc:2468] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:14.953852  7317 raft_consensus.cc:2468] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 255aee4d52c8404fbd9da5f2aa03e346 in term 2.
I20260709 12:30:14.954712  7138 leader_election.cc:304] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:14.955751  7492 raft_consensus.cc:2804] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:14.956269  7492 raft_consensus.cc:697] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Leader. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:14.957163  7492 consensus_queue.cc:237] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:14.964145  7068 catalog_manager.cc:5697] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 reported cstate change: term changed from 1 to 2, leader changed from 4124877098414253a130e4e140806788 (127.24.71.130) to 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "255aee4d52c8404fbd9da5f2aa03e346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.124121  7317 raft_consensus.cc:1275] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.125945  7534 consensus_queue.cc:1048] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.135578  7170 raft_consensus.cc:1275] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.137341  7534 consensus_queue.cc:1048] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.230795  7318 raft_consensus.cc:1275] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.232355  7492 consensus_queue.cc:1048] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.241715  7247 raft_consensus.cc:1275] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.243467  7556 consensus_queue.cc:1048] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.259202  7318 raft_consensus.cc:1275] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.260869  7534 consensus_queue.cc:1048] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.269739  7170 raft_consensus.cc:1275] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 4124877098414253a130e4e140806788: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.271160  7513 consensus_queue.cc:1048] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.314225  7318 raft_consensus.cc:1275] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.315608  7556 consensus_queue.cc:1048] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.325106  7247 raft_consensus.cc:1275] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.327046  7492 consensus_queue.cc:1048] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.369513  7247 raft_consensus.cc:1275] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.371258  7492 consensus_queue.cc:1048] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:15.380724  7318 raft_consensus.cc:1275] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Refusing update from remote peer 255aee4d52c8404fbd9da5f2aa03e346: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:15.382916  7580 consensus_queue.cc:1048] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:17.628340  7068 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50668:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:30:17.649247  7068 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50668:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:30:17.677338 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:30:17.871801  7136 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41695, user_credentials={real_user=slave}} blocked reactor thread for 45303us
W20260709 12:30:17.906534  7211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.131:41695, user_credentials={real_user=slave}} blocked reactor thread for 60317.5us
I20260709 12:30:17.929989  7627 tablet.cc:1724] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:17.976332  7068 catalog_manager.cc:5789] TS 4124877098414253a130e4e140806788 (127.24.71.130:36863) does not have the latest schema for tablet 491a5616a3df46adba0b22b9731ad2df (table soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]). Expected version 1 got 0
I20260709 12:30:17.981599  7068 catalog_manager.cc:5789] TS 4124877098414253a130e4e140806788 (127.24.71.130:36863) does not have the latest schema for tablet b4b4fec5921e4dafa914dc0a63a80bcb (table soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]). Expected version 1 got 0
I20260709 12:30:17.982399  7068 catalog_manager.cc:5789] TS 4124877098414253a130e4e140806788 (127.24.71.130:36863) does not have the latest schema for tablet bf09a69e231c4890b4a80587cb30a4d0 (table soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]). Expected version 1 got 0
I20260709 12:30:17.983495  7068 catalog_manager.cc:5789] TS 4124877098414253a130e4e140806788 (127.24.71.130:36863) does not have the latest schema for tablet 6972b81958a24101b9e7b7182d43abcd (table soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6]). Expected version 1 got 0
I20260709 12:30:18.038158  7642 tablet.cc:1724] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:18.039228  7625 log.cc:927] Time spent T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:30:18.066946  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.130:36863, user_credentials={real_user=slave}} blocked reactor thread for 52626.2us
I20260709 12:30:18.107705  7244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df"
dest_uuid: "4124877098414253a130e4e140806788"
mode: GRACEFUL
new_leader_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:57392
I20260709 12:30:18.027302  7634 tablet.cc:1724] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.067875  7641 tablet.cc:1724] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.087579  7640 tablet.cc:1724] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:18.162708  7347 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:0bd7b1fae3ec4166ae6c100580a91a78)
W20260709 12:30:18.157073  7198 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:0bd7b1fae3ec4166ae6c100580a91a78)
I20260709 12:30:18.135816  7244 raft_consensus.cc:606] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 LEADER]: Received request to transfer leadership to TS 273e786494a845ff87c99139442c371d
W20260709 12:30:18.171289  7198 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:0bd7b1fae3ec4166ae6c100580a91a78)
I20260709 12:30:18.133689  7656 tablet.cc:1724] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.159878  7646 tablet.cc:1724] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:18.181177  7648 log.cc:927] Time spent T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:30:18.131201  7650 tablet.cc:1724] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.185030  7659 tablet.cc:1724] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.188146  7654 tablet.cc:1724] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.189074  7637 raft_consensus.cc:993] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: : Instructing follower 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.189659  7637 raft_consensus.cc:1081] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 LEADER]: Signalling peer 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.191969  7320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df"
dest_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:55694
I20260709 12:30:18.192616  7320 raft_consensus.cc:493] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:30:18.192878  7647 log.cc:927] Time spent T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:30:18.193035  7320 raft_consensus.cc:3060] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:18.134619  7658 tablet.cc:1724] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.196727  7642 tablet.cc:1724] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.201495  7653 tablet.cc:1724] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.213470  7646 tablet.cc:1724] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.216418  7641 tablet.cc:1724] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.217615  7627 tablet.cc:1724] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.216418  7634 tablet.cc:1724] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.227720  7650 tablet.cc:1724] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.231856  7640 tablet.cc:1724] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:18.227497  7655 log.cc:927] Time spent T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:30:18.229126  7654 tablet.cc:1724] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.250144  7320 raft_consensus.cc:515] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.253296  7320 leader_election.cc:290] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 2 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:18.257608  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:18.274813  7246 raft_consensus.cc:3055] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:18.274410  7659 tablet.cc:1724] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.259128  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491a5616a3df46adba0b22b9731ad2df" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:18.274897  7653 tablet.cc:1724] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.275818  7172 raft_consensus.cc:3060] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:18.260084  7658 tablet.cc:1724] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.263617  7627 tablet.cc:1724] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.275214  7246 raft_consensus.cc:740] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:18.278434  7246 consensus_queue.cc:260] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.279644  7246 raft_consensus.cc:3060] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:18.261911  7640 tablet.cc:1724] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.285024  7172 raft_consensus.cc:2468] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 2.
I20260709 12:30:18.286511  7284 leader_election.cc:304] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:18.287052  7246 raft_consensus.cc:2468] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 2.
I20260709 12:30:18.287668  7649 raft_consensus.cc:2804] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:18.288317  7649 raft_consensus.cc:697] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 2 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:18.289191  7649 consensus_queue.cc:237] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.297003  7656 tablet.cc:1724] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.307292  7653 tablet.cc:1724] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:18.321262  7066 catalog_manager.cc:5697] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 1 to 2, leader changed from 4124877098414253a130e4e140806788 (127.24.71.130) to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 2 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:18.326793  7172 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
mode: GRACEFUL
new_leader_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:50364
I20260709 12:30:18.327538  7172 raft_consensus.cc:606] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Received request to transfer leadership to TS 273e786494a845ff87c99139442c371d
I20260709 12:30:18.329069 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:30:18.329732 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:18.337963  7172 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60"
dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
mode: GRACEFUL
new_leader_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:50364
I20260709 12:30:18.338629  7172 raft_consensus.cc:606] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Received request to transfer leadership to TS 273e786494a845ff87c99139442c371d
I20260709 12:30:18.341863  7246 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f"
dest_uuid: "4124877098414253a130e4e140806788"
mode: GRACEFUL
new_leader_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:57392
I20260709 12:30:18.342422  7246 raft_consensus.cc:606] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Received request to transfer leadership to TS 273e786494a845ff87c99139442c371d
I20260709 12:30:18.344044 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:18.344717 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:18.349867  7066 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509: soft_deleted_table [id=e63b90661def40f5bd1125b14a12d0a6] alter complete (version 1)
I20260709 12:30:18.415671  7612 raft_consensus.cc:993] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: : Instructing follower 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.416364  7612 raft_consensus.cc:1081] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Signalling peer 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.418035  7320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f"
dest_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:55694
I20260709 12:30:18.418537  7320 raft_consensus.cc:493] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:18.418864  7320 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.424181  7320 raft_consensus.cc:515] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.426409  7628 raft_consensus.cc:993] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: : Instructing follower 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.426638  7320 leader_election.cc:290] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 3 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:18.427075  7555 raft_consensus.cc:1081] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Signalling peer 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.427052  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:18.428015  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e384731a794542d3aa4feec313f2a98f" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:18.428488  7172 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.428673  7246 raft_consensus.cc:3055] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:18.429096  7246 raft_consensus.cc:740] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4124877098414253a130e4e140806788, State: Running, Role: LEADER
I20260709 12:30:18.429283  7320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6"
dest_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:55700
I20260709 12:30:18.430032  7320 raft_consensus.cc:493] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:18.429988  7246 consensus_queue.cc:260] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.430644  7320 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.431432  7246 raft_consensus.cc:3060] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.436832  7246 raft_consensus.cc:2468] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 3.
I20260709 12:30:18.436709  7172 raft_consensus.cc:2468] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 3.
I20260709 12:30:18.438153  7287 leader_election.cc:304] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:18.438354  7320 raft_consensus.cc:515] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.439019  7649 raft_consensus.cc:2804] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:18.439507  7649 raft_consensus.cc:697] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 3 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:18.440690  7320 leader_election.cc:290] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 3 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:18.440657  7649 consensus_queue.cc:237] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.441282  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:18.441963  7172 raft_consensus.cc:3055] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:18.442073  7246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d727a6836cf41c0a6d6a557bf425bb6" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:18.442446  7172 raft_consensus.cc:740] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:18.442793  7246 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.443356  7172 consensus_queue.cc:260] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.444499  7172 raft_consensus.cc:3060] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.451803  7172 raft_consensus.cc:2468] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 3.
I20260709 12:30:18.452267  7246 raft_consensus.cc:2468] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 3.
I20260709 12:30:18.453120  7284 leader_election.cc:304] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 255aee4d52c8404fbd9da5f2aa03e346, 273e786494a845ff87c99139442c371d; no voters: 
I20260709 12:30:18.454227  7649 raft_consensus.cc:2804] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:18.454725  7649 raft_consensus.cc:697] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 3 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:18.455531  7649 consensus_queue.cc:237] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.458200  7068 catalog_manager.cc:5697] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 2 to 3, leader changed from 4124877098414253a130e4e140806788 (127.24.71.130) to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 3 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:18.471640  7068 catalog_manager.cc:5697] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 2 to 3, leader changed from 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129) to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 3 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:18.626801  7628 raft_consensus.cc:993] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: : Instructing follower 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.627305  7555 raft_consensus.cc:1081] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Signalling peer 273e786494a845ff87c99139442c371d to start an election
I20260709 12:30:18.628798  7320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60"
dest_uuid: "273e786494a845ff87c99139442c371d"
 from {username='slave'} at 127.0.0.1:55700
I20260709 12:30:18.629313  7320 raft_consensus.cc:493] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:18.629630  7320 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.634850  7320 raft_consensus.cc:515] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.636852  7320 leader_election.cc:290] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 3 election: Requested vote from peers 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067), 4124877098414253a130e4e140806788 (127.24.71.130:36863)
I20260709 12:30:18.637610  7172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "255aee4d52c8404fbd9da5f2aa03e346"
I20260709 12:30:18.638304  7172 raft_consensus.cc:3055] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:18.638388  7244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9b66df1b924fa0831f6be36ed78a60" candidate_uuid: "273e786494a845ff87c99139442c371d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4124877098414253a130e4e140806788"
I20260709 12:30:18.638700  7172 raft_consensus.cc:740] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 255aee4d52c8404fbd9da5f2aa03e346, State: Running, Role: LEADER
I20260709 12:30:18.639084  7244 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.640223  7172 consensus_queue.cc:260] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.641678  7172 raft_consensus.cc:3060] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:18.647341  7244 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 3.
I20260709 12:30:18.648706  7287 leader_election.cc:304] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e786494a845ff87c99139442c371d, 4124877098414253a130e4e140806788; no voters: 
I20260709 12:30:18.649199  7172 raft_consensus.cc:2468] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273e786494a845ff87c99139442c371d in term 3.
I20260709 12:30:18.649580  7649 raft_consensus.cc:2804] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:18.650198  7649 raft_consensus.cc:697] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 3 LEADER]: Becoming Leader. State: Replica: 273e786494a845ff87c99139442c371d, State: Running, Role: LEADER
I20260709 12:30:18.650938  7649 consensus_queue.cc:237] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } }
I20260709 12:30:18.668030  7068 catalog_manager.cc:5697] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d reported cstate change: term changed from 2 to 3, leader changed from 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129) to 273e786494a845ff87c99139442c371d (127.24.71.131). New cstate: current_term: 3 leader_uuid: "273e786494a845ff87c99139442c371d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273e786494a845ff87c99139442c371d" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 41695 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:18.705981  7172 raft_consensus.cc:1275] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:30:18.708843  7649 consensus_queue.cc:1048] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:18.719305  7244 raft_consensus.cc:1275] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:30:18.722949  7649 consensus_queue.cc:1048] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:18.861428  7172 raft_consensus.cc:1275] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:30:18.862896  7649 consensus_queue.cc:1048] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:30:18.872254  7244 raft_consensus.cc:1275] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:30:18.873798  7667 consensus_queue.cc:1048] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:30:18.938740  7244 raft_consensus.cc:1275] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:18.940140  7667 consensus_queue.cc:1048] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:18.949631  7170 raft_consensus.cc:1275] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:18.951541  7523 consensus_queue.cc:1048] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:19.145258  7244 raft_consensus.cc:1275] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:19.146783  7523 consensus_queue.cc:1048] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124877098414253a130e4e140806788" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:19.156888  7170 raft_consensus.cc:1275] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Refusing update from remote peer 273e786494a845ff87c99139442c371d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:19.162130  7523 consensus_queue.cc:1048] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [LEADER]: Connected to new peer: Peer: permanent_uuid: "255aee4d52c8404fbd9da5f2aa03e346" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:19.345893 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:19.351400 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:19.351872 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:19.356805 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:19.357249 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:20.359087 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:20.371116 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:20.371606 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:20.380836 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:20.381397 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:21.382786 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:21.387825 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:21.388295 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:21.392799 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:21.393215 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:22.394229 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:22.399477 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:22.399919 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:22.404613 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:22.405025 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:23.406148 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:23.412948 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:23.413419 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:23.418388 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:23.418784 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:24.419907 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:24.425755 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:24.426195 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:24.430833 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:24.431353 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:25.432744 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:25.440654 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:25.441102 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:25.445952 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:25.446357 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:26.447341 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:26.453321 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:26.453758 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:26.458539 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:26.458958 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:27.459947 24862 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:27.465183 24862 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:27.465678 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:27.471633 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:27.472076 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:28.538686 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
W20260709 12:30:28.587296  7284 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 20 similar messages]
W20260709 12:30:28.591974  7284 consensus_peers.cc:597] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.593318 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:28.594583 24862 tablet_replica.cc:333] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.595839 24862 raft_consensus.cc:2243] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.597159 24862 raft_consensus.cc:2272] T 6972b81958a24101b9e7b7182d43abcd P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:28.599388  7210 consensus_peers.cc:597] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.600769 24862 tablet_replica.cc:333] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.601552 24862 raft_consensus.cc:2243] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.602113 24862 raft_consensus.cc:2272] T bf09a69e231c4890b4a80587cb30a4d0 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.604831 24862 tablet_replica.cc:333] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.605453 24862 raft_consensus.cc:2243] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.605942 24862 raft_consensus.cc:2272] T 8d727a6836cf41c0a6d6a557bf425bb6 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.608801 24862 tablet_replica.cc:333] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.609486 24862 raft_consensus.cc:2243] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.610013 24862 raft_consensus.cc:2272] T 491a5616a3df46adba0b22b9731ad2df P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.612437 24862 tablet_replica.cc:333] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.613055 24862 raft_consensus.cc:2243] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.613988 24862 raft_consensus.cc:2272] T 99753f55a73849b5946167ef4bb928ff P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.616482 24862 tablet_replica.cc:333] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.617111 24862 raft_consensus.cc:2243] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.617676 24862 raft_consensus.cc:2272] T 5e3992fbed914c4787d3089d7932f264 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.620333 24862 tablet_replica.cc:333] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.620982 24862 raft_consensus.cc:2243] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.621878 24862 raft_consensus.cc:2272] T 0bd7b1fae3ec4166ae6c100580a91a78 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.624472 24862 tablet_replica.cc:333] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.625097 24862 raft_consensus.cc:2243] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.625787 24862 raft_consensus.cc:2272] T b4b4fec5921e4dafa914dc0a63a80bcb P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.629252 24862 tablet_replica.cc:333] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.630121 24862 raft_consensus.cc:2243] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.630896 24862 raft_consensus.cc:2272] T 70b8bd49811b4b6f8108cdaad13fbada P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.634461 24862 tablet_replica.cc:333] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.635330 24862 raft_consensus.cc:2243] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.635929 24862 raft_consensus.cc:2272] T 701e4ba57f074d4b97828ec0d9c6a705 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.639961 24862 tablet_replica.cc:333] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.640784 24862 raft_consensus.cc:2243] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.642177 24862 raft_consensus.cc:2272] T 239fc23a25f8480b97f6b032d88d9105 P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.645026 24862 tablet_replica.cc:333] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.645711 24862 raft_consensus.cc:2243] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.646275 24862 raft_consensus.cc:2272] T ca42f5a8e6e14af896744c41a9c2d65f P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.648998 24862 tablet_replica.cc:333] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.649716 24862 raft_consensus.cc:2243] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.650331 24862 raft_consensus.cc:2272] T aa90fd87ec3e44068275986fa849c02e P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.655395 24862 tablet_replica.cc:333] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.656687 24862 raft_consensus.cc:2243] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.658433 24862 raft_consensus.cc:2272] T 2ba3b888b8cf49398138f9a55bcdf7be P 255aee4d52c8404fbd9da5f2aa03e346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.662917 24862 tablet_replica.cc:333] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.663895 24862 raft_consensus.cc:2243] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.665347 24862 raft_consensus.cc:2272] T 64e2d5ba97c54e5a9130875f239c8833 P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.669226 24862 tablet_replica.cc:333] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.670118 24862 raft_consensus.cc:2243] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.670761 24862 raft_consensus.cc:2272] T 9f9b66df1b924fa0831f6be36ed78a60 P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.673861 24862 tablet_replica.cc:333] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.674633 24862 raft_consensus.cc:2243] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.675922 24862 raft_consensus.cc:2272] T 70bd6da3b3924e2896be369b7e6a253c P 255aee4d52c8404fbd9da5f2aa03e346 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.679059 24862 tablet_replica.cc:333] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346: stopping tablet replica
I20260709 12:30:28.679637 24862 raft_consensus.cc:2243] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.680266 24862 raft_consensus.cc:2272] T e384731a794542d3aa4feec313f2a98f P 255aee4d52c8404fbd9da5f2aa03e346 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:28.700875  7284 consensus_peers.cc:597] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:28.706950  7210 consensus_peers.cc:597] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.733307 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
W20260709 12:30:28.762712  7210 consensus_peers.cc:597] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:33067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:28.763381  7210 consensus_peers.cc:597] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:33067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.786106 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
W20260709 12:30:28.800002  7284 consensus_peers.cc:597] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:33067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:28.800004  7287 consensus_peers.cc:597] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d -> Peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Couldn't send request to peer 4124877098414253a130e4e140806788. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.825683 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:28.826669 24862 tablet_replica.cc:333] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.827956 24862 raft_consensus.cc:2243] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.829098 24862 raft_consensus.cc:2272] T b4b4fec5921e4dafa914dc0a63a80bcb P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.831614 24862 tablet_replica.cc:333] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.832396 24862 raft_consensus.cc:2243] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.832856 24862 raft_consensus.cc:2272] T 99753f55a73849b5946167ef4bb928ff P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.835193 24862 tablet_replica.cc:333] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.835760 24862 raft_consensus.cc:2243] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.836328 24862 raft_consensus.cc:2272] T 491a5616a3df46adba0b22b9731ad2df P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.838625 24862 tablet_replica.cc:333] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.839123 24862 raft_consensus.cc:2243] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.839877 24862 raft_consensus.cc:2272] T bf09a69e231c4890b4a80587cb30a4d0 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.842136 24862 tablet_replica.cc:333] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.842762 24862 raft_consensus.cc:2243] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.843279 24862 raft_consensus.cc:2272] T 8d727a6836cf41c0a6d6a557bf425bb6 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.845916 24862 tablet_replica.cc:333] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.846720 24862 raft_consensus.cc:2243] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.847718 24862 raft_consensus.cc:2272] T 5e3992fbed914c4787d3089d7932f264 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.850359 24862 tablet_replica.cc:333] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.851059 24862 raft_consensus.cc:2243] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.851667 24862 raft_consensus.cc:2272] T 0bd7b1fae3ec4166ae6c100580a91a78 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.854305 24862 tablet_replica.cc:333] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.854910 24862 raft_consensus.cc:2243] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.856429 24862 raft_consensus.cc:2272] T 6972b81958a24101b9e7b7182d43abcd P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:28.859477  7287 consensus_peers.cc:597] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d -> Peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Couldn't send request to peer 4124877098414253a130e4e140806788. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.860188 24862 tablet_replica.cc:333] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.860966 24862 raft_consensus.cc:2243] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.861558 24862 raft_consensus.cc:2272] T 70b8bd49811b4b6f8108cdaad13fbada P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.863929 24862 tablet_replica.cc:333] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.864562 24862 raft_consensus.cc:2243] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.865129 24862 raft_consensus.cc:2272] T 701e4ba57f074d4b97828ec0d9c6a705 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.867496 24862 tablet_replica.cc:333] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.868059 24862 raft_consensus.cc:2243] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.868542 24862 raft_consensus.cc:2272] T 239fc23a25f8480b97f6b032d88d9105 P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.870832 24862 tablet_replica.cc:333] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.871464 24862 raft_consensus.cc:2243] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.871915 24862 raft_consensus.cc:2272] T e384731a794542d3aa4feec313f2a98f P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.874063 24862 tablet_replica.cc:333] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.874845 24862 raft_consensus.cc:2243] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.875340 24862 raft_consensus.cc:2272] T 70bd6da3b3924e2896be369b7e6a253c P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.877645 24862 tablet_replica.cc:333] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.878216 24862 raft_consensus.cc:2243] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.879101 24862 raft_consensus.cc:2272] T aa90fd87ec3e44068275986fa849c02e P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.881477 24862 tablet_replica.cc:333] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.882105 24862 raft_consensus.cc:2243] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:28.882959 24862 raft_consensus.cc:2272] T ca42f5a8e6e14af896744c41a9c2d65f P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.886088 24862 tablet_replica.cc:333] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.886718 24862 raft_consensus.cc:2243] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.887269 24862 raft_consensus.cc:2272] T 9f9b66df1b924fa0831f6be36ed78a60 P 4124877098414253a130e4e140806788 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.890241 24862 tablet_replica.cc:333] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.890844 24862 raft_consensus.cc:2243] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.891398 24862 raft_consensus.cc:2272] T 2ba3b888b8cf49398138f9a55bcdf7be P 4124877098414253a130e4e140806788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:28.894061 24862 tablet_replica.cc:333] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788: stopping tablet replica
I20260709 12:30:28.894639 24862 raft_consensus.cc:2243] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:28.895200 24862 raft_consensus.cc:2272] T 64e2d5ba97c54e5a9130875f239c8833 P 4124877098414253a130e4e140806788 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:28.903203  7284 consensus_peers.cc:597] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:33067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:28.906559  7287 consensus_peers.cc:597] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d -> Peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Couldn't send request to peer 4124877098414253a130e4e140806788. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:28.948630 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
W20260709 12:30:28.962744  7287 consensus_peers.cc:597] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d -> Peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Couldn't send request to peer 4124877098414253a130e4e140806788. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:36863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:28.987708  7284 consensus_peers.cc:597] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d -> Peer 255aee4d52c8404fbd9da5f2aa03e346 (127.24.71.129:33067): Couldn't send request to peer 255aee4d52c8404fbd9da5f2aa03e346. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:33067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:28.997928  7287 consensus_peers.cc:597] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d -> Peer 4124877098414253a130e4e140806788 (127.24.71.130:36863): Couldn't send request to peer 4124877098414253a130e4e140806788. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:36863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:29.001067 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
I20260709 12:30:29.040566 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:29.041430 24862 tablet_replica.cc:333] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.042188 24862 raft_consensus.cc:2243] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.042829 24862 raft_consensus.cc:2272] T 6972b81958a24101b9e7b7182d43abcd P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.045264 24862 tablet_replica.cc:333] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.045915 24862 raft_consensus.cc:2243] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.046777 24862 raft_consensus.cc:2272] T 491a5616a3df46adba0b22b9731ad2df P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.049005 24862 tablet_replica.cc:333] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.049625 24862 raft_consensus.cc:2243] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.050144 24862 raft_consensus.cc:2272] T 99753f55a73849b5946167ef4bb928ff P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.052455 24862 tablet_replica.cc:333] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.053046 24862 raft_consensus.cc:2243] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.053541 24862 raft_consensus.cc:2272] T b4b4fec5921e4dafa914dc0a63a80bcb P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.056068 24862 tablet_replica.cc:333] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.056675 24862 raft_consensus.cc:2243] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.057633 24862 raft_consensus.cc:2272] T 8d727a6836cf41c0a6d6a557bf425bb6 P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.059628 24862 tablet_replica.cc:333] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.060223 24862 raft_consensus.cc:2243] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.060797 24862 raft_consensus.cc:2272] T bf09a69e231c4890b4a80587cb30a4d0 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.062902 24862 tablet_replica.cc:333] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.063484 24862 raft_consensus.cc:2243] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.064441 24862 raft_consensus.cc:2272] T 70b8bd49811b4b6f8108cdaad13fbada P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.066599 24862 tablet_replica.cc:333] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.067333 24862 raft_consensus.cc:2243] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.068166 24862 raft_consensus.cc:2272] T 701e4ba57f074d4b97828ec0d9c6a705 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.070400 24862 tablet_replica.cc:333] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.071022 24862 raft_consensus.cc:2243] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.071521 24862 raft_consensus.cc:2272] T ca42f5a8e6e14af896744c41a9c2d65f P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.073774 24862 tablet_replica.cc:333] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.074330 24862 raft_consensus.cc:2243] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.074801 24862 raft_consensus.cc:2272] T aa90fd87ec3e44068275986fa849c02e P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.077000 24862 tablet_replica.cc:333] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.077562 24862 raft_consensus.cc:2243] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.078047 24862 raft_consensus.cc:2272] T 70bd6da3b3924e2896be369b7e6a253c P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.080163 24862 tablet_replica.cc:333] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.080710 24862 raft_consensus.cc:2243] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.081569 24862 raft_consensus.cc:2272] T e384731a794542d3aa4feec313f2a98f P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.083842 24862 tablet_replica.cc:333] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.084375 24862 raft_consensus.cc:2243] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.084837 24862 raft_consensus.cc:2272] T 239fc23a25f8480b97f6b032d88d9105 P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.086862 24862 tablet_replica.cc:333] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.087383 24862 raft_consensus.cc:2243] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.087859 24862 raft_consensus.cc:2272] T 0bd7b1fae3ec4166ae6c100580a91a78 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.089917 24862 tablet_replica.cc:333] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.090487 24862 raft_consensus.cc:2243] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.090953 24862 raft_consensus.cc:2272] T 5e3992fbed914c4787d3089d7932f264 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.093209 24862 tablet_replica.cc:333] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.093698 24862 raft_consensus.cc:2243] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.094182 24862 raft_consensus.cc:2272] T 2ba3b888b8cf49398138f9a55bcdf7be P 273e786494a845ff87c99139442c371d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.096210 24862 tablet_replica.cc:333] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.096746 24862 raft_consensus.cc:2243] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:29.097225 24862 raft_consensus.cc:2272] T 64e2d5ba97c54e5a9130875f239c8833 P 273e786494a845ff87c99139442c371d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.099526 24862 tablet_replica.cc:333] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d: stopping tablet replica
I20260709 12:30:29.100004 24862 raft_consensus.cc:2243] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.100944 24862 raft_consensus.cc:2272] T 9f9b66df1b924fa0831f6be36ed78a60 P 273e786494a845ff87c99139442c371d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.144910 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:30:29.184849 24862 master.cc:562] Master@127.24.71.190:42049 shutting down...
I20260709 12:30:29.215374 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:29.215975 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:29.216566 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P 04e6ae9d4b584aa6937959e5840cc509: stopping tablet replica
I20260709 12:30:29.238487 24862 master.cc:584] Master@127.24.71.190:42049 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (23219 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:30:29.297974 24862 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.71.190:37433
I20260709 12:30:29.299194 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:29.305987  7699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:29.305987  7700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:29.306852  7702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:29.307503 24862 server_base.cc:1061] running on GCE node
I20260709 12:30:29.308848 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:29.309052 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:29.309171 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600229309159 us; error 0 us; skew 500 ppm
I20260709 12:30:29.309782 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:29.312781 24862 webserver.cc:533] Webserver started at http://127.24.71.190:41107/ using document root <none> and password file <none>
I20260709 12:30:29.313344 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:29.313550 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:29.313880 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:29.315208 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/master-0-root/instance:
uuid: "e168a38330e14236bcb04bbd50b470bb"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.320350 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.007s
I20260709 12:30:29.324049  7707 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.325093 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:29.325518 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/master-0-root
uuid: "e168a38330e14236bcb04bbd50b470bb"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.325857 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:29.341159 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:29.342629 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:29.389464 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.190:37433
I20260709 12:30:29.389557  7758 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.190:37433 every 8 connection(s)
I20260709 12:30:29.393941  7759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.406286  7759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb: Bootstrap starting.
I20260709 12:30:29.411417  7759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:29.416270  7759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb: No bootstrap required, opened a new log
I20260709 12:30:29.418607  7759 raft_consensus.cc:359] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e168a38330e14236bcb04bbd50b470bb" member_type: VOTER }
I20260709 12:30:29.419039  7759 raft_consensus.cc:385] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:29.419456  7759 raft_consensus.cc:740] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e168a38330e14236bcb04bbd50b470bb, State: Initialized, Role: FOLLOWER
I20260709 12:30:29.420147  7759 consensus_queue.cc:260] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e168a38330e14236bcb04bbd50b470bb" member_type: VOTER }
I20260709 12:30:29.420624  7759 raft_consensus.cc:399] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:29.420818  7759 raft_consensus.cc:493] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:29.421036  7759 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:29.427258  7759 raft_consensus.cc:515] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e168a38330e14236bcb04bbd50b470bb" member_type: VOTER }
I20260709 12:30:29.427945  7759 leader_election.cc:304] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e168a38330e14236bcb04bbd50b470bb; no voters: 
I20260709 12:30:29.429180  7759 leader_election.cc:290] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:29.429664  7762 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:29.431413  7762 raft_consensus.cc:697] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 1 LEADER]: Becoming Leader. State: Replica: e168a38330e14236bcb04bbd50b470bb, State: Running, Role: LEADER
I20260709 12:30:29.432346  7762 consensus_queue.cc:237] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e168a38330e14236bcb04bbd50b470bb" member_type: VOTER }
I20260709 12:30:29.432691  7759 sys_catalog.cc:565] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:29.437377  7763 sys_catalog.cc:455] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e168a38330e14236bcb04bbd50b470bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e168a38330e14236bcb04bbd50b470bb" member_type: VOTER } }
I20260709 12:30:29.437637  7764 sys_catalog.cc:455] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e168a38330e14236bcb04bbd50b470bb. Latest consensus state: current_term: 1 leader_uuid: "e168a38330e14236bcb04bbd50b470bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e168a38330e14236bcb04bbd50b470bb" member_type: VOTER } }
I20260709 12:30:29.438308  7763 sys_catalog.cc:458] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:29.438433  7764 sys_catalog.cc:458] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:29.443295  7769 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:29.451292  7769 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:29.452360 24862 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:29.462663  7769 catalog_manager.cc:1383] Generated new cluster ID: c3e108ffa39247689aec986997a27bb8
I20260709 12:30:29.462986  7769 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:29.477788  7769 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:29.479184  7769 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:29.491603  7769 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb: Generated new TSK 0
I20260709 12:30:29.492455  7769 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:29.519587 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:29.527067  7780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:29.528016  7781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:29.530836 24862 server_base.cc:1061] running on GCE node
W20260709 12:30:29.530887  7783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:29.532357 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:29.532604 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:29.532795 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600229532776 us; error 0 us; skew 500 ppm
I20260709 12:30:29.533381 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:29.535992 24862 webserver.cc:533] Webserver started at http://127.24.71.129:42059/ using document root <none> and password file <none>
I20260709 12:30:29.536617 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:29.536842 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:29.537130 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:29.538329 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-0-root/instance:
uuid: "ffae259334c845d795b4dde932584030"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.543337 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:30:29.547209  7788 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.548098 24862 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:30:29.548534 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-0-root
uuid: "ffae259334c845d795b4dde932584030"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.548867 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:29.566493 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:29.567862 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:29.569545 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:29.572026 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:29.572333 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:30:29.572604 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:29.572810 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.620219 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.129:38171
I20260709 12:30:29.620463  7850 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.129:38171 every 8 connection(s)
I20260709 12:30:29.625526 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:29.634675  7855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:29.634938  7856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:29.638684 24862 server_base.cc:1061] running on GCE node
W20260709 12:30:29.638799  7858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:29.640657 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:29.640931 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:29.641130 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600229641114 us; error 0 us; skew 500 ppm
I20260709 12:30:29.641693 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:29.644654  7851 heartbeater.cc:344] Connected to a master server at 127.24.71.190:37433
I20260709 12:30:29.645056  7851 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:29.645429 24862 webserver.cc:533] Webserver started at http://127.24.71.130:42881/ using document root <none> and password file <none>
I20260709 12:30:29.646003 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:29.645931  7851 heartbeater.cc:507] Master 127.24.71.190:37433 requested a full tablet report, sending...
I20260709 12:30:29.646391 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:29.646711 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:29.647866 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-1-root/instance:
uuid: "9369944381df4d7a85ab65dba5feb6e3"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.648568  7724 ts_manager.cc:194] Registered new tserver with Master: ffae259334c845d795b4dde932584030 (127.24.71.129:38171)
I20260709 12:30:29.650630  7724 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58596
I20260709 12:30:29.653077 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:30:29.657250  7863 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.658157 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:30:29.658509 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-1-root
uuid: "9369944381df4d7a85ab65dba5feb6e3"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.658824 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:29.671087 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:29.672452 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:29.673993 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:29.676291 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:29.676534 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.676779 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:29.676949 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.720736 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.130:36469
I20260709 12:30:29.720861  7925 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.130:36469 every 8 connection(s)
I20260709 12:30:29.725700 24862 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:29.734237  7929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:29.735023  7930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:29.738775  7932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:29.739467  7926 heartbeater.cc:344] Connected to a master server at 127.24.71.190:37433
I20260709 12:30:29.739863 24862 server_base.cc:1061] running on GCE node
I20260709 12:30:29.740413  7926 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:29.741012 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:29.741319 24862 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:29.741308  7926 heartbeater.cc:507] Master 127.24.71.190:37433 requested a full tablet report, sending...
I20260709 12:30:29.741811 24862 hybrid_clock.cc:648] HybridClock initialized: now 1783600229741776 us; error 0 us; skew 500 ppm
I20260709 12:30:29.742492 24862 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:29.744385  7724 ts_manager.cc:194] Registered new tserver with Master: 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:29.745607 24862 webserver.cc:533] Webserver started at http://127.24.71.131:38731/ using document root <none> and password file <none>
I20260709 12:30:29.746345 24862 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:29.746585 24862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:29.746573  7724 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58610
I20260709 12:30:29.747231 24862 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:29.748737 24862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-2-root/instance:
uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.753393 24862 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:30:29.757074  7937 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.758266 24862 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:30:29.758620 24862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-2-root
uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
format_stamp: "Formatted at 2026-07-09 12:30:29 on dist-test-slave-3zgk"
I20260709 12:30:29.758965 24862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskLD0lFw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599732358058-24862-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:29.774315 24862 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:29.775615 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:29.777359 24862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:29.779793 24862 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:29.780059 24862 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.780362 24862 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:29.780550 24862 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:29.826685 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.131:38977
I20260709 12:30:29.826807  7999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.71.131:38977 every 8 connection(s)
I20260709 12:30:29.842921  8000 heartbeater.cc:344] Connected to a master server at 127.24.71.190:37433
I20260709 12:30:29.843532  8000 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:29.844377  8000 heartbeater.cc:507] Master 127.24.71.190:37433 requested a full tablet report, sending...
I20260709 12:30:29.846609  7724 ts_manager.cc:194] Registered new tserver with Master: 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:29.846709 24862 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016483109s
I20260709 12:30:29.848341  7724 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58616
I20260709 12:30:29.874924  7724 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58628:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:29.877920  7724 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:29.948668  7891 tablet_service.cc:1511] Processing CreateTablet for tablet 292d0c1d1f9943b89c71643750b0a8b9 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:29.950299  7891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 292d0c1d1f9943b89c71643750b0a8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.953727  7889 tablet_service.cc:1511] Processing CreateTablet for tablet 872a7630d8ef4e1593c182c019b311e5 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:29.955595  7889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872a7630d8ef4e1593c182c019b311e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.957748  7965 tablet_service.cc:1511] Processing CreateTablet for tablet 292d0c1d1f9943b89c71643750b0a8b9 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:29.959529  7965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 292d0c1d1f9943b89c71643750b0a8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.961864  7887 tablet_service.cc:1511] Processing CreateTablet for tablet d9286dadba914ce6aab4563d5ae867a2 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:29.963691  7887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9286dadba914ce6aab4563d5ae867a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.965579  7886 tablet_service.cc:1511] Processing CreateTablet for tablet 8123af90ae414fb78560153531759233 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:29.967188  7886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8123af90ae414fb78560153531759233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.969697  7885 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1a7c43ee5347a9aaed1533de6c6a66 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:29.971191  7885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1a7c43ee5347a9aaed1533de6c6a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.948822  7890 tablet_service.cc:1511] Processing CreateTablet for tablet b6ef5286c8054f3b93d957e5e4cf1d77 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:29.972627  7890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ef5286c8054f3b93d957e5e4cf1d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.951869  7888 tablet_service.cc:1511] Processing CreateTablet for tablet 2729076d19a1445e893ae62f628fed91 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:29.976819  7884 tablet_service.cc:1511] Processing CreateTablet for tablet 104a1bd3387c412d9566d4fb6a9e5ad0 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:29.977427  7888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2729076d19a1445e893ae62f628fed91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.978230  7884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104a1bd3387c412d9566d4fb6a9e5ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.981472  7961 tablet_service.cc:1511] Processing CreateTablet for tablet d9286dadba914ce6aab4563d5ae867a2 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:29.982914  7961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9286dadba914ce6aab4563d5ae867a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.985477  7960 tablet_service.cc:1511] Processing CreateTablet for tablet 8123af90ae414fb78560153531759233 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:29.989478  7958 tablet_service.cc:1511] Processing CreateTablet for tablet 104a1bd3387c412d9566d4fb6a9e5ad0 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:29.991755  7883 tablet_service.cc:1511] Processing CreateTablet for tablet cc9ab233018d40eb90109978c12e9201 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:29.993180  7883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ab233018d40eb90109978c12e9201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:29.997548  7964 tablet_service.cc:1511] Processing CreateTablet for tablet b6ef5286c8054f3b93d957e5e4cf1d77 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:29.999050  7964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ef5286c8054f3b93d957e5e4cf1d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.001389  7957 tablet_service.cc:1511] Processing CreateTablet for tablet cc9ab233018d40eb90109978c12e9201 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:30.002908  7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ab233018d40eb90109978c12e9201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.005307  7963 tablet_service.cc:1511] Processing CreateTablet for tablet 872a7630d8ef4e1593c182c019b311e5 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:29.987793  7959 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1a7c43ee5347a9aaed1533de6c6a66 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:30.007009  7962 tablet_service.cc:1511] Processing CreateTablet for tablet 2729076d19a1445e893ae62f628fed91 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:30.007464  7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104a1bd3387c412d9566d4fb6a9e5ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.012681  7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8123af90ae414fb78560153531759233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.019950  8015 tablet_bootstrap.cc:492] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.017985  7962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2729076d19a1445e893ae62f628fed91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.025126  7963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872a7630d8ef4e1593c182c019b311e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.027838  7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1a7c43ee5347a9aaed1533de6c6a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.038793  8015 tablet_bootstrap.cc:654] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.046305  7816 tablet_service.cc:1511] Processing CreateTablet for tablet 292d0c1d1f9943b89c71643750b0a8b9 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:30.047809  7816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 292d0c1d1f9943b89c71643750b0a8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.070708  7815 tablet_service.cc:1511] Processing CreateTablet for tablet b6ef5286c8054f3b93d957e5e4cf1d77 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:30.072966  7815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ef5286c8054f3b93d957e5e4cf1d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.090263  7814 tablet_service.cc:1511] Processing CreateTablet for tablet 872a7630d8ef4e1593c182c019b311e5 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:30.092171  7814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872a7630d8ef4e1593c182c019b311e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.102892  8017 tablet_bootstrap.cc:492] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.109773  8017 tablet_bootstrap.cc:654] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.110941  7813 tablet_service.cc:1511] Processing CreateTablet for tablet 2729076d19a1445e893ae62f628fed91 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:30.114392  8018 tablet_bootstrap.cc:492] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.128115  7813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2729076d19a1445e893ae62f628fed91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.130802  8018 tablet_bootstrap.cc:654] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.142797  8015 tablet_bootstrap.cc:492] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.142426  7812 tablet_service.cc:1511] Processing CreateTablet for tablet d9286dadba914ce6aab4563d5ae867a2 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:30.143401  8015 ts_tablet_manager.cc:1403] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.124s	user 0.027s	sys 0.028s
I20260709 12:30:30.154328  8015 raft_consensus.cc:359] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.155731  8015 raft_consensus.cc:385] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.149048  7812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9286dadba914ce6aab4563d5ae867a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.161891  7808 tablet_service.cc:1511] Processing CreateTablet for tablet cc9ab233018d40eb90109978c12e9201 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:30.165637  7811 tablet_service.cc:1511] Processing CreateTablet for tablet 8123af90ae414fb78560153531759233 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:30.168722  7810 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1a7c43ee5347a9aaed1533de6c6a66 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:30.171864  7809 tablet_service.cc:1511] Processing CreateTablet for tablet 104a1bd3387c412d9566d4fb6a9e5ad0 (DEFAULT_TABLE table=test-workload [id=f8392fc9008c4abea25cfa82917e26a8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:30.181913  8017 tablet_bootstrap.cc:492] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.182758  8017 ts_tablet_manager.cc:1403] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.019s
I20260709 12:30:30.164143  7808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ab233018d40eb90109978c12e9201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.187539  8017 raft_consensus.cc:359] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.188848  7810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1a7c43ee5347a9aaed1533de6c6a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.192756  7809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104a1bd3387c412d9566d4fb6a9e5ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.196626  7811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8123af90ae414fb78560153531759233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:30.200371  8017 raft_consensus.cc:385] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.200779  8017 raft_consensus.cc:740] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.201654  8017 consensus_queue.cc:260] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.156139  8015 raft_consensus.cc:740] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.207460  8015 consensus_queue.cc:260] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.235587  7926 heartbeater.cc:499] Master 127.24.71.190:37433 was elected leader, sending a full tablet report...
I20260709 12:30:30.236557  8000 heartbeater.cc:499] Master 127.24.71.190:37433 was elected leader, sending a full tablet report...
I20260709 12:30:30.237218  8015 ts_tablet_manager.cc:1434] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.093s	user 0.013s	sys 0.003s
I20260709 12:30:30.240785  8018 tablet_bootstrap.cc:492] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.241427  8018 ts_tablet_manager.cc:1403] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.127s	user 0.037s	sys 0.012s
I20260709 12:30:30.244632  8018 raft_consensus.cc:359] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.245391  8018 raft_consensus.cc:385] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.245675  8018 raft_consensus.cc:740] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.246404  8018 consensus_queue.cc:260] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.248544  8015 tablet_bootstrap.cc:492] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.249902  8017 ts_tablet_manager.cc:1434] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.066s	user 0.024s	sys 0.010s
I20260709 12:30:30.250532  8018 ts_tablet_manager.cc:1434] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:30:30.251683  8017 tablet_bootstrap.cc:492] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.251892  8018 tablet_bootstrap.cc:492] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.297853  7851 heartbeater.cc:499] Master 127.24.71.190:37433 was elected leader, sending a full tablet report...
I20260709 12:30:30.317044  8015 tablet_bootstrap.cc:654] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.330911  8018 tablet_bootstrap.cc:654] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:30.331821  8001 tablet.cc:2406] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:30:30.340193  8017 tablet_bootstrap.cc:654] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.341378  8015 tablet_bootstrap.cc:492] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.343084  8015 ts_tablet_manager.cc:1403] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.095s	user 0.008s	sys 0.012s
I20260709 12:30:30.348858  8015 raft_consensus.cc:359] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.350735  8015 raft_consensus.cc:385] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.351174  8018 tablet_bootstrap.cc:492] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.351256  8015 raft_consensus.cc:740] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.354717  8018 ts_tablet_manager.cc:1403] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.008s
I20260709 12:30:30.357641  8017 tablet_bootstrap.cc:492] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.353652  8015 consensus_queue.cc:260] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.358560  8017 ts_tablet_manager.cc:1403] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.004s
I20260709 12:30:30.358937  8018 raft_consensus.cc:359] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.359951  8018 raft_consensus.cc:385] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.360298  8018 raft_consensus.cc:740] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.361174  8015 ts_tablet_manager.cc:1434] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.006s
I20260709 12:30:30.361430  8018 consensus_queue.cc:260] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.363073  8017 raft_consensus.cc:359] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.364295  8017 raft_consensus.cc:385] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.364663  8017 raft_consensus.cc:740] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.364837  8018 ts_tablet_manager.cc:1434] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:30:30.366513  8018 tablet_bootstrap.cc:492] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.366465  8017 consensus_queue.cc:260] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.369727  8017 ts_tablet_manager.cc:1434] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 12:30:30.371573  8017 tablet_bootstrap.cc:492] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.375216  8023 raft_consensus.cc:493] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.375697  8018 tablet_bootstrap.cc:654] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.375922  8023 raft_consensus.cc:515] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.378386  8015 tablet_bootstrap.cc:492] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.379679  8017 tablet_bootstrap.cc:654] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.389799  8018 tablet_bootstrap.cc:492] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.390479  8018 ts_tablet_manager.cc:1403] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.001s
I20260709 12:30:30.391177  8015 tablet_bootstrap.cc:654] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.392334  8017 tablet_bootstrap.cc:492] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.392973  8017 ts_tablet_manager.cc:1403] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.006s
I20260709 12:30:30.394871  8018 raft_consensus.cc:359] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.395422  8023 leader_election.cc:290] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977), ffae259334c845d795b4dde932584030 (127.24.71.129:38171)
I20260709 12:30:30.395853  8018 raft_consensus.cc:385] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.396265  8018 raft_consensus.cc:740] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.397413  8017 raft_consensus.cc:359] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.397668  8018 consensus_queue.cc:260] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.398650  8017 raft_consensus.cc:385] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.399196  8017 raft_consensus.cc:740] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.401342  8018 ts_tablet_manager.cc:1434] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.003s
I20260709 12:30:30.400770  8017 consensus_queue.cc:260] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.402930  8018 tablet_bootstrap.cc:492] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.405256  8017 ts_tablet_manager.cc:1434] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 12:30:30.406625  8017 tablet_bootstrap.cc:492] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.411564  8018 tablet_bootstrap.cc:654] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.413998  8017 tablet_bootstrap.cc:654] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.418458  8015 tablet_bootstrap.cc:492] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.419834  8015 ts_tablet_manager.cc:1403] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.004s
I20260709 12:30:30.422348  8022 raft_consensus.cc:493] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.423010  8022 raft_consensus.cc:515] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.424896  8018 tablet_bootstrap.cc:492] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.424687  8015 raft_consensus.cc:359] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.425515  8018 ts_tablet_manager.cc:1403] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 12:30:30.425585  8015 raft_consensus.cc:385] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.426043  8015 raft_consensus.cc:740] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.426265  8022 leader_election.cc:290] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), ffae259334c845d795b4dde932584030 (127.24.71.129:38171)
I20260709 12:30:30.426981  8015 consensus_queue.cc:260] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.429188  8018 raft_consensus.cc:359] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.430032  8018 raft_consensus.cc:385] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.430413  8018 raft_consensus.cc:740] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.434310  8018 consensus_queue.cc:260] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.439596  8017 tablet_bootstrap.cc:492] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.440363  8017 ts_tablet_manager.cc:1403] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.008s
I20260709 12:30:30.441568  8015 ts_tablet_manager.cc:1434] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:30:30.444097  8017 raft_consensus.cc:359] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.445007  8017 raft_consensus.cc:385] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.445375  8017 raft_consensus.cc:740] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.446496  8017 consensus_queue.cc:260] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.449770  8018 ts_tablet_manager.cc:1434] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 12:30:30.452112  8018 tablet_bootstrap.cc:492] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030: Bootstrap starting.
W20260709 12:30:30.455461  7772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:30.452373  8017 ts_tablet_manager.cc:1434] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:30:30.443368  8015 tablet_bootstrap.cc:492] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.461925  8017 tablet_bootstrap.cc:492] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.477175  8018 tablet_bootstrap.cc:654] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.477411  8017 tablet_bootstrap.cc:654] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.477571  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:30.483757  7975 raft_consensus.cc:2468] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:30.485329  7865 leader_election.cc:304] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:30.488420  8023 raft_consensus.cc:2804] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.488840  8023 raft_consensus.cc:493] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.489187  8023 raft_consensus.cc:3060] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.491602  8015 tablet_bootstrap.cc:654] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.494268  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.495127  7826 raft_consensus.cc:2468] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:30.501396  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:30.509068  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.510099  7826 raft_consensus.cc:2468] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:30.511768  7938 leader_election.cc:304] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.512032  8018 tablet_bootstrap.cc:492] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.512706  8018 ts_tablet_manager.cc:1403] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.011s
I20260709 12:30:30.512826  8022 raft_consensus.cc:2804] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.513302  8022 raft_consensus.cc:493] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.513667  8022 raft_consensus.cc:3060] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.513971  8015 tablet_bootstrap.cc:492] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.514173  8017 tablet_bootstrap.cc:492] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.514660  8015 ts_tablet_manager.cc:1403] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.014s
I20260709 12:30:30.516106  8018 raft_consensus.cc:359] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.514741  8017 ts_tablet_manager.cc:1403] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.006s
I20260709 12:30:30.516947  8018 raft_consensus.cc:385] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.516929  8023 raft_consensus.cc:515] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.517555  8018 raft_consensus.cc:740] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.518392  7901 raft_consensus.cc:2393] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in current term 1: Already voted for candidate 9369944381df4d7a85ab65dba5feb6e3 in this term.
I20260709 12:30:30.518520  8018 consensus_queue.cc:260] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.521989  8018 ts_tablet_manager.cc:1434] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:30:30.523201  8018 tablet_bootstrap.cc:492] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.523481  8017 raft_consensus.cc:359] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.525206  8017 raft_consensus.cc:385] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.525388  8023 leader_election.cc:290] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977), ffae259334c845d795b4dde932584030 (127.24.71.129:38171)
I20260709 12:30:30.525648  8017 raft_consensus.cc:740] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.522081  8015 raft_consensus.cc:359] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.526734  8015 raft_consensus.cc:385] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.527122  8015 raft_consensus.cc:740] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.527568  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:30.528257  8015 consensus_queue.cc:260] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.528734  8017 consensus_queue.cc:260] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.531067  8015 ts_tablet_manager.cc:1434] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:30:30.531275  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:30.535564  8015 tablet_bootstrap.cc:492] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.535983  7826 raft_consensus.cc:3060] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.536634  8018 tablet_bootstrap.cc:654] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.538626  8022 raft_consensus.cc:515] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.540195  7975 raft_consensus.cc:2393] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9369944381df4d7a85ab65dba5feb6e3 in current term 1: Already voted for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in this term.
I20260709 12:30:30.542191  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:30.543022  7901 raft_consensus.cc:2393] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in current term 1: Already voted for candidate 9369944381df4d7a85ab65dba5feb6e3 in this term.
I20260709 12:30:30.544448  8015 tablet_bootstrap.cc:654] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.550812  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:30.553546  7938 leader_election.cc:304] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807; no voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030
I20260709 12:30:30.557461  8022 leader_election.cc:290] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), ffae259334c845d795b4dde932584030 (127.24.71.129:38171)
I20260709 12:30:30.558704  8022 raft_consensus.cc:2749] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:30.569056  7826 raft_consensus.cc:2468] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:30.571247  7864 leader_election.cc:304] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807
I20260709 12:30:30.572408  8023 raft_consensus.cc:2804] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:30.576936  8018 tablet_bootstrap.cc:492] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.577358  8017 ts_tablet_manager.cc:1434] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.060s	user 0.023s	sys 0.026s
I20260709 12:30:30.577584  8018 ts_tablet_manager.cc:1403] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.016s
I20260709 12:30:30.578814  8017 tablet_bootstrap.cc:492] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.581081  8015 tablet_bootstrap.cc:492] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.581775  8015 ts_tablet_manager.cc:1403] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.047s	user 0.004s	sys 0.029s
I20260709 12:30:30.581164  8018 raft_consensus.cc:359] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.582190  8018 raft_consensus.cc:385] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.582543  8018 raft_consensus.cc:740] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.583534  8018 consensus_queue.cc:260] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.588249  8018 ts_tablet_manager.cc:1434] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:30:30.588625  8023 raft_consensus.cc:697] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:30.589440  8018 tablet_bootstrap.cc:492] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.589740  8017 tablet_bootstrap.cc:654] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.589934  8023 consensus_queue.cc:237] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:30.596199  8045 raft_consensus.cc:493] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.596969  8045 raft_consensus.cc:515] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.598453  8041 raft_consensus.cc:493] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.599273  8018 tablet_bootstrap.cc:654] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.599097  8041 raft_consensus.cc:515] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.600402  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2729076d19a1445e893ae62f628fed91" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.601265  7826 raft_consensus.cc:2468] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:30.601066  8015 raft_consensus.cc:359] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.601866  8015 raft_consensus.cc:385] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.602178  8015 raft_consensus.cc:740] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.602618  7864 leader_election.cc:304] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.603717  8046 raft_consensus.cc:2804] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.604336  8046 raft_consensus.cc:493] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.604681  8046 raft_consensus.cc:3060] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.607025  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:30.607650  8015 consensus_queue.cc:260] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
W20260709 12:30:30.609007  7940 leader_election.cc:343] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.607002  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.610141  7826 raft_consensus.cc:2468] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:30.610570  8017 tablet_bootstrap.cc:492] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.610900  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2729076d19a1445e893ae62f628fed91" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:30.611316  8017 ts_tablet_manager.cc:1403] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.006s
I20260709 12:30:30.611433  7938 leader_election.cc:304] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:30.612582  8044 raft_consensus.cc:2804] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:30.613353  7865 leader_election.cc:343] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.612957  8044 raft_consensus.cc:493] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.614235  8041 leader_election.cc:290] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:30.614283  8044 raft_consensus.cc:3060] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.614782  8045 leader_election.cc:290] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.617199  8024 raft_consensus.cc:493] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.617986  8041 raft_consensus.cc:493] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.617955  8024 raft_consensus.cc:515] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.618743  8041 raft_consensus.cc:515] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.621899  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.622606  7826 raft_consensus.cc:2468] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:30.628974  8024 leader_election.cc:290] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.637503  7724 catalog_manager.cc:5697] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:30.639492  8015 ts_tablet_manager.cc:1434] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:30:30.633438  8044 raft_consensus.cc:515] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.648638  8015 tablet_bootstrap.cc:492] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.651243  8041 leader_election.cc:290] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:30.651539  8044 leader_election.cc:290] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:30.649372  8017 raft_consensus.cc:359] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.652877  8017 raft_consensus.cc:385] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.652899  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:30.653364  8017 raft_consensus.cc:740] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.653746  7901 raft_consensus.cc:2468] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:30.653676  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:30.654256  8017 consensus_queue.cc:260] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.657804  8018 tablet_bootstrap.cc:492] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.658430  8018 ts_tablet_manager.cc:1403] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.069s	user 0.038s	sys 0.024s
I20260709 12:30:30.660375  7940 leader_election.cc:304] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
W20260709 12:30:30.661454  7940 leader_election.cc:343] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.661741  8018 raft_consensus.cc:359] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.662395  8044 raft_consensus.cc:2804] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.663089  8044 raft_consensus.cc:493] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.663522  8044 raft_consensus.cc:3060] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.667023  8017 ts_tablet_manager.cc:1434] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.003s
I20260709 12:30:30.668548  8017 tablet_bootstrap.cc:492] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.669576  8046 raft_consensus.cc:515] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.662573  8018 raft_consensus.cc:385] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.670864  8018 raft_consensus.cc:740] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.672766  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2729076d19a1445e893ae62f628fed91" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:30.675523  7826 raft_consensus.cc:3060] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.676002  8018 consensus_queue.cc:260] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.678663  8018 ts_tablet_manager.cc:1434] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:30:30.679790  8018 tablet_bootstrap.cc:492] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.674477  8044 raft_consensus.cc:515] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.687000  8018 tablet_bootstrap.cc:654] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.687911  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:30.688807  7900 raft_consensus.cc:3060] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.690591  8015 tablet_bootstrap.cc:654] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.700428  7826 raft_consensus.cc:2468] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:30.701574  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:30.702289  7864 leader_election.cc:304] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.703598  8045 raft_consensus.cc:2804] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:30:30.703471  7791 leader_election.cc:343] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.704154  8044 leader_election.cc:290] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:30.704504  8045 raft_consensus.cc:697] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:30.705770  8045 consensus_queue.cc:237] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.703433  7900 raft_consensus.cc:2468] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:30.709803  7940 leader_election.cc:304] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:30.710866  8044 raft_consensus.cc:2804] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:30.712312  8044 raft_consensus.cc:697] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:30.713502  8023 raft_consensus.cc:493] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.713338  8044 consensus_queue.cc:237] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.714293  8023 raft_consensus.cc:515] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.724787  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:30.724917  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ab233018d40eb90109978c12e9201" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.725684  7825 raft_consensus.cc:3060] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.675724  8017 tablet_bootstrap.cc:654] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.731564  8046 leader_election.cc:290] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
W20260709 12:30:30.738590  7864 leader_election.cc:343] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.739784  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2729076d19a1445e893ae62f628fed91" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:30.740530  7975 raft_consensus.cc:3060] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.742830  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:30.743606  7824 raft_consensus.cc:3060] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.746263  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ab233018d40eb90109978c12e9201" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
W20260709 12:30:30.748518  7865 leader_election.cc:343] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.749166  7865 leader_election.cc:304] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030
I20260709 12:30:30.744168  8023 leader_election.cc:290] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.751302  8023 raft_consensus.cc:2749] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:30.753506  7824 raft_consensus.cc:2468] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:30.754108  8018 tablet_bootstrap.cc:492] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.754698  8018 ts_tablet_manager.cc:1403] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.075s	user 0.006s	sys 0.011s
I20260709 12:30:30.754125  7975 raft_consensus.cc:2468] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:30.756650  7825 raft_consensus.cc:2468] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:30.758694  8018 raft_consensus.cc:359] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.759624  8018 raft_consensus.cc:385] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.759802  8018 raft_consensus.cc:740] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.760396  7938 leader_election.cc:304] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:30.764091  8017 tablet_bootstrap.cc:492] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.764283  8015 tablet_bootstrap.cc:492] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.764735  8017 ts_tablet_manager.cc:1403] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.013s
I20260709 12:30:30.764915  8015 ts_tablet_manager.cc:1403] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.124s	user 0.018s	sys 0.000s
I20260709 12:30:30.767727  8024 raft_consensus.cc:493] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.768464  8024 raft_consensus.cc:515] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.769227  8015 raft_consensus.cc:359] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.769902  8015 raft_consensus.cc:385] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.770182  8015 raft_consensus.cc:740] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.770887  8015 consensus_queue.cc:260] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.772750  8015 ts_tablet_manager.cc:1434] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 12:30:30.773864  8015 tablet_bootstrap.cc:492] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.766451  8041 raft_consensus.cc:2804] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:30.776543  8041 raft_consensus.cc:697] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:30.773782  8024 leader_election.cc:290] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.777496  8041 consensus_queue.cc:237] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.780175  8015 tablet_bootstrap.cc:654] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.772413  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:30.781571  7900 raft_consensus.cc:2468] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 0.
I20260709 12:30:30.783185  7791 leader_election.cc:304] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.784498  8018 consensus_queue.cc:260] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.787600  8018 ts_tablet_manager.cc:1434] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:30:30.792634  7722 catalog_manager.cc:5697] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:30.797992  8046 raft_consensus.cc:493] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:30.798589  8046 raft_consensus.cc:515] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.806087  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1a7c43ee5347a9aaed1533de6c6a66" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:30.806749  8046 leader_election.cc:290] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.806811  7825 raft_consensus.cc:2468] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:30.810360  7864 leader_election.cc:304] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.812836  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1a7c43ee5347a9aaed1533de6c6a66" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:30.812217  8046 raft_consensus.cc:2804] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.813231  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:30.814008  8046 raft_consensus.cc:493] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.814229  7973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:30.814553  7974 raft_consensus.cc:2468] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 0.
I20260709 12:30:30.813190  8024 raft_consensus.cc:2804] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.815442  8024 raft_consensus.cc:493] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.815807  8024 raft_consensus.cc:3060] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.815723  7975 raft_consensus.cc:2468] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:30.816957  8018 tablet_bootstrap.cc:492] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:30.814482  8046 raft_consensus.cc:3060] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:30.819298  7790 leader_election.cc:343] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.821183  8017 raft_consensus.cc:359] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.821938  7790 leader_election.cc:304] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:30.828846  8018 tablet_bootstrap.cc:654] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.829443  8063 raft_consensus.cc:2804] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:30.830051  8063 raft_consensus.cc:493] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:30.830627  8063 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:30.832782  7791 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.71.130:36469, user_credentials={real_user=slave}} blocked reactor thread for 49688us
I20260709 12:30:30.838174  7724 catalog_manager.cc:5697] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:30.821975  8017 raft_consensus.cc:385] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.844103  8018 tablet_bootstrap.cc:492] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:30.844486  8017 raft_consensus.cc:740] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.844766  8018 ts_tablet_manager.cc:1403] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.008s
I20260709 12:30:30.845391  8017 consensus_queue.cc:260] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.848858  8018 raft_consensus.cc:359] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.849463  8018 raft_consensus.cc:385] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.849717  8018 raft_consensus.cc:740] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.850325  8018 consensus_queue.cc:260] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.854652  8017 ts_tablet_manager.cc:1434] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 12:30:30.857376  8017 tablet_bootstrap.cc:492] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.860795  8024 raft_consensus.cc:515] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.863760  8015 tablet_bootstrap.cc:492] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.863984  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:30.864382  8015 ts_tablet_manager.cc:1403] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.091s	user 0.027s	sys 0.027s
I20260709 12:30:30.864629  7900 raft_consensus.cc:3060] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.865607  8017 tablet_bootstrap.cc:654] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.867686  8015 raft_consensus.cc:359] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.868456  8015 raft_consensus.cc:385] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.868732  8015 raft_consensus.cc:740] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.869541  8015 consensus_queue.cc:260] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.871402  8018 ts_tablet_manager.cc:1434] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.009s
I20260709 12:30:30.871913  8015 ts_tablet_manager.cc:1434] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:30.872073  7722 catalog_manager.cc:5697] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:30.873274  8015 tablet_bootstrap.cc:492] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.880334  8015 tablet_bootstrap.cc:654] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.882355  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:30.883090  7975 raft_consensus.cc:3060] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.892385  8024 leader_election.cc:290] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.899784  7900 raft_consensus.cc:2468] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:30.903910  8046 raft_consensus.cc:515] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.907013  7791 leader_election.cc:304] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.907598  8046 leader_election.cc:290] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.908576  8066 raft_consensus.cc:2804] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:30.909173  8066 raft_consensus.cc:697] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:30.909050  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1a7c43ee5347a9aaed1533de6c6a66" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:30.910266  8066 consensus_queue.cc:237] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.911363  7825 raft_consensus.cc:3060] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.912551  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1a7c43ee5347a9aaed1533de6c6a66" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:30.913403  7974 raft_consensus.cc:3060] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.918596  7975 raft_consensus.cc:2468] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:30.919292  8015 tablet_bootstrap.cc:492] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.919883  8015 ts_tablet_manager.cc:1403] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.016s
I20260709 12:30:30.922489  8015 raft_consensus.cc:359] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.923130  8015 raft_consensus.cc:385] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.923399  8015 raft_consensus.cc:740] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.923400  8063 raft_consensus.cc:515] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.924765  8015 consensus_queue.cc:260] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.925747  7720 catalog_manager.cc:5697] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:30.926503  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
W20260709 12:30:30.928200  7791 leader_election.cc:343] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:30.929112  8017 tablet_bootstrap.cc:492] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.929672  8017 ts_tablet_manager.cc:1403] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.018s
I20260709 12:30:30.930936  8015 ts_tablet_manager.cc:1434] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:30:30.932127  8015 tablet_bootstrap.cc:492] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:30.934219  7825 raft_consensus.cc:2468] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:30.935999  7864 leader_election.cc:304] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:30.937474  8046 raft_consensus.cc:2804] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:30.937603  8063 leader_election.cc:290] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:30.938040  8046 raft_consensus.cc:697] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:30.939430  8015 tablet_bootstrap.cc:654] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.938979  8046 consensus_queue.cc:237] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.932582  8017 raft_consensus.cc:359] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.941730  8017 raft_consensus.cc:385] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.942066  8017 raft_consensus.cc:740] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.942960  8017 consensus_queue.cc:260] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.946206  8017 ts_tablet_manager.cc:1434] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:30:30.947775  8017 tablet_bootstrap.cc:492] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:30.949371  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:30.950677  7975 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:30.950305  7974 raft_consensus.cc:2468] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:30.959232  8017 tablet_bootstrap.cc:654] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:30.962706  8015 tablet_bootstrap.cc:492] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:30.963230  8015 ts_tablet_manager.cc:1403] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 12:30:30.964563  7975 raft_consensus.cc:2468] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:30.964316  7722 catalog_manager.cc:5697] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:30.965884  7790 leader_election.cc:304] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:30.966070  8015 raft_consensus.cc:359] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.966794  8015 raft_consensus.cc:385] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.967065  8015 raft_consensus.cc:740] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.967166  8063 raft_consensus.cc:2804] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:30.967695  8063 raft_consensus.cc:697] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:30.968557  8063 consensus_queue.cc:237] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.967855  8015 consensus_queue.cc:260] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.974956  8015 ts_tablet_manager.cc:1434] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:30:30.981086  8017 tablet_bootstrap.cc:492] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:30.981696  8017 ts_tablet_manager.cc:1403] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.034s	user 0.022s	sys 0.007s
I20260709 12:30:30.983136  7720 catalog_manager.cc:5697] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:30.984822  8017 raft_consensus.cc:359] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.985675  8017 raft_consensus.cc:385] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:30.985947  8017 raft_consensus.cc:740] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:30.986711  8017 consensus_queue.cc:260] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:30.988761  8017 ts_tablet_manager.cc:1434] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:31.001212  8041 raft_consensus.cc:493] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:31.001832  8041 raft_consensus.cc:515] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.003908  8041 leader_election.cc:290] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:31.004801  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:31.005013  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:31.005641  7825 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:31.005782  7900 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:31.006778  7938 leader_election.cc:304] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.007565  8041 raft_consensus.cc:2804] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:31.007953  8041 raft_consensus.cc:493] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:31.008235  8041 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.015127  8041 raft_consensus.cc:515] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.017060  8041 leader_election.cc:290] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:31.017892  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:31.018241  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:31.018687  7825 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.018818  7900 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.026926  7900 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:31.026926  7825 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:31.028394  7940 leader_election.cc:304] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:31.029138  8041 raft_consensus.cc:2804] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:31.029604  8041 raft_consensus.cc:697] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:31.030377  8041 consensus_queue.cc:237] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.037808  7720 catalog_manager.cc:5697] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:31.044770  8046 raft_consensus.cc:493] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:31.045378  8046 raft_consensus.cc:515] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.047641  8046 leader_election.cc:290] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.048909  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:31.049840  7825 raft_consensus.cc:2393] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9369944381df4d7a85ab65dba5feb6e3 in current term 1: Already voted for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in this term.
I20260709 12:30:31.051420  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:31.051540  8046 consensus_queue.cc:1048] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.052760  7865 leader_election.cc:304] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030
I20260709 12:30:31.053577  8046 raft_consensus.cc:3060] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.065107  8050 consensus_queue.cc:1048] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.071985  8046 raft_consensus.cc:2749] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:31.091252  8041 raft_consensus.cc:493] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:31.091914  8041 raft_consensus.cc:515] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.093815  8041 leader_election.cc:290] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:31.094595  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ab233018d40eb90109978c12e9201" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:31.095014  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ab233018d40eb90109978c12e9201" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:31.095450  7825 raft_consensus.cc:2468] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:31.095678  7900 raft_consensus.cc:2468] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:31.096661  7938 leader_election.cc:304] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.097431  8041 raft_consensus.cc:2804] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:31.097821  8041 raft_consensus.cc:493] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:31.098099  8041 raft_consensus.cc:3060] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.103462  8041 raft_consensus.cc:515] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.105338  8041 leader_election.cc:290] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:31.106154  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ab233018d40eb90109978c12e9201" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:31.106398  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ab233018d40eb90109978c12e9201" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:31.106745  7825 raft_consensus.cc:3060] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.106973  7900 raft_consensus.cc:3060] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.112437  7900 raft_consensus.cc:2468] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:31.112521  7825 raft_consensus.cc:2468] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:31.113564  7940 leader_election.cc:304] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:31.114289  8041 raft_consensus.cc:2804] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:31.114797  8041 raft_consensus.cc:697] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:31.115526  8041 consensus_queue.cc:237] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.123489  7720 catalog_manager.cc:5697] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:31.162508  8046 raft_consensus.cc:493] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:31.163039  8046 raft_consensus.cc:515] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.164889  8046 leader_election.cc:290] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.165793  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:31.166116  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:31.166874  7975 raft_consensus.cc:2393] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9369944381df4d7a85ab65dba5feb6e3 in current term 1: Already voted for candidate ffae259334c845d795b4dde932584030 in this term.
I20260709 12:30:31.167829  7865 leader_election.cc:304] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030
I20260709 12:30:31.168619  8046 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:31.173866  8046 raft_consensus.cc:2749] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:31.215342  7900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:41622
I20260709 12:30:31.216041  7900 raft_consensus.cc:606] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:31.223591  8041 consensus_queue.cc:1048] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.233330  7975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:31.234136  7975 raft_consensus.cc:606] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:31.238459  8084 consensus_queue.cc:1048] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.242978  7975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:31.243634  7975 raft_consensus.cc:606] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:31.248760  8041 raft_consensus.cc:993] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.249284  8041 raft_consensus.cc:1081] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.251124  7825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55028
I20260709 12:30:31.251798  7825 raft_consensus.cc:493] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.252172  7825 raft_consensus.cc:3060] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.253906  7900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2729076d19a1445e893ae62f628fed91"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:41622
I20260709 12:30:31.254606  7900 raft_consensus.cc:606] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:31.258610  7825 raft_consensus.cc:515] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.261282  7825 leader_election.cc:290] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.262070  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:31.262498  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872a7630d8ef4e1593c182c019b311e5" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.262831  7900 raft_consensus.cc:3060] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.263120  7974 raft_consensus.cc:3055] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.263361  7974 raft_consensus.cc:740] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:31.264029  7974 consensus_queue.cc:260] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.261305  7975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:31.265054  7975 raft_consensus.cc:606] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:31.265431  7974 raft_consensus.cc:3060] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.270560  7900 raft_consensus.cc:2468] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.272045  7791 leader_election.cc:304] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.272693  7974 raft_consensus.cc:2468] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.273358  8063 raft_consensus.cc:2804] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.273944  8063 raft_consensus.cc:697] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:31.274718  8063 consensus_queue.cc:237] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.283273  7724 catalog_manager.cc:5697] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:31.286716  7825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8123af90ae414fb78560153531759233"
dest_uuid: "ffae259334c845d795b4dde932584030"
mode: GRACEFUL
new_leader_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:55042
I20260709 12:30:31.287490  7825 raft_consensus.cc:606] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Received request to transfer leadership to TS 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:31.297976  7825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0"
dest_uuid: "ffae259334c845d795b4dde932584030"
mode: GRACEFUL
new_leader_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:55042
I20260709 12:30:31.298547  7825 raft_consensus.cc:606] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Received request to transfer leadership to TS 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:31.308187  8046 consensus_queue.cc:1048] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.316494  8045 raft_consensus.cc:993] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.316555  8050 consensus_queue.cc:1048] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.317175  8023 raft_consensus.cc:1081] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.319662  7825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2729076d19a1445e893ae62f628fed91"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:30:31.320307  7825 raft_consensus.cc:493] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.320684  7825 raft_consensus.cc:3060] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.329715  7825 raft_consensus.cc:515] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.330659  7824 raft_consensus.cc:1240] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Rejecting Update request from peer 9369944381df4d7a85ab65dba5feb6e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:31.332687  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2729076d19a1445e893ae62f628fed91" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:31.332916  7825 leader_election.cc:290] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.333451  7900 raft_consensus.cc:3055] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.333851  7900 raft_consensus.cc:740] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:31.333665  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2729076d19a1445e893ae62f628fed91" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.334830  8046 consensus_queue.cc:1059] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.335444  7974 raft_consensus.cc:3060] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.335980  7900 consensus_queue.cc:260] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.337736  7900 raft_consensus.cc:3060] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.343199  7974 raft_consensus.cc:2468] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.344503  7790 leader_election.cc:304] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.345291  8063 raft_consensus.cc:2804] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.345501  7900 raft_consensus.cc:2468] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.345799  8063 raft_consensus.cc:697] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
W20260709 12:30:31.347046  8045 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:30:31.346767  8063 consensus_queue.cc:237] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.348601  8041 consensus_queue.cc:1048] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.355372  7720 catalog_manager.cc:5697] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:31.364111  8084 consensus_queue.cc:1048] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.372396  8044 raft_consensus.cc:993] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:31.372937  8044 raft_consensus.cc:1081] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:31.374444  7900 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:41610
I20260709 12:30:31.374950  7900 raft_consensus.cc:493] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.375277  7900 raft_consensus.cc:3060] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.381076  7900 raft_consensus.cc:515] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.383348  7900 leader_election.cc:290] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.383747  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:31.384027  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9286dadba914ce6aab4563d5ae867a2" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.384485  7825 raft_consensus.cc:3060] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.384683  7974 raft_consensus.cc:3055] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.385133  7974 raft_consensus.cc:740] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:31.385856  7974 consensus_queue.cc:260] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.386839  7974 raft_consensus.cc:3060] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.391314  7825 raft_consensus.cc:2468] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:31.392584  7864 leader_election.cc:304] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.392864  7974 raft_consensus.cc:2468] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:31.393503  8045 raft_consensus.cc:2804] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.393996  8045 raft_consensus.cc:697] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:31.394840  8045 consensus_queue.cc:237] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.401895  7720 catalog_manager.cc:5697] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:31.415671  8063 consensus_queue.cc:1048] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.428912  8092 consensus_queue.cc:1048] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.437055  8045 consensus_queue.cc:1048] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.438426  8092 raft_consensus.cc:993] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: : Instructing follower 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:31.438989  8063 raft_consensus.cc:1081] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Signalling peer 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:31.439857  8066 consensus_queue.cc:1048] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.440878  7901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:41616
I20260709 12:30:31.441581  7901 raft_consensus.cc:493] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.441991  7901 raft_consensus.cc:3060] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.446319  8063 consensus_queue.cc:1048] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.451885  7901 raft_consensus.cc:515] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.452877  8023 consensus_queue.cc:1048] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.455183  7901 leader_election.cc:290] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.461270  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:31.461988  7824 raft_consensus.cc:3055] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.462285  7824 raft_consensus.cc:740] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:31.463025  7824 consensus_queue.cc:260] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.464663  7824 raft_consensus.cc:3060] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.466298  8092 raft_consensus.cc:993] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: : Instructing follower 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:31.466809  8092 raft_consensus.cc:1081] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Signalling peer 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:31.467597  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104a1bd3387c412d9566d4fb6a9e5ad0" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.468283  7975 raft_consensus.cc:3060] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.476933  7900 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8123af90ae414fb78560153531759233"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:41616
I20260709 12:30:31.477571  7900 raft_consensus.cc:493] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.477957  7900 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.483255  7824 raft_consensus.cc:2468] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:31.484629  7864 leader_election.cc:304] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.485340  8023 raft_consensus.cc:2804] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.485826  8023 raft_consensus.cc:697] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:31.486732  7975 raft_consensus.cc:2468] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:31.486665  8023 consensus_queue.cc:237] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.488225  7900 raft_consensus.cc:515] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.491777  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:31.492643  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.492947  7824 raft_consensus.cc:3055] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.493335  7975 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.493603  7824 raft_consensus.cc:740] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:31.495630  7824 consensus_queue.cc:260] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.497632  7900 leader_election.cc:290] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.498894  7824 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.506490  7975 raft_consensus.cc:2468] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:31.508014  7865 leader_election.cc:304] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:31.509044  8023 raft_consensus.cc:2804] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.509558  8023 raft_consensus.cc:697] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:31.509158  7724 catalog_manager.cc:5697] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 1 to 2, leader changed from ffae259334c845d795b4dde932584030 (127.24.71.129) to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:31.510464  8023 consensus_queue.cc:237] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.519259  7824 raft_consensus.cc:2468] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:31.525713  7724 catalog_manager.cc:5697] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 1 to 2, leader changed from ffae259334c845d795b4dde932584030 (127.24.71.129) to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:31.531055  8044 consensus_queue.cc:1048] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.538416  8084 raft_consensus.cc:993] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.538977  8041 raft_consensus.cc:1081] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.540737  7825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55028
I20260709 12:30:31.544178  8084 consensus_queue.cc:1048] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:31.549247  7825 raft_consensus.cc:493] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.549966  7825 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.557345  7825 raft_consensus.cc:515] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.562973  7825 leader_election.cc:290] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.563198  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.563391  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:31.563906  7974 raft_consensus.cc:3055] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.564177  7900 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.564648  7974 raft_consensus.cc:740] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:31.565430  7974 consensus_queue.cc:260] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.566690  7974 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.570561  7900 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.571838  7791 leader_election.cc:304] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.572353  7974 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.572817  8092 raft_consensus.cc:2804] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.573360  8092 raft_consensus.cc:697] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:31.574093  8092 consensus_queue.cc:237] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:31.580891  7724 catalog_manager.cc:5697] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:31.627234  8045 raft_consensus.cc:993] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.627781  8023 raft_consensus.cc:1081] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:31.629693  7825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:30:31.630393  7825 raft_consensus.cc:493] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:31.631001  7825 raft_consensus.cc:3060] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.637521  7825 raft_consensus.cc:515] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:31.639281  7825 leader_election.cc:290] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:31.640300  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:31.640394  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "292d0c1d1f9943b89c71643750b0a8b9" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:31.641194  7900 raft_consensus.cc:3055] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:31.641215  7974 raft_consensus.cc:3060] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.641711  7900 raft_consensus.cc:740] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:31.642663  7900 consensus_queue.cc:260] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:31.643952  7900 raft_consensus.cc:3060] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:31.649339  7974 raft_consensus.cc:2468] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.650913  7790 leader_election.cc:304] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:31.651429  7900 raft_consensus.cc:2468] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:31.651891  8092 raft_consensus.cc:2804] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:31.652491  8092 raft_consensus.cc:697] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:31.653375  8092 consensus_queue.cc:237] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } }
I20260709 12:30:31.661085  7724 catalog_manager.cc:5697] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:31.693682  7900 raft_consensus.cc:1275] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:31.695093  8092 consensus_queue.cc:1048] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:31.699671  8044 consensus_queue.cc:1048] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.704984  8084 consensus_queue.cc:1048] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:31.706223  7974 raft_consensus.cc:1275] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:31.707974  8066 consensus_queue.cc:1048] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:31.857512  7974 raft_consensus.cc:1275] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:31.859048  8066 consensus_queue.cc:1048] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:31.870572  7899 raft_consensus.cc:1275] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:31.871987  8092 consensus_queue.cc:1048] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:31.878496  7825 raft_consensus.cc:1275] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:31.880303  8045 consensus_queue.cc:1048] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:31.889405  7974 raft_consensus.cc:1275] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:31.890762  8045 consensus_queue.cc:1048] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.030247  7899 raft_consensus.cc:1275] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.031673  8092 consensus_queue.cc:1048] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.037829  7975 raft_consensus.cc:1275] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.040045  8023 consensus_queue.cc:1048] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.042529  7974 raft_consensus.cc:1275] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.049168  8103 consensus_queue.cc:1048] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.049674  7825 raft_consensus.cc:1275] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.051574  8050 consensus_queue.cc:1048] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.082460  7974 raft_consensus.cc:1275] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.083664  8050 consensus_queue.cc:1048] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.089006  7825 raft_consensus.cc:1275] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.091061  8023 consensus_queue.cc:1048] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.148136  7974 raft_consensus.cc:1275] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.149912  8103 consensus_queue.cc:1048] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:32.159914  7899 raft_consensus.cc:1275] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:32.161572  8103 consensus_queue.cc:1048] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:34.337563  7722 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38142:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:34.340047  7722 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:34.394655  7815 tablet_service.cc:1511] Processing CreateTablet for tablet 7974d124c88843479ba5624cb6ad89a7 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:34.395999  7815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7974d124c88843479ba5624cb6ad89a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.396451  7886 tablet_service.cc:1511] Processing CreateTablet for tablet 7974d124c88843479ba5624cb6ad89a7 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:34.398141  7886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7974d124c88843479ba5624cb6ad89a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.398617  7957 tablet_service.cc:1511] Processing CreateTablet for tablet 7974d124c88843479ba5624cb6ad89a7 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:34.399993  7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7974d124c88843479ba5624cb6ad89a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.400779  7808 tablet_service.cc:1511] Processing CreateTablet for tablet 64b4c722d8e8495da36c89350e29bcb2 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:34.402385  7808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b4c722d8e8495da36c89350e29bcb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.416965  7889 tablet_service.cc:1511] Processing CreateTablet for tablet 64b4c722d8e8495da36c89350e29bcb2 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:34.418552  7889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b4c722d8e8495da36c89350e29bcb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.418359  7964 tablet_service.cc:1511] Processing CreateTablet for tablet 64b4c722d8e8495da36c89350e29bcb2 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:34.421376  7964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b4c722d8e8495da36c89350e29bcb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.421249  7809 tablet_service.cc:1511] Processing CreateTablet for tablet 7bcb087f3a694ba090b2ea713d4f9efd (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:34.422811  7809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bcb087f3a694ba090b2ea713d4f9efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.426023  7958 tablet_service.cc:1511] Processing CreateTablet for tablet 7bcb087f3a694ba090b2ea713d4f9efd (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:34.427700  7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bcb087f3a694ba090b2ea713d4f9efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.431555  7890 tablet_service.cc:1511] Processing CreateTablet for tablet 7bcb087f3a694ba090b2ea713d4f9efd (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:34.437472  7813 tablet_service.cc:1511] Processing CreateTablet for tablet 054091d9725042bcbc077f81bf42312f (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:34.433281  7890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bcb087f3a694ba090b2ea713d4f9efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.438993  7813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054091d9725042bcbc077f81bf42312f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.449806  7884 tablet_service.cc:1511] Processing CreateTablet for tablet 054091d9725042bcbc077f81bf42312f (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:34.451489  7884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054091d9725042bcbc077f81bf42312f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.455170  8119 tablet_bootstrap.cc:492] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:34.457175  7963 tablet_service.cc:1511] Processing CreateTablet for tablet 054091d9725042bcbc077f81bf42312f (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:34.459846  7963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054091d9725042bcbc077f81bf42312f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.465345  8119 tablet_bootstrap.cc:654] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.470443  7813 tablet_service.cc:1511] Processing CreateTablet for tablet 3b1b1e5503324ec29c0eb2432e28a4dc (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:34.472198  7813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b1b1e5503324ec29c0eb2432e28a4dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.473455  7964 tablet_service.cc:1511] Processing CreateTablet for tablet 3b1b1e5503324ec29c0eb2432e28a4dc (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:34.474961  7964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b1b1e5503324ec29c0eb2432e28a4dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.475322  7808 tablet_service.cc:1511] Processing CreateTablet for tablet 0703408f26944656ace444a76f3b74ac (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:34.475395  7883 tablet_service.cc:1511] Processing CreateTablet for tablet 3b1b1e5503324ec29c0eb2432e28a4dc (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:34.476975  7808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0703408f26944656ace444a76f3b74ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.477034  7883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b1b1e5503324ec29c0eb2432e28a4dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.488044  7887 tablet_service.cc:1511] Processing CreateTablet for tablet 0703408f26944656ace444a76f3b74ac (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:34.489815  7887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0703408f26944656ace444a76f3b74ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.490741  7962 tablet_service.cc:1511] Processing CreateTablet for tablet 0703408f26944656ace444a76f3b74ac (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:34.492349  7962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0703408f26944656ace444a76f3b74ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.497841  7815 tablet_service.cc:1511] Processing CreateTablet for tablet 627ca9faca0a4c84b4d440484f6367c5 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:34.499217  7815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627ca9faca0a4c84b4d440484f6367c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.505270  7885 tablet_service.cc:1511] Processing CreateTablet for tablet 627ca9faca0a4c84b4d440484f6367c5 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:34.506970  7885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627ca9faca0a4c84b4d440484f6367c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.507606  7958 tablet_service.cc:1511] Processing CreateTablet for tablet 627ca9faca0a4c84b4d440484f6367c5 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:34.531498  7812 tablet_service.cc:1511] Processing CreateTablet for tablet a2b6444c31ca4468b202ef77a8982cee (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:34.533702  7812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2b6444c31ca4468b202ef77a8982cee. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:34.539252  7772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:34.509756  7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627ca9faca0a4c84b4d440484f6367c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.602377  8118 tablet_bootstrap.cc:492] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:34.609283  8118 tablet_bootstrap.cc:654] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.625360  7885 tablet_service.cc:1511] Processing CreateTablet for tablet a600fd77b0ad4ae6926bc35630c98605 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:34.626984  7885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a600fd77b0ad4ae6926bc35630c98605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.626783  7813 tablet_service.cc:1511] Processing CreateTablet for tablet a600fd77b0ad4ae6926bc35630c98605 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:34.628283  7813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a600fd77b0ad4ae6926bc35630c98605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.618762  7890 tablet_service.cc:1511] Processing CreateTablet for tablet a2b6444c31ca4468b202ef77a8982cee (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:34.631716  7964 tablet_service.cc:1511] Processing CreateTablet for tablet a2b6444c31ca4468b202ef77a8982cee (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:34.646270  7964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2b6444c31ca4468b202ef77a8982cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.649945  7962 tablet_service.cc:1511] Processing CreateTablet for tablet a600fd77b0ad4ae6926bc35630c98605 (DEFAULT_TABLE table=soft_deleted_table [id=e780332f365149cc864df0ca98836c0e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:34.651968  7962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a600fd77b0ad4ae6926bc35630c98605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.656904  7890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2b6444c31ca4468b202ef77a8982cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:34.669363  8121 tablet_bootstrap.cc:492] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:34.659976  8119 tablet_bootstrap.cc:492] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:34.671440  8119 ts_tablet_manager.cc:1403] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.217s	user 0.042s	sys 0.033s
I20260709 12:30:34.675719  8119 raft_consensus.cc:359] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.677080  8119 raft_consensus.cc:385] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.677516  8119 raft_consensus.cc:740] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.677768  8121 tablet_bootstrap.cc:654] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.678743  8119 consensus_queue.cc:260] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.695900  8118 tablet_bootstrap.cc:492] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:34.696591  8118 ts_tablet_manager.cc:1403] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.095s	user 0.013s	sys 0.012s
I20260709 12:30:34.704774  8119 ts_tablet_manager.cc:1434] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:30:34.706171  8119 tablet_bootstrap.cc:492] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:34.707914  8118 raft_consensus.cc:359] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.708945  8118 raft_consensus.cc:385] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.709306  8118 raft_consensus.cc:740] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.710240  8118 consensus_queue.cc:260] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.715193  8118 ts_tablet_manager.cc:1434] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:30:34.716354  8118 tablet_bootstrap.cc:492] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:34.723215  8118 tablet_bootstrap.cc:654] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.736609  8123 raft_consensus.cc:493] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:34.737274  8123 raft_consensus.cc:515] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.740595  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7974d124c88843479ba5624cb6ad89a7" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
W20260709 12:30:34.742466  7791 leader_election.cc:343] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:34.744052  8123 leader_election.cc:290] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:34.746161  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7974d124c88843479ba5624cb6ad89a7" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:34.747054  7975 raft_consensus.cc:2468] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 0.
I20260709 12:30:34.748624  7790 leader_election.cc:304] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:34.753868  8121 tablet_bootstrap.cc:492] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:34.754688  8121 ts_tablet_manager.cc:1403] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.012s
I20260709 12:30:34.757683  8123 raft_consensus.cc:2804] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:34.765309  8123 raft_consensus.cc:493] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:34.765734  8123 raft_consensus.cc:3060] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:34.772203  8119 tablet_bootstrap.cc:654] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.772318  8121 raft_consensus.cc:359] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.773857  8121 raft_consensus.cc:385] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.774228  8121 raft_consensus.cc:740] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.775130  8121 consensus_queue.cc:260] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.775233  8123 raft_consensus.cc:515] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.778002  8121 ts_tablet_manager.cc:1434] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 12:30:34.779138  8121 tablet_bootstrap.cc:492] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:34.779461  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7974d124c88843479ba5624cb6ad89a7" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:34.780365  7975 raft_consensus.cc:3060] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:34.784863  8119 tablet_bootstrap.cc:492] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:34.785842  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7974d124c88843479ba5624cb6ad89a7" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:34.786504  7899 raft_consensus.cc:3060] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:34.787868  8118 tablet_bootstrap.cc:492] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:34.788138  8121 tablet_bootstrap.cc:654] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.788517  8118 ts_tablet_manager.cc:1403] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.072s	user 0.019s	sys 0.015s
I20260709 12:30:34.792517  8118 raft_consensus.cc:359] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.793622  8119 ts_tablet_manager.cc:1403] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.004s
I20260709 12:30:34.797646  7899 raft_consensus.cc:2468] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:34.793668  8118 raft_consensus.cc:385] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.799206  7975 raft_consensus.cc:2468] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:34.799202  8118 raft_consensus.cc:740] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.800630  8123 leader_election.cc:290] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:34.801276  7791 leader_election.cc:304] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:34.800850  8118 consensus_queue.cc:260] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.802459  8119 raft_consensus.cc:359] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.803303  8119 raft_consensus.cc:385] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.803653  8119 raft_consensus.cc:740] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.804395  8121 tablet_bootstrap.cc:492] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:34.805020  8121 ts_tablet_manager.cc:1403] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.013s
I20260709 12:30:34.804522  8119 consensus_queue.cc:260] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.805915  8107 raft_consensus.cc:2804] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:34.810065  8121 raft_consensus.cc:359] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.811199  8121 raft_consensus.cc:385] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.811277  8118 ts_tablet_manager.cc:1434] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:30:34.811739  8121 raft_consensus.cc:740] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.812711  8118 tablet_bootstrap.cc:492] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:34.814412  8119 ts_tablet_manager.cc:1434] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:30:34.812794  8121 consensus_queue.cc:260] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.816478  8119 tablet_bootstrap.cc:492] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:34.820375  8121 ts_tablet_manager.cc:1434] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 12:30:34.822429  8107 raft_consensus.cc:697] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:34.823804  8121 tablet_bootstrap.cc:492] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:34.823589  8107 consensus_queue.cc:237] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.826435  8119 tablet_bootstrap.cc:654] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.847393  7722 catalog_manager.cc:5697] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:34.847931  8118 tablet_bootstrap.cc:654] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.849627  8121 tablet_bootstrap.cc:654] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.854897  8118 tablet_bootstrap.cc:492] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:34.855196  8119 tablet_bootstrap.cc:492] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:34.855427  8118 ts_tablet_manager.cc:1403] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.043s	user 0.007s	sys 0.005s
I20260709 12:30:34.855755  8119 ts_tablet_manager.cc:1403] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.004s
I20260709 12:30:34.858402  8118 raft_consensus.cc:359] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.859129  8118 raft_consensus.cc:385] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.859380  8118 raft_consensus.cc:740] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.859673  8121 tablet_bootstrap.cc:492] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:34.860262  8121 ts_tablet_manager.cc:1403] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.000s
I20260709 12:30:34.860127  8118 consensus_queue.cc:260] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.863052  8118 ts_tablet_manager.cc:1434] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:30:34.864120  8118 tablet_bootstrap.cc:492] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:34.863801  8121 raft_consensus.cc:359] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.864642  8121 raft_consensus.cc:385] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.864935  8121 raft_consensus.cc:740] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.865675  8121 consensus_queue.cc:260] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.868048  8121 ts_tablet_manager.cc:1434] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:34.868863  8119 raft_consensus.cc:359] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.869650  8119 raft_consensus.cc:385] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.869952  8119 raft_consensus.cc:740] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.870723  8119 consensus_queue.cc:260] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.872714  8118 tablet_bootstrap.cc:654] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.873122  8119 ts_tablet_manager.cc:1434] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 12:30:34.873174  8121 tablet_bootstrap.cc:492] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:34.874571  8119 tablet_bootstrap.cc:492] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:34.886821  8121 tablet_bootstrap.cc:654] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.895227  8118 tablet_bootstrap.cc:492] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:34.895746  8118 ts_tablet_manager.cc:1403] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.015s
I20260709 12:30:34.898275  8119 tablet_bootstrap.cc:654] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.899200  8121 tablet_bootstrap.cc:492] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:34.898836  8118 raft_consensus.cc:359] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.899685  8118 raft_consensus.cc:385] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.899830  8121 ts_tablet_manager.cc:1403] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.003s
I20260709 12:30:34.899995  8118 raft_consensus.cc:740] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.900954  8118 consensus_queue.cc:260] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.902711  8118 ts_tablet_manager.cc:1434] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:34.904202  8118 tablet_bootstrap.cc:492] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:34.906175  8121 raft_consensus.cc:359] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.906991  8121 raft_consensus.cc:385] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.907299  8121 raft_consensus.cc:740] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.907410  8119 tablet_bootstrap.cc:492] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:34.908193  8119 ts_tablet_manager.cc:1403] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.008s
I20260709 12:30:34.908113  8121 consensus_queue.cc:260] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.910660  8121 ts_tablet_manager.cc:1434] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.004s
I20260709 12:30:34.911327  8118 tablet_bootstrap.cc:654] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.911705  8121 tablet_bootstrap.cc:492] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:34.915403  8119 raft_consensus.cc:359] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.916723  8119 raft_consensus.cc:385] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.917202  8119 raft_consensus.cc:740] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.918496  8119 consensus_queue.cc:260] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.922359  8119 ts_tablet_manager.cc:1434] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:30:34.923659  8119 tablet_bootstrap.cc:492] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:34.926523  8084 raft_consensus.cc:493] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:34.927111  8084 raft_consensus.cc:515] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.932540  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:34.932631  8107 raft_consensus.cc:493] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:34.933211  7825 raft_consensus.cc:2468] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:34.933480  8107 raft_consensus.cc:515] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.934746  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:34.935768  7899 raft_consensus.cc:2468] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:34.936610  8107 leader_election.cc:290] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:34.938215  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:34.938434  7938 leader_election.cc:304] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
W20260709 12:30:34.942188  7791 leader_election.cc:343] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:34.944886  8119 tablet_bootstrap.cc:654] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.940609  8084 leader_election.cc:290] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:34.945580  8084 raft_consensus.cc:493] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:34.946208  8084 raft_consensus.cc:515] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.944394  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:34.948417  8118 tablet_bootstrap.cc:492] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:34.949980  8084 leader_election.cc:290] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:34.950150  8118 ts_tablet_manager.cc:1403] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.022s
I20260709 12:30:34.951671  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1b1e5503324ec29c0eb2432e28a4dc" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:34.951021  8084 raft_consensus.cc:2804] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:34.952860  8084 raft_consensus.cc:493] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:30:34.952734  7790 leader_election.cc:343] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:34.953462  8121 tablet_bootstrap.cc:654] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:34.954533  7940 leader_election.cc:343] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:34.957381  8118 raft_consensus.cc:359] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.957924  8119 tablet_bootstrap.cc:492] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:34.958163  8118 raft_consensus.cc:385] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.954104  7790 leader_election.cc:304] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ffae259334c845d795b4dde932584030; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:34.958513  8119 ts_tablet_manager.cc:1403] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.005s
I20260709 12:30:34.958545  8118 raft_consensus.cc:740] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.959582  8118 consensus_queue.cc:260] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.959976  8107 raft_consensus.cc:2749] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:34.953630  8084 raft_consensus.cc:3060] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:34.961021  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1b1e5503324ec29c0eb2432e28a4dc" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:34.962037  8119 raft_consensus.cc:359] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.962073  8118 ts_tablet_manager.cc:1434] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:30:34.962781  8119 raft_consensus.cc:385] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:34.964748  8119 raft_consensus.cc:740] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:34.964849  8118 tablet_bootstrap.cc:492] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:34.965581  8119 consensus_queue.cc:260] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.968173  8119 ts_tablet_manager.cc:1434] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:30:34.971773  8119 tablet_bootstrap.cc:492] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:34.972522  8118 tablet_bootstrap.cc:654] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.976308  8126 raft_consensus.cc:493] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:34.976881  8126 raft_consensus.cc:515] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.979519  8119 tablet_bootstrap.cc:654] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:34.980008  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
W20260709 12:30:34.981598  7864 leader_election.cc:343] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:34.981905  7938 leader_election.cc:343] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:34.982599  7938 leader_election.cc:304] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807; no voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030
I20260709 12:30:34.983408  8126 leader_election.cc:290] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:34.975684  8084 raft_consensus.cc:515] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.984954  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:34.987744  8121 tablet_bootstrap.cc:492] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:34.989632  8121 ts_tablet_manager.cc:1403] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.008s
W20260709 12:30:34.991364  7865 leader_election.cc:343] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:34.992224  7865 leader_election.cc:304] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030
I20260709 12:30:34.994225  8121 raft_consensus.cc:359] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.998809  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:34.999737  8121 raft_consensus.cc:385] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.000129  7825 raft_consensus.cc:3060] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.000470  8121 raft_consensus.cc:740] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.002009  8121 consensus_queue.cc:260] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:34.995360  8126 raft_consensus.cc:2749] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:34.998171  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.004822  7899 raft_consensus.cc:3060] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.008524  8106 raft_consensus.cc:493] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.009341  8106 raft_consensus.cc:515] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.013240  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1b1e5503324ec29c0eb2432e28a4dc" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.013940  7824 raft_consensus.cc:2468] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:35.015202  7864 leader_election.cc:304] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.016503  8139 raft_consensus.cc:2749] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:34.998924  8084 leader_election.cc:290] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.018891  8126 raft_consensus.cc:2804] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.019608  8126 raft_consensus.cc:493] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.020159  8126 raft_consensus.cc:3060] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.020655  8106 leader_election.cc:290] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.026757  8121 ts_tablet_manager.cc:1434] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.037s	user 0.002s	sys 0.005s
I20260709 12:30:35.031339  8118 tablet_bootstrap.cc:492] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:35.031919  8119 tablet_bootstrap.cc:492] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:35.033485  8118 ts_tablet_manager.cc:1403] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.069s	user 0.019s	sys 0.045s
I20260709 12:30:35.033893  8121 tablet_bootstrap.cc:492] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:35.037520  8118 raft_consensus.cc:359] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.038365  8118 raft_consensus.cc:385] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.038707  8118 raft_consensus.cc:740] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.033892  8119 ts_tablet_manager.cc:1403] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.062s	user 0.028s	sys 0.028s
I20260709 12:30:35.039175  8126 raft_consensus.cc:515] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.039719  8118 consensus_queue.cc:260] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.042888  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1b1e5503324ec29c0eb2432e28a4dc" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:35.043659  7975 raft_consensus.cc:2468] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:35.043821  8118 ts_tablet_manager.cc:1434] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:30:35.044994  8118 tablet_bootstrap.cc:492] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:35.042922  8119 raft_consensus.cc:359] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.045708  8119 raft_consensus.cc:385] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.046029  8119 raft_consensus.cc:740] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.046821  8119 consensus_queue.cc:260] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.049680  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1b1e5503324ec29c0eb2432e28a4dc" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.050299  7824 raft_consensus.cc:3060] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.052352  8118 tablet_bootstrap.cc:654] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.059309  8121 tablet_bootstrap.cc:654] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.060415  8126 leader_election.cc:290] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.061607  7824 raft_consensus.cc:2468] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:35.063251  8119 ts_tablet_manager.cc:1434] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:30:35.063611  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1b1e5503324ec29c0eb2432e28a4dc" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.064482  8119 tablet_bootstrap.cc:492] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:35.064388  7864 leader_election.cc:304] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.065023  7899 raft_consensus.cc:2468] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.064412  7975 raft_consensus.cc:3060] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.066536  7940 leader_election.cc:304] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:35.066905  8118 tablet_bootstrap.cc:492] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:35.067442  8118 ts_tablet_manager.cc:1403] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.010s
I20260709 12:30:35.067476  8084 raft_consensus.cc:2804] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.068111  8084 raft_consensus.cc:697] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.069047  8084 consensus_queue.cc:237] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.071233  8118 raft_consensus.cc:359] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.072044  8118 raft_consensus.cc:385] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.072393  8119 tablet_bootstrap.cc:654] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.072218  8126 raft_consensus.cc:2804] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.072397  8118 raft_consensus.cc:740] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.073784  8118 consensus_queue.cc:260] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.074121  8126 raft_consensus.cc:697] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:35.075258  8126 consensus_queue.cc:237] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.076303  8118 ts_tablet_manager.cc:1434] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.008s	user 0.011s	sys 0.000s
I20260709 12:30:35.081915  8118 tablet_bootstrap.cc:492] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:35.080274  7975 raft_consensus.cc:2468] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:35.089236  8118 tablet_bootstrap.cc:654] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.097141  8119 tablet_bootstrap.cc:492] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:35.097694  8119 ts_tablet_manager.cc:1403] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.016s
I20260709 12:30:35.100580  8118 tablet_bootstrap.cc:492] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:35.101161  8118 ts_tablet_manager.cc:1403] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:30:35.100760  8119 raft_consensus.cc:359] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.101594  8119 raft_consensus.cc:385] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.101913  8119 raft_consensus.cc:740] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.102113  7825 raft_consensus.cc:2468] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.102650  8119 consensus_queue.cc:260] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.102491  7720 catalog_manager.cc:5697] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.103997  8118 raft_consensus.cc:359] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.104740  8118 raft_consensus.cc:385] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.104992  8118 raft_consensus.cc:740] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.105146  8119 ts_tablet_manager.cc:1434] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:30:35.106297  8119 tablet_bootstrap.cc:492] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:35.105690  8118 consensus_queue.cc:260] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.110850  8118 ts_tablet_manager.cc:1434] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:30:35.111876  8118 tablet_bootstrap.cc:492] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Bootstrap starting.
I20260709 12:30:35.113366  8119 tablet_bootstrap.cc:654] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.114125  7722 catalog_manager.cc:5697] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:35.118211  8118 tablet_bootstrap.cc:654] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.119356  8121 tablet_bootstrap.cc:492] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:35.120038  8121 ts_tablet_manager.cc:1403] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.004s
I20260709 12:30:35.123387  8121 raft_consensus.cc:359] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.124049  8118 tablet_bootstrap.cc:492] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: No bootstrap required, opened a new log
I20260709 12:30:35.124127  8121 raft_consensus.cc:385] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.124794  8118 ts_tablet_manager.cc:1403] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 12:30:35.124768  8121 raft_consensus.cc:740] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.125752  8119 tablet_bootstrap.cc:492] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:35.126420  8119 ts_tablet_manager.cc:1403] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 12:30:35.125818  8121 consensus_queue.cc:260] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.127729  8118 raft_consensus.cc:359] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.128437  8118 raft_consensus.cc:385] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.128698  8118 raft_consensus.cc:740] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.129179  8119 raft_consensus.cc:359] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.129364  8118 consensus_queue.cc:260] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.129935  8119 raft_consensus.cc:385] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.130244  8119 raft_consensus.cc:740] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.130898  8119 consensus_queue.cc:260] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.131564  8118 ts_tablet_manager.cc:1434] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:35.132967  8119 ts_tablet_manager.cc:1434] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:30:35.133962  8121 ts_tablet_manager.cc:1434] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.006s
I20260709 12:30:35.134189  8119 tablet_bootstrap.cc:492] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: Bootstrap starting.
I20260709 12:30:35.135071  8121 tablet_bootstrap.cc:492] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:35.141438  8119 tablet_bootstrap.cc:654] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.142681  8121 tablet_bootstrap.cc:654] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.143136  8126 raft_consensus.cc:493] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.143675  8126 raft_consensus.cc:515] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.145989  8126 leader_election.cc:290] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.146936  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.147203  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:35.147650  7825 raft_consensus.cc:2468] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:35.147859  7975 raft_consensus.cc:2468] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:35.149056  7864 leader_election.cc:304] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.150451  8126 raft_consensus.cc:2804] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.151083  8126 raft_consensus.cc:493] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.151549  8126 raft_consensus.cc:3060] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.161871  8119 tablet_bootstrap.cc:492] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: No bootstrap required, opened a new log
I20260709 12:30:35.162465  8119 ts_tablet_manager.cc:1403] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.001s
I20260709 12:30:35.160164  8126 raft_consensus.cc:515] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.165248  8119 raft_consensus.cc:359] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.165802  8119 raft_consensus.cc:385] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.166036  8119 raft_consensus.cc:740] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.166766  8119 consensus_queue.cc:260] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.167496  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.168121  7825 raft_consensus.cc:3060] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.168283  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.168929  7975 raft_consensus.cc:3060] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.168944  8119 ts_tablet_manager.cc:1434] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:30:35.171599  8121 tablet_bootstrap.cc:492] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:35.172210  8121 ts_tablet_manager.cc:1403] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.037s	user 0.005s	sys 0.014s
I20260709 12:30:35.174122  7825 raft_consensus.cc:2468] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:35.175448  7864 leader_election.cc:304] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.175599  8121 raft_consensus.cc:359] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.176473  8144 raft_consensus.cc:2804] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.176617  8126 leader_election.cc:290] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.177055  7975 raft_consensus.cc:2468] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 1.
I20260709 12:30:35.177066  8144 raft_consensus.cc:697] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:35.176503  8121 raft_consensus.cc:385] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.178916  8121 raft_consensus.cc:740] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.178668  8144 consensus_queue.cc:237] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.179718  8121 consensus_queue.cc:260] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.181950  8121 ts_tablet_manager.cc:1434] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:30:35.182906  8121 tablet_bootstrap.cc:492] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:35.188692  8121 tablet_bootstrap.cc:654] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.188498  7722 catalog_manager.cc:5697] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 1 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.193332  8121 tablet_bootstrap.cc:492] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:35.193794  8121 ts_tablet_manager.cc:1403] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 12:30:35.196758  8107 raft_consensus.cc:493] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.196625  8121 raft_consensus.cc:359] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.197412  8121 raft_consensus.cc:385] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.197726  8121 raft_consensus.cc:740] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.197496  8107 raft_consensus.cc:515] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.198408  8121 consensus_queue.cc:260] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.199716  8107 leader_election.cc:290] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.200280  8121 ts_tablet_manager.cc:1434] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:30:35.200946  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:35.201283  8121 tablet_bootstrap.cc:492] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: Bootstrap starting.
I20260709 12:30:35.201304  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:35.201622  7975 raft_consensus.cc:2468] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 0.
I20260709 12:30:35.202103  7899 raft_consensus.cc:2468] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 0.
I20260709 12:30:35.202939  7790 leader_election.cc:304] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.204116  8107 raft_consensus.cc:2804] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.204562  8107 raft_consensus.cc:493] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.204870  8107 raft_consensus.cc:3060] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.207787  8121 tablet_bootstrap.cc:654] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:35.212438  8107 raft_consensus.cc:515] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.215283  8121 tablet_bootstrap.cc:492] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: No bootstrap required, opened a new log
I20260709 12:30:35.215457  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.215866  8121 ts_tablet_manager.cc:1403] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 12:30:35.216143  7899 raft_consensus.cc:3060] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.218808  8121 raft_consensus.cc:359] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.219426  8121 raft_consensus.cc:385] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:35.219698  8121 raft_consensus.cc:740] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:35.220409  8121 consensus_queue.cc:260] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.220803  8107 leader_election.cc:290] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.221535  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.222174  7975 raft_consensus.cc:3060] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.222536  8121 ts_tablet_manager.cc:1434] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:30:35.223500  8084 raft_consensus.cc:493] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.223585  7899 raft_consensus.cc:2468] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:35.224395  8084 raft_consensus.cc:515] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.225103  7791 leader_election.cc:304] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.225965  8107 raft_consensus.cc:2804] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.226578  8107 raft_consensus.cc:697] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:35.227506  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.228185  7825 raft_consensus.cc:2468] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:35.227937  8107 consensus_queue.cc:237] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.229511  7938 leader_election.cc:304] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.230427  8139 raft_consensus.cc:2804] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.230787  8139 raft_consensus.cc:493] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.231187  8139 raft_consensus.cc:3060] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.234138  7975 raft_consensus.cc:2468] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 1.
I20260709 12:30:35.239789  8139 raft_consensus.cc:515] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.241243  8084 leader_election.cc:290] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.241899  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:35.241503  7722 catalog_manager.cc:5697] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 1 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.242579  7899 raft_consensus.cc:2468] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:35.242725  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.243359  7825 raft_consensus.cc:3060] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.248822  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.249538  7899 raft_consensus.cc:3060] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.250049  7825 raft_consensus.cc:2468] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.250821  8139 leader_election.cc:290] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.251605  7938 leader_election.cc:304] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.253470  8139 raft_consensus.cc:2804] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.254703  8139 raft_consensus.cc:697] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.255496  8139 consensus_queue.cc:237] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.256655  7899 raft_consensus.cc:2468] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.263108  7722 catalog_manager.cc:5697] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:35.263599  8139 raft_consensus.cc:493] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.264209  8139 raft_consensus.cc:515] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.266095  8139 leader_election.cc:290] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.266754  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0703408f26944656ace444a76f3b74ac" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.267158  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0703408f26944656ace444a76f3b74ac" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:35.267488  7825 raft_consensus.cc:2468] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:35.267869  7899 raft_consensus.cc:2468] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:35.268548  7938 leader_election.cc:304] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.269404  8139 raft_consensus.cc:2804] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.269731  8139 raft_consensus.cc:493] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.270033  8139 raft_consensus.cc:3060] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.275679  8139 raft_consensus.cc:515] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.277449  8139 leader_election.cc:290] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.278226  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0703408f26944656ace444a76f3b74ac" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.278486  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0703408f26944656ace444a76f3b74ac" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.278908  7825 raft_consensus.cc:3060] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.279150  7899 raft_consensus.cc:3060] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.280954  8107 consensus_queue.cc:1048] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.285096  7825 raft_consensus.cc:2468] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.286365  7899 raft_consensus.cc:2468] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.286620  7938 leader_election.cc:304] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.287588  8139 raft_consensus.cc:2804] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.288462  8139 raft_consensus.cc:697] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.289297  8139 consensus_queue.cc:237] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.293797  8106 raft_consensus.cc:493] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.294400  8106 raft_consensus.cc:515] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.296676  8106 leader_election.cc:290] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.297358  8084 raft_consensus.cc:493] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.298175  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.298141  8084 raft_consensus.cc:515] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.299201  7824 raft_consensus.cc:2468] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:35.301120  8084 leader_election.cc:290] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.301108  7864 leader_election.cc:304] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.303247  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.304189  7824 raft_consensus.cc:2468] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:35.305141  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:35.303288  8144 raft_consensus.cc:2804] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.307716  8144 raft_consensus.cc:493] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.308053  8144 raft_consensus.cc:3060] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.317042  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" is_pre_election: true
I20260709 12:30:35.317359  8084 raft_consensus.cc:493] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:35.317809  7975 raft_consensus.cc:2468] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 0.
I20260709 12:30:35.317952  8084 raft_consensus.cc:515] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.320220  8084 leader_election.cc:290] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.324290  8144 raft_consensus.cc:515] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.326077  7899 raft_consensus.cc:2393] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in current term 1: Already voted for candidate 9369944381df4d7a85ab65dba5feb6e3 in this term.
I20260709 12:30:35.330191  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a600fd77b0ad4ae6926bc35630c98605" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3" is_pre_election: true
I20260709 12:30:35.331183  7899 raft_consensus.cc:2468] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 0.
I20260709 12:30:35.320101  7938 leader_election.cc:304] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.332996  8084 raft_consensus.cc:2804] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.333389  8084 raft_consensus.cc:493] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.333753  8084 raft_consensus.cc:3060] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.334443  7940 leader_election.cc:304] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:35.338344  8139 raft_consensus.cc:2804] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:35.338733  8139 raft_consensus.cc:493] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:35.339073  8139 raft_consensus.cc:3060] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.345950  8084 raft_consensus.cc:515] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.347211  8139 raft_consensus.cc:515] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.346757  7722 catalog_manager.cc:5697] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:35.349439  7901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.350334  7901 raft_consensus.cc:2393] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in current term 1: Already voted for candidate 9369944381df4d7a85ab65dba5feb6e3 in this term.
I20260709 12:30:35.350557  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a600fd77b0ad4ae6926bc35630c98605" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.351262  7899 raft_consensus.cc:3060] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.353967  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.354866  7824 raft_consensus.cc:3060] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.356361  8084 leader_election.cc:290] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.356762  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a600fd77b0ad4ae6926bc35630c98605" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.357492  8144 leader_election.cc:290] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.357865  7826 raft_consensus.cc:3060] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:35.359045  8139 leader_election.cc:290] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:35.360004  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a600fd77b0ad4ae6926bc35630c98605" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030" is_pre_election: true
I20260709 12:30:35.360424  7823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.364549  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.365800  7974 raft_consensus.cc:2393] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9369944381df4d7a85ab65dba5feb6e3 in current term 1: Already voted for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in this term.
I20260709 12:30:35.369828  7824 raft_consensus.cc:2468] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.370682  7899 raft_consensus.cc:2468] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.370858  8123 consensus_queue.cc:1048] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:30:35.374316  7940 leader_election.cc:304] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:35.375334  8139 raft_consensus.cc:2804] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.376842  7865 leader_election.cc:304] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3; no voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030
I20260709 12:30:35.377185  8139 raft_consensus.cc:697] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.375339  7826 raft_consensus.cc:2468] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 1.
I20260709 12:30:35.378188  8139 consensus_queue.cc:237] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.381994  8144 raft_consensus.cc:2749] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:35.383647  7938 leader_election.cc:304] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:35.385933  8139 raft_consensus.cc:2804] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:35.386699  8139 raft_consensus.cc:697] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.387619  8139 consensus_queue.cc:237] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.388635  7724 catalog_manager.cc:5697] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:35.404407  7724 catalog_manager.cc:5697] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 1 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:35.535408  8139 consensus_queue.cc:1048] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.551712  8169 consensus_queue.cc:1048] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:35.596848  8144 consensus_queue.cc:1048] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.607978  8106 consensus_queue.cc:1048] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.615089  7975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:35.615697  7975 raft_consensus.cc:606] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:35.622653  7899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:41622
I20260709 12:30:35.623494  7899 raft_consensus.cc:606] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:35.629053  7974 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "054091d9725042bcbc077f81bf42312f"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:35.629591  7974 raft_consensus.cc:606] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS ffae259334c845d795b4dde932584030
I20260709 12:30:35.637480  7974 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0703408f26944656ace444a76f3b74ac"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:35.638192  7974 raft_consensus.cc:606] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:35.645454  7826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5"
dest_uuid: "ffae259334c845d795b4dde932584030"
mode: GRACEFUL
new_leader_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:55042
I20260709 12:30:35.646049  7826 raft_consensus.cc:606] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Received request to transfer leadership to TS 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:35.651805  7974 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
mode: GRACEFUL
new_leader_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:30:35.652539  7974 raft_consensus.cc:606] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Received request to transfer leadership to TS 9369944381df4d7a85ab65dba5feb6e3
I20260709 12:30:35.699534  8144 consensus_queue.cc:1048] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.710068  8126 consensus_queue.cc:1048] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.718222  8126 raft_consensus.cc:993] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:35.718654  8172 raft_consensus.cc:1081] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:35.720294  7824 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:30:35.720804  7824 raft_consensus.cc:493] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:35.721099  7824 raft_consensus.cc:3060] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.729292  7824 raft_consensus.cc:515] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.729658  8169 consensus_queue.cc:1048] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.731563  7824 leader_election.cc:290] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.732216  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.732926  7899 raft_consensus.cc:3055] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:35.732831  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcb087f3a694ba090b2ea713d4f9efd" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.733467  7899 raft_consensus.cc:740] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:35.733693  7974 raft_consensus.cc:3060] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.734310  7899 consensus_queue.cc:260] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.735237  7899 raft_consensus.cc:3060] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.741680  7974 raft_consensus.cc:2468] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:35.741725  7899 raft_consensus.cc:2468] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:35.743052  7790 leader_election.cc:304] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.743935  8129 raft_consensus.cc:2804] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:35.744539  8129 raft_consensus.cc:697] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:35.745561  8129 consensus_queue.cc:237] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.753603  8139 consensus_queue.cc:1048] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.755367  8129 consensus_queue.cc:1048] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.758594  7722 catalog_manager.cc:5697] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.767586  8139 raft_consensus.cc:993] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:35.768126  8139 raft_consensus.cc:1081] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:35.771382  7900 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:41610
I20260709 12:30:35.772032  7900 raft_consensus.cc:493] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:35.772517  7900 raft_consensus.cc:3060] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.772953  8139 consensus_queue.cc:1048] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:35.782158  8123 consensus_queue.cc:1048] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.786859  7900 raft_consensus.cc:515] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.790820  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.791482  7824 raft_consensus.cc:3060] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.794437  8138 consensus_queue.cc:1048] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:35.802337  7900 leader_election.cc:290] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.803323  7824 raft_consensus.cc:2468] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:35.803946  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2b6444c31ca4468b202ef77a8982cee" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.804569  7975 raft_consensus.cc:3055] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:35.804888  7975 raft_consensus.cc:740] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.804593  7864 leader_election.cc:304] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.806202  8107 raft_consensus.cc:993] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: : Instructing follower 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:35.806403  8172 raft_consensus.cc:2804] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:35.806753  8107 raft_consensus.cc:1081] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Signalling peer 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:35.807098  8172 raft_consensus.cc:697] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:35.808071  8172 consensus_queue.cc:237] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.809034  7901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:41616
I20260709 12:30:35.809799  7901 raft_consensus.cc:493] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:35.810425  7901 raft_consensus.cc:3060] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.806301  7975 consensus_queue.cc:260] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.811684  7975 raft_consensus.cc:3060] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.820914  7901 raft_consensus.cc:515] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.825783  7826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.826045  7901 leader_election.cc:290] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.826471  7826 raft_consensus.cc:3055] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:35.826849  7826 raft_consensus.cc:740] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:35.827677  7974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627ca9faca0a4c84b4d440484f6367c5" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.828439  7974 raft_consensus.cc:3060] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.827854  7826 consensus_queue.cc:260] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.829456  7724 catalog_manager.cc:5697] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.830546  7975 raft_consensus.cc:2468] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:35.829530  7826 raft_consensus.cc:3060] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.838794  7974 raft_consensus.cc:2468] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:35.840095  7865 leader_election.cc:304] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:35.840962  8172 raft_consensus.cc:2804] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:35.841488  8172 raft_consensus.cc:697] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:35.842296  8172 consensus_queue.cc:237] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.842607  8138 consensus_queue.cc:1048] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:35.846459  7826 raft_consensus.cc:2468] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:35.850911  8169 raft_consensus.cc:993] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:35.851418  8169 raft_consensus.cc:1081] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:35.851747  7720 catalog_manager.cc:5697] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 1 to 2, leader changed from ffae259334c845d795b4dde932584030 (127.24.71.129) to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.853405  7826 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "054091d9725042bcbc077f81bf42312f"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55028
I20260709 12:30:35.854085  7826 raft_consensus.cc:493] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:35.854472  7826 raft_consensus.cc:3060] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.859983  8084 consensus_queue.cc:1048] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:35.862174  7826 raft_consensus.cc:515] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.864868  7826 leader_election.cc:290] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.865940  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.866591  7975 raft_consensus.cc:3055] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:35.867033  7975 raft_consensus.cc:740] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.868189  7975 consensus_queue.cc:260] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.868745  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054091d9725042bcbc077f81bf42312f" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:35.869190  7975 raft_consensus.cc:3060] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.869417  7900 raft_consensus.cc:3060] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.869541  7824 raft_consensus.cc:1240] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Rejecting Update request from peer 91b980f5b5d04b6a9e5d4fa2af0c5807 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:35.871186  8139 consensus_queue.cc:1048] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:35.877594  7900 raft_consensus.cc:2468] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:35.877727  7975 raft_consensus.cc:2468] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:35.878489  7901 raft_consensus.cc:1240] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Rejecting Update request from peer 91b980f5b5d04b6a9e5d4fa2af0c5807 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:30:35.879015  7791 leader_election.cc:304] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.879967  8123 raft_consensus.cc:2804] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:35.880471  8123 raft_consensus.cc:697] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:35.881337  8123 consensus_queue.cc:237] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.896998  7722 catalog_manager.cc:5697] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:35.899282  8139 consensus_queue.cc:1048] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:35.910544  8084 raft_consensus.cc:993] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:35.911095  8139 raft_consensus.cc:1081] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer 9369944381df4d7a85ab65dba5feb6e3 to start an election
I20260709 12:30:35.912859  7900 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0703408f26944656ace444a76f3b74ac"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
 from {username='slave'} at 127.0.0.1:41610
I20260709 12:30:35.913385  7900 raft_consensus.cc:493] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:35.913772  7900 raft_consensus.cc:3060] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.919047  7900 raft_consensus.cc:515] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.920817  7900 leader_election.cc:290] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:35.921663  7824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0703408f26944656ace444a76f3b74ac" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:35.922034  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0703408f26944656ace444a76f3b74ac" candidate_uuid: "9369944381df4d7a85ab65dba5feb6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:35.922370  7824 raft_consensus.cc:3060] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.922627  7975 raft_consensus.cc:3055] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:35.922963  7975 raft_consensus.cc:740] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:35.923635  7975 consensus_queue.cc:260] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.924517  7975 raft_consensus.cc:3060] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:35.928166  7824 raft_consensus.cc:2468] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:35.929557  7864 leader_election.cc:304] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:35.930307  8172 raft_consensus.cc:2804] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:35.930758  8172 raft_consensus.cc:697] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:35.931463  7975 raft_consensus.cc:2468] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9369944381df4d7a85ab65dba5feb6e3 in term 2.
I20260709 12:30:35.931604  8172 consensus_queue.cc:237] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:35.938851  7724 catalog_manager.cc:5697] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130). New cstate: current_term: 2 leader_uuid: "9369944381df4d7a85ab65dba5feb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:36.028602  8169 raft_consensus.cc:993] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: : Instructing follower ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:36.029095  8139 raft_consensus.cc:1081] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Signalling peer ffae259334c845d795b4dde932584030 to start an election
I20260709 12:30:36.030714  7824 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2"
dest_uuid: "ffae259334c845d795b4dde932584030"
 from {username='slave'} at 127.0.0.1:55028
I20260709 12:30:36.031224  7824 raft_consensus.cc:493] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:36.031562  7824 raft_consensus.cc:3060] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:36.037778  7824 raft_consensus.cc:515] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:36.039474  7824 leader_election.cc:290] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Requested vote from peers 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469), 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131:38977)
I20260709 12:30:36.040443  7900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:36.040566  7975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b4c722d8e8495da36c89350e29bcb2" candidate_uuid: "ffae259334c845d795b4dde932584030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
I20260709 12:30:36.041141  7900 raft_consensus.cc:3060] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:36.041255  7975 raft_consensus.cc:3055] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:36.041678  7975 raft_consensus.cc:740] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:36.042363  7975 consensus_queue.cc:260] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:36.043251  7975 raft_consensus.cc:3060] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:36.046412  7900 raft_consensus.cc:2468] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:36.047737  7791 leader_election.cc:304] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9369944381df4d7a85ab65dba5feb6e3, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:36.048446  7975 raft_consensus.cc:2468] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffae259334c845d795b4dde932584030 in term 2.
I20260709 12:30:36.048774  8123 raft_consensus.cc:2804] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:36.049384  8123 raft_consensus.cc:697] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Leader. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:36.050334  8123 consensus_queue.cc:237] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:36.057917  7724 catalog_manager.cc:5697] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 reported cstate change: term changed from 1 to 2, leader changed from 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131) to ffae259334c845d795b4dde932584030 (127.24.71.129). New cstate: current_term: 2 leader_uuid: "ffae259334c845d795b4dde932584030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:36.148833  7975 raft_consensus.cc:1275] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.150161  8123 consensus_queue.cc:1048] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:36.158200  7900 raft_consensus.cc:1275] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.160995  8123 consensus_queue.cc:1048] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.267637  7975 raft_consensus.cc:1275] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.268958  8106 consensus_queue.cc:1048] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.279981  7824 raft_consensus.cc:1275] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.281648  8126 consensus_queue.cc:1048] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.291815  7974 raft_consensus.cc:1275] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.293589  8123 consensus_queue.cc:1048] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.305569  7900 raft_consensus.cc:1275] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.307403  8123 consensus_queue.cc:1048] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:36.321326  7824 raft_consensus.cc:1275] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.323139  8126 consensus_queue.cc:1048] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.334225  7975 raft_consensus.cc:1275] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.335843  8106 consensus_queue.cc:1048] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:36.426784  7824 raft_consensus.cc:1275] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.428375  8106 consensus_queue.cc:1048] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.438009  7974 raft_consensus.cc:1275] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer 9369944381df4d7a85ab65dba5feb6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.445062  8126 consensus_queue.cc:1048] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.586413  7900 raft_consensus.cc:1275] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.587770  8123 consensus_queue.cc:1048] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:36.596880  7974 raft_consensus.cc:1275] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Refusing update from remote peer ffae259334c845d795b4dde932584030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:36.598523  8129 consensus_queue.cc:1048] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:38.659125  7724 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38142:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:30:38.682611  7724 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38142:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:30:38.715166 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:30:38.842834  8220 log.cc:927] Time spent T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 12:30:38.878446  8238 tablet.cc:1724] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:38.903882  8226 log.cc:927] Time spent T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:30:38.883311  8231 tablet.cc:1724] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:38.918807  8239 tablet.cc:1724] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:38.883967  8232 tablet.cc:1724] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:38.920120  8234 tablet.cc:1724] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:38.996660  7864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:38171, user_credentials={real_user=slave}} blocked reactor thread for 62910.4us
I20260709 12:30:39.029220  8248 tablet.cc:1724] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:39.030053  8233 log.cc:927] Time spent T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:30:39.036340  8241 tablet.cc:1724] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:39.083112  7938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.71.129:38171, user_credentials={real_user=slave}} blocked reactor thread for 45910.3us
I20260709 12:30:39.016332  8249 tablet.cc:1724] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.005488  8243 tablet.cc:1724] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.092661  8251 tablet.cc:1724] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.151036  8239 tablet.cc:1724] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.152295  8231 tablet.cc:1724] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.162597  7899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8123af90ae414fb78560153531759233"
dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
mode: GRACEFUL
new_leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
 from {username='slave'} at 127.0.0.1:45376
I20260709 12:30:39.163604  7899 raft_consensus.cc:606] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Received request to transfer leadership to TS 91b980f5b5d04b6a9e5d4fa2af0c5807
I20260709 12:30:39.168385  8253 tablet.cc:1724] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.206777  8251 tablet.cc:1724] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.210124  8263 tablet.cc:1724] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.218708  8232 tablet.cc:1724] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.226776  8251 tablet.cc:1724] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.232211  8234 tablet.cc:1724] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.236043  8238 tablet.cc:1724] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.236603  8249 tablet.cc:1724] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.243113  8241 tablet.cc:1724] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.254314  8232 tablet.cc:1724] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.251863  8253 tablet.cc:1724] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.255528  8126 raft_consensus.cc:993] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: : Instructing follower 91b980f5b5d04b6a9e5d4fa2af0c5807 to start an election
I20260709 12:30:39.257045  8126 raft_consensus.cc:1081] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Signalling peer 91b980f5b5d04b6a9e5d4fa2af0c5807 to start an election
I20260709 12:30:39.247044  8248 tablet.cc:1724] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.263200  8239 tablet.cc:1724] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.279489  8243 tablet.cc:1724] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.282312  7825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77"
dest_uuid: "ffae259334c845d795b4dde932584030"
mode: GRACEFUL
new_leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
 from {username='slave'} at 127.0.0.1:42432
I20260709 12:30:39.283007  7825 raft_consensus.cc:606] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Received request to transfer leadership to TS 91b980f5b5d04b6a9e5d4fa2af0c5807
I20260709 12:30:39.284516 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:39.285187 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:39.298893  7974 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8123af90ae414fb78560153531759233"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
 from {username='slave'} at 127.0.0.1:42916
I20260709 12:30:39.299611  7974 raft_consensus.cc:493] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:39.300011  7974 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:39.310832  7974 raft_consensus.cc:515] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:39.312263  8234 tablet.cc:1724] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:39.313723  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:39.314393  7899 raft_consensus.cc:3055] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:39.314729  7899 raft_consensus.cc:740] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9369944381df4d7a85ab65dba5feb6e3, State: Running, Role: LEADER
I20260709 12:30:39.315470  7899 consensus_queue.cc:260] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:39.316442  7899 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:39.323916  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8123af90ae414fb78560153531759233" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:39.324734  7825 raft_consensus.cc:3060] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:39.332221  7825 raft_consensus.cc:2468] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 3.
I20260709 12:30:39.333110  7899 raft_consensus.cc:2468] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 3.
I20260709 12:30:39.333521  7938 leader_election.cc:304] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, ffae259334c845d795b4dde932584030; no voters: 
I20260709 12:30:39.334105  7974 leader_election.cc:290] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 3 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:39.334339  8084 raft_consensus.cc:2804] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:39.334981  8084 raft_consensus.cc:697] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:39.335759  8084 consensus_queue.cc:237] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:39.354689  7720 catalog_manager.cc:6242] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb: soft_deleted_table [id=e780332f365149cc864df0ca98836c0e] alter complete (version 1)
I20260709 12:30:39.364776  7720 catalog_manager.cc:5697] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 2 to 3, leader changed from 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130) to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:39.648934  8186 raft_consensus.cc:993] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: : Instructing follower 91b980f5b5d04b6a9e5d4fa2af0c5807 to start an election
I20260709 12:30:39.649632  8245 raft_consensus.cc:1081] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Signalling peer 91b980f5b5d04b6a9e5d4fa2af0c5807 to start an election
I20260709 12:30:39.651644  7974 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77"
dest_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807"
 from {username='slave'} at 127.0.0.1:42918
I20260709 12:30:39.652275  7974 raft_consensus.cc:493] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:39.652653  7974 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:39.659442  7974 raft_consensus.cc:515] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:39.661756  7974 leader_election.cc:290] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 3 election: Requested vote from peers ffae259334c845d795b4dde932584030 (127.24.71.129:38171), 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469)
I20260709 12:30:39.662011  7825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ffae259334c845d795b4dde932584030"
I20260709 12:30:39.662683  7825 raft_consensus.cc:3055] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:39.662788  7899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ef5286c8054f3b93d957e5e4cf1d77" candidate_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9369944381df4d7a85ab65dba5feb6e3"
I20260709 12:30:39.663034  7825 raft_consensus.cc:740] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ffae259334c845d795b4dde932584030, State: Running, Role: LEADER
I20260709 12:30:39.663416  7899 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:39.663787  7825 consensus_queue.cc:260] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:39.665064  7825 raft_consensus.cc:3060] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:39.670387  7899 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 3.
I20260709 12:30:39.671672  7825 raft_consensus.cc:2468] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b980f5b5d04b6a9e5d4fa2af0c5807 in term 3.
I20260709 12:30:39.671955  7940 leader_election.cc:304] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b980f5b5d04b6a9e5d4fa2af0c5807, 9369944381df4d7a85ab65dba5feb6e3; no voters: 
I20260709 12:30:39.672988  8084 raft_consensus.cc:2804] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:39.673508  8084 raft_consensus.cc:697] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 LEADER]: Becoming Leader. State: Replica: 91b980f5b5d04b6a9e5d4fa2af0c5807, State: Running, Role: LEADER
I20260709 12:30:39.674324  8084 consensus_queue.cc:237] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } }
I20260709 12:30:39.682530  7720 catalog_manager.cc:5697] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 reported cstate change: term changed from 2 to 3, leader changed from ffae259334c845d795b4dde932584030 (127.24.71.129) to 91b980f5b5d04b6a9e5d4fa2af0c5807 (127.24.71.131). New cstate: current_term: 3 leader_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b980f5b5d04b6a9e5d4fa2af0c5807" member_type: VOTER last_known_addr { host: "127.24.71.131" port: 38977 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:39.734573  7899 raft_consensus.cc:1275] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Refusing update from remote peer 91b980f5b5d04b6a9e5d4fa2af0c5807: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:39.735921  8084 consensus_queue.cc:1048] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:39.746578  7825 raft_consensus.cc:1275] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Refusing update from remote peer 91b980f5b5d04b6a9e5d4fa2af0c5807: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:39.748660  8262 consensus_queue.cc:1048] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:40.117717  7899 raft_consensus.cc:1275] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Refusing update from remote peer 91b980f5b5d04b6a9e5d4fa2af0c5807: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:40.119166  8262 consensus_queue.cc:1048] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9369944381df4d7a85ab65dba5feb6e3" member_type: VOTER last_known_addr { host: "127.24.71.130" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:40.129760  7826 raft_consensus.cc:1275] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Refusing update from remote peer 91b980f5b5d04b6a9e5d4fa2af0c5807: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:40.131189  8262 consensus_queue.cc:1048] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffae259334c845d795b4dde932584030" member_type: VOTER last_known_addr { host: "127.24.71.129" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:40.286437 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:40.291778 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:40.292320 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:41.293745 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:41.299743 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:41.300307 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:42.301661 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:42.307287 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:42.307798 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:43.308883 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:43.314815 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:43.315718 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:44.316849 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:44.322062 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:44.322551 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:45.323616 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:45.329698 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:45.330235 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:46.331311 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:46.337256 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:46.337850 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:47.338970 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:47.345532 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:47.346143 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:48.347400 24862 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:48.354081 24862 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:48.354497 24862 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:49.424747 24862 tablet_server.cc:179] TabletServer@127.24.71.129:0 shutting down...
W20260709 12:30:49.458791  7938 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
W20260709 12:30:49.469620  7938 consensus_peers.cc:597] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:49.493604  7864 consensus_peers.cc:597] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:49.495244 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:49.496418 24862 tablet_replica.cc:333] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.497949 24862 raft_consensus.cc:2243] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.498791 24862 raft_consensus.cc:2272] T a600fd77b0ad4ae6926bc35630c98605 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.501693 24862 tablet_replica.cc:333] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.502346 24862 raft_consensus.cc:2243] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.502911 24862 raft_consensus.cc:2272] T 0703408f26944656ace444a76f3b74ac P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.506600 24862 tablet_replica.cc:333] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030: stopping tablet replica
W20260709 12:30:49.507485  7938 consensus_peers.cc:597] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:49.507536 24862 raft_consensus.cc:2243] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.508742 24862 raft_consensus.cc:2272] T 627ca9faca0a4c84b4d440484f6367c5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.511580 24862 tablet_replica.cc:333] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.512374 24862 raft_consensus.cc:2243] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.513418 24862 raft_consensus.cc:2272] T 054091d9725042bcbc077f81bf42312f P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.516515 24862 tablet_replica.cc:333] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.517246 24862 raft_consensus.cc:2243] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.518031 24862 raft_consensus.cc:2272] T a2b6444c31ca4468b202ef77a8982cee P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.521276 24862 tablet_replica.cc:333] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.521976 24862 raft_consensus.cc:2243] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.522980 24862 raft_consensus.cc:2272] T 64b4c722d8e8495da36c89350e29bcb2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.525789 24862 tablet_replica.cc:333] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.526464 24862 raft_consensus.cc:2243] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.527400 24862 raft_consensus.cc:2272] T 7974d124c88843479ba5624cb6ad89a7 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.530107 24862 tablet_replica.cc:333] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.530699 24862 raft_consensus.cc:2243] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.531265 24862 raft_consensus.cc:2272] T 104a1bd3387c412d9566d4fb6a9e5ad0 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.533735 24862 tablet_replica.cc:333] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.534497 24862 raft_consensus.cc:2243] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.535096 24862 raft_consensus.cc:2272] T d9286dadba914ce6aab4563d5ae867a2 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.537739 24862 tablet_replica.cc:333] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.538372 24862 raft_consensus.cc:2243] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.539402 24862 raft_consensus.cc:2272] T 872a7630d8ef4e1593c182c019b311e5 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.542026 24862 tablet_replica.cc:333] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.542692 24862 raft_consensus.cc:2243] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.543335 24862 raft_consensus.cc:2272] T 3b1b1e5503324ec29c0eb2432e28a4dc P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.546123 24862 tablet_replica.cc:333] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.546746 24862 raft_consensus.cc:2243] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.547806 24862 raft_consensus.cc:2272] T 2729076d19a1445e893ae62f628fed91 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.550683 24862 tablet_replica.cc:333] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.551746 24862 raft_consensus.cc:2243] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.552838 24862 raft_consensus.cc:2272] T 7bcb087f3a694ba090b2ea713d4f9efd P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.555370 24862 tablet_replica.cc:333] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.556128 24862 raft_consensus.cc:2243] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.556795 24862 raft_consensus.cc:2272] T 6a1a7c43ee5347a9aaed1533de6c6a66 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.560343 24862 tablet_replica.cc:333] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.561338 24862 raft_consensus.cc:2243] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.561976 24862 raft_consensus.cc:2272] T 8123af90ae414fb78560153531759233 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.564577 24862 tablet_replica.cc:333] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.565096 24862 raft_consensus.cc:2243] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.566037 24862 raft_consensus.cc:2272] T 292d0c1d1f9943b89c71643750b0a8b9 P ffae259334c845d795b4dde932584030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.568554 24862 tablet_replica.cc:333] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.569034 24862 raft_consensus.cc:2243] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.569480 24862 raft_consensus.cc:2272] T cc9ab233018d40eb90109978c12e9201 P ffae259334c845d795b4dde932584030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.571839 24862 tablet_replica.cc:333] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030: stopping tablet replica
I20260709 12:30:49.572654 24862 raft_consensus.cc:2243] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.573158 24862 raft_consensus.cc:2272] T b6ef5286c8054f3b93d957e5e4cf1d77 P ffae259334c845d795b4dde932584030 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:49.585641  7864 consensus_peers.cc:597] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:49.589408  7864 consensus_peers.cc:597] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:49.600637  7938 consensus_peers.cc:597] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:49.602196  7864 consensus_peers.cc:597] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:49.619320 24862 tablet_server.cc:196] TabletServer@127.24.71.129:0 shutdown complete.
W20260709 12:30:49.620822  7864 consensus_peers.cc:597] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:38171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:49.656301  7864 consensus_peers.cc:597] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:38171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:49.675807 24862 tablet_server.cc:179] TabletServer@127.24.71.130:0 shutting down...
I20260709 12:30:49.714206 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:49.714962 24862 tablet_replica.cc:333] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.716223 24862 raft_consensus.cc:2243] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.717003 24862 raft_consensus.cc:2272] T a600fd77b0ad4ae6926bc35630c98605 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.719410 24862 tablet_replica.cc:333] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.719980 24862 raft_consensus.cc:2243] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.720860 24862 raft_consensus.cc:2272] T 0703408f26944656ace444a76f3b74ac P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.723508 24862 tablet_replica.cc:333] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.724057 24862 raft_consensus.cc:2243] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.724619 24862 raft_consensus.cc:2272] T 054091d9725042bcbc077f81bf42312f P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.726809 24862 tablet_replica.cc:333] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.727342 24862 raft_consensus.cc:2243] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.728462 24862 raft_consensus.cc:2272] T a2b6444c31ca4468b202ef77a8982cee P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.731010 24862 tablet_replica.cc:333] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.731632 24862 raft_consensus.cc:2243] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.732158 24862 raft_consensus.cc:2272] T 64b4c722d8e8495da36c89350e29bcb2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.734555 24862 tablet_replica.cc:333] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.735141 24862 raft_consensus.cc:2243] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.735613 24862 raft_consensus.cc:2272] T 872a7630d8ef4e1593c182c019b311e5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.737893 24862 tablet_replica.cc:333] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.738502 24862 raft_consensus.cc:2243] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.739006 24862 raft_consensus.cc:2272] T 8123af90ae414fb78560153531759233 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.741335 24862 tablet_replica.cc:333] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.741902 24862 raft_consensus.cc:2243] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.742733 24862 raft_consensus.cc:2272] T d9286dadba914ce6aab4563d5ae867a2 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.745229 24862 tablet_replica.cc:333] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.745781 24862 raft_consensus.cc:2243] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.746853 24862 raft_consensus.cc:2272] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.749853 24862 tablet_replica.cc:333] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.750452 24862 raft_consensus.cc:2243] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.750980 24862 raft_consensus.cc:2272] T b6ef5286c8054f3b93d957e5e4cf1d77 P 9369944381df4d7a85ab65dba5feb6e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.753237 24862 tablet_replica.cc:333] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.753834 24862 raft_consensus.cc:2243] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.754324 24862 raft_consensus.cc:2272] T 292d0c1d1f9943b89c71643750b0a8b9 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.756654 24862 tablet_replica.cc:333] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.757239 24862 raft_consensus.cc:2243] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.757709 24862 raft_consensus.cc:2272] T cc9ab233018d40eb90109978c12e9201 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.760001 24862 tablet_replica.cc:333] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.760762 24862 raft_consensus.cc:2243] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.761511 24862 raft_consensus.cc:2272] T 7974d124c88843479ba5624cb6ad89a7 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.763736 24862 tablet_replica.cc:333] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.764283 24862 raft_consensus.cc:2243] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.765193 24862 raft_consensus.cc:2272] T 627ca9faca0a4c84b4d440484f6367c5 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.768365 24862 tablet_replica.cc:333] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.768924 24862 raft_consensus.cc:2243] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.769876 24862 raft_consensus.cc:2272] T 3b1b1e5503324ec29c0eb2432e28a4dc P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.772406 24862 tablet_replica.cc:333] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.772998 24862 raft_consensus.cc:2243] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.773545 24862 raft_consensus.cc:2272] T 2729076d19a1445e893ae62f628fed91 P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.775982 24862 tablet_replica.cc:333] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.776574 24862 raft_consensus.cc:2243] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.777086 24862 raft_consensus.cc:2272] T 7bcb087f3a694ba090b2ea713d4f9efd P 9369944381df4d7a85ab65dba5feb6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.779454 24862 tablet_replica.cc:333] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3: stopping tablet replica
I20260709 12:30:49.779973 24862 raft_consensus.cc:2243] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.780984 24862 raft_consensus.cc:2272] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 9369944381df4d7a85ab65dba5feb6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:49.801211  7938 consensus_peers.cc:597] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 -> Peer ffae259334c845d795b4dde932584030 (127.24.71.129:38171): Couldn't send request to peer ffae259334c845d795b4dde932584030. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.129:38171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:49.829883 24862 tablet_server.cc:196] TabletServer@127.24.71.130:0 shutdown complete.
W20260709 12:30:49.884249  7940 consensus_peers.cc:597] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 -> Peer 9369944381df4d7a85ab65dba5feb6e3 (127.24.71.130:36469): Couldn't send request to peer 9369944381df4d7a85ab65dba5feb6e3. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.130:36469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:49.886286 24862 tablet_server.cc:179] TabletServer@127.24.71.131:0 shutting down...
I20260709 12:30:49.916743 24862 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:49.917613 24862 tablet_replica.cc:333] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.918337 24862 raft_consensus.cc:2243] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.918918 24862 raft_consensus.cc:2272] T 627ca9faca0a4c84b4d440484f6367c5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.921444 24862 tablet_replica.cc:333] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.922119 24862 raft_consensus.cc:2243] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.923000 24862 raft_consensus.cc:2272] T a600fd77b0ad4ae6926bc35630c98605 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.925318 24862 tablet_replica.cc:333] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.925917 24862 raft_consensus.cc:2243] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.926436 24862 raft_consensus.cc:2272] T 054091d9725042bcbc077f81bf42312f P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.928808 24862 tablet_replica.cc:333] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.929391 24862 raft_consensus.cc:2243] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.929940 24862 raft_consensus.cc:2272] T 0703408f26944656ace444a76f3b74ac P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.932324 24862 tablet_replica.cc:333] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.932945 24862 raft_consensus.cc:2243] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.933446 24862 raft_consensus.cc:2272] T d9286dadba914ce6aab4563d5ae867a2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.935724 24862 tablet_replica.cc:333] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.936417 24862 raft_consensus.cc:2243] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.936941 24862 raft_consensus.cc:2272] T 104a1bd3387c412d9566d4fb6a9e5ad0 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.939126 24862 tablet_replica.cc:333] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.939780 24862 raft_consensus.cc:2243] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.940747 24862 raft_consensus.cc:2272] T 8123af90ae414fb78560153531759233 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.943413 24862 tablet_replica.cc:333] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.944288 24862 raft_consensus.cc:2243] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.944916 24862 raft_consensus.cc:2272] T 292d0c1d1f9943b89c71643750b0a8b9 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.947896 24862 tablet_replica.cc:333] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.948500 24862 raft_consensus.cc:2243] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.949426 24862 raft_consensus.cc:2272] T cc9ab233018d40eb90109978c12e9201 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.951766 24862 tablet_replica.cc:333] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.952351 24862 raft_consensus.cc:2243] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:49.953167 24862 raft_consensus.cc:2272] T b6ef5286c8054f3b93d957e5e4cf1d77 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.955329 24862 tablet_replica.cc:333] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.955863 24862 raft_consensus.cc:2243] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.956449 24862 raft_consensus.cc:2272] T 3b1b1e5503324ec29c0eb2432e28a4dc P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.958851 24862 tablet_replica.cc:333] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.959332 24862 raft_consensus.cc:2243] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.959805 24862 raft_consensus.cc:2272] T 2729076d19a1445e893ae62f628fed91 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.962260 24862 tablet_replica.cc:333] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.962771 24862 raft_consensus.cc:2243] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.963300 24862 raft_consensus.cc:2272] T 7bcb087f3a694ba090b2ea713d4f9efd P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.965590 24862 tablet_replica.cc:333] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.966049 24862 raft_consensus.cc:2243] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.966450 24862 raft_consensus.cc:2272] T 6a1a7c43ee5347a9aaed1533de6c6a66 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.968551 24862 tablet_replica.cc:333] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.969108 24862 raft_consensus.cc:2243] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.969605 24862 raft_consensus.cc:2272] T 872a7630d8ef4e1593c182c019b311e5 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.971848 24862 tablet_replica.cc:333] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.972430 24862 raft_consensus.cc:2243] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.973003 24862 raft_consensus.cc:2272] T a2b6444c31ca4468b202ef77a8982cee P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.975149 24862 tablet_replica.cc:333] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.975651 24862 raft_consensus.cc:2243] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.976166 24862 raft_consensus.cc:2272] T 64b4c722d8e8495da36c89350e29bcb2 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:49.978386 24862 tablet_replica.cc:333] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807: stopping tablet replica
I20260709 12:30:49.978878 24862 raft_consensus.cc:2243] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:49.979380 24862 raft_consensus.cc:2272] T 7974d124c88843479ba5624cb6ad89a7 P 91b980f5b5d04b6a9e5d4fa2af0c5807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:50.023115 24862 tablet_server.cc:196] TabletServer@127.24.71.131:0 shutdown complete.
I20260709 12:30:50.063077 24862 master.cc:562] Master@127.24.71.190:37433 shutting down...
I20260709 12:30:50.088135 24862 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:50.088881 24862 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:50.089244 24862 tablet_replica.cc:333] T 00000000000000000000000000000000 P e168a38330e14236bcb04bbd50b470bb: stopping tablet replica
I20260709 12:30:50.110021 24862 master.cc:584] Master@127.24.71.190:37433 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20874 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (44094 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (517637 ms total)
[  PASSED  ] 8 tests.
I20260709 12:30:50.175544 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2235 messages since previous log ~32 seconds ago
I20260709 12:30:50.175829 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2235 messages since previous log ~32 seconds ago
I20260709 12:30:50.176060 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2230 messages since previous log ~32 seconds ago
I20260709 12:30:50.176343 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~57 seconds ago
I20260709 12:30:50.176586 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~57 seconds ago
I20260709 12:30:50.176854 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 5 messages since previous log ~176 seconds ago
I20260709 12:30:50.177147 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 12:30:50.177524 24862 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
